|
After using them and building several of them, I still prefer building my parsers by hand.
The only reason I don't is error handling and maintenance, although maintenance with generated parsers is a double edged sword. For starters, it generates a lot more code than you would if you hand rolled a parser, typically. Second, with a hand rolled parser you can parse directly into your final tree. With a generated parser you must first parse into a parse tree, before converting *that* to your final tree, and so maintenance becomes an issue again because there's that second phase which is usually hand-written. Although to be fair, the data you're working with in the second phase has already been validated and error checked for syntax and structure.
I provide macros to help with the second phase, but it doesn't help that much. It would be nice if i could make a grammar description that could completely shape a tree, and create arbitrary objects off of it, to mimic the creation-into-the-final-tree that hand rolled parsers can do.
I don't like that the generated parser will take my codedomgokit source from 500KB to about 1.5-2MB
But I also don't like how sketchy my hand rolled parser is.
Is the extra 1+MB worth it? I guess that's probably 400-500k compiled based on previous experience. Currently CodeDOM Go Kit is about 250K compiled.
I don't know. It's something to think about. I really want a Better Way(TM) - something that handles that second phase and rolls it into the first.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
honey the codewitch writes: I still prefer building my parsers by hand.
I even built lexical code by hand, so no arguments from this 🦕.
With a generated parser you must first parse into a parse tree, before converting that to your final tree... 🤮
I provide macros to help with the second phase... Naughty! I'd deprecate macros with prejudice. No, not even deprecate: they'd just vanish overnight. I've only seen one instance where they were used elegantly: to generate assembly opcodes of all things. Maybe I've led a sheltered life. But I'll confess that I never "got" templates until I read a comment describing them as "macros on steroids".
But I also don't like how sketchy my hand rolled parser is. Would you like to borrow my rubber 🦆, who's a refactoring taskmaster? He promises to be gentle.
Quote: Is the extra 1+MB worth it?
I'd bet this is a problem with all non-trivial generated code.
|
|
|
|
|
I'm glad you read this post. I was thinking of you when I wrote it. I enjoy our conversations, as we have similar passions, it seems.
My "Macros" are actually type checked. After the code gets parsed and dumped into an AST then I go back and resolve macros and I do it in a type safe manner. They resolve to function calls. It isn't a simple preprocessor, so doesn't have a lot of those traditional drawbacks.
Sure, templates are "macros on steroids" but more importantly perhaps, they're *type checked* "macros".
I love templates in C++. Once you learn Generic Programming(TM) you never go back unless you have to.
By the way, you might want to consider using a lexer like flex or lex for your parser.
The reason is speed, not just ease.
If you learn how to use it (the learning curve isn't big, lex is stupid simple and mostly you write it in C/C++ code) - it will do things like shortest string discovery. It uses these optimizations to skip examining characters in the input stream, speeding up the parse. This is near impossible in a hand built lexer, because you need a lot of tabular data to do it properly.
It also makes your parser less buggy than hand lexing, in my experience. Evaluating tokens is a hell of a lot less error prone than evaluating individual characters, although I'm only assuming you're doing scannerless/lexerless parsing.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
If you could recommend a link for lex or flex, I'll take a look, though it'd probably be hard to retrofit it at this point. The shortest string stuff sounds nice. Even worse is having to back up and re-parse an entire identifier. I ran into this today while writing code to support a label that's the target of a goto. Not that they're used much, but goto is on the list of keywords that I never got around to supporting.
My lexer code is very string oriented, so there aren't distinct lexer and parser phases, which is what I assume you mean by "scannerless/lexerless". It's probably a trade-off between things like the shortest string optimization that you mentioned, and the ability to override rules (e.g. to allow punctuation in a name so that a template instance can be referred to by its verbatim name rather than some mangled monstrosity).
|
|
|
|
|
The man pages are actually VERY good. But also Using <CODE>flex</CODE> - An Overview of @code{flex}, with Examples[^]
That's a very basic tutorial on using flex.
Download Flex[^]
Now, here's something for some reason nobody really talks about.
In practice, forget those samples. They give you basics but nothing real world. Real world though, is even easier.
Have a set of constants for all your symbols. If you intend to parse >= you need a GREATERTHANOREQUAL/GTE constant. Give it an int value. Do it for all your terminals. It doesn't matter what the values are. You're going to use these to distinguish your tokens.
In your scanner:
%option unicode, codepage:raw, out: outputfile.c
%{
%}
%%
<<EOF>> { return EOS; }
"int" {return INT_TYPE; }
"uint" { return UINT_TYPE; }
(_|[[:IsLetter:]])(_|[[:IsLetterOrDigit:]])* { return IDENT; }
[\r\n\t\v\f ]
[\n]|[^\n] { return ERROR; }
basically you just return a constant a value for every token you capture. You can do more - whatever code you need in those brackets is fine. I use them for building my skip lists inside those little brace breakouts above where we return consts. Like
{ yyskip(LINE_COMMENT); }
since i didn't return a value, flex simply "hides" it. It only gives you something back when you return. otherwise it just keeps going. yyskip() is a function I made that would have been included in the "extra code" part above. It adds the current value to my skiplist.
Oh to get access to these values you use int symid = yylex();
and the text returned is in yytext
It's all pretty simple. It's a bit of work defining your tokens and consts up front but hey, it's a lot less work than parsing them yourself.
And it will make this FAST.
I actually generate these files using Parsley 
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
One thing I forgot to mention to you - regarding skipping comments.
I learned the hard way, if you're going to skip, you need to manually clear the skip list periodically.
I was trying to clear it after every read, and it doesn't work, because trailing items get lost if you aren't checking *everywhere*
I learned the hard way as the way I exposed these things doesn't leave me room to do this. (Part my fault, and part limitations of the lexer generator i'm using - skipped lists weren't really in the design  )
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I used to build databases by hand, and use PostScript and mark-up languages (the vast majority of which were far better suited to the job than XML) to produce documents.
... But then better tools came along, which allowed me to spend my time and energy on doing the work, rather than on setting up the infrastructure for it.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
There's one main reason I don't switch back to linux and that is Visual Studio. MonoDevelop is good, but not great. Linux C++ IDE offerings are bunk on linux even if gcc itself is good. KDevelop had potential way back when but oh well.
Has anyone ever tried to get Visual Studio (not VSCode!) running in linux in something other than a virtual machine?
I don't want all that hassle of running windows at all if I don't have to.
I just got my computer back from microsoft. I had to wait for it to install some unplanned updates that rebooted the machine 3 times and took almost 10 minutes. And MS says windows boot times are fast. 
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
The last VS version was better than garbage on wine was 2008... with a bronze... 2005 got gold...
Maybe here you could find something... .NET tools & editors for Windows, Linux and macOS[^] (I'm using VS Code)
"The only place where Success comes before Work is in the dictionary." Vidal Sassoon, 1928 - 2012
|
|
|
|
|
honey the codewitch wrote: Has anyone ever tried to get Visual Studio (not VSCode!) running in linux in something other than a virtual machine?
Little chance for that; too much components in there that will fail the install.
honey the codewitch wrote: I had to wait for it to install some unplanned updates that rebooted the machine 3 times and took almost 10 minutes. And MS says windows boot times are fast.
..an update is not a normal boot. I love my Kubuntu-machine, but to be fair, I also love the VS on Windows, and it certainly doesn't take 10 minutes for a normal boot.
In Linux you'd be loosing more time after each update, since not every packages' dependence will update nicely, going through weird errors and copy/pasting a lot of commands in the console window.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
Eddy Vluggen wrote: In Linux you'd be loosing more time after each update, since not every packages' dependence will update nicely, going through weird errors and copy/pasting a lot of commands in the console window.
You know, for some reason linux and I just get along. I don't tend to have a lot dependency problems, and I understand the various package tools pretty well.
It tends to be less cranky for me than windows overall. Usually if it doesn't work it's because i did something wrong. Windows won't work if I look at it sideways.
Not that I'm down on windows. I love windows, but I get sick of windows' $*(#@& sometimes and I want a good old ubuntu machine or something.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
honey the codewitch wrote: Usually if it doesn't work it's because i did something wrong.
That's not what most people report, as you already know.
honey the codewitch wrote: I love windows, but I get sick of windows' $*(#@& sometimes and I want a good old ubuntu machine or something.
And you can, but Linux is not stable enough for VS.
I'd be surprised if VS came past the installation of .NET; it's not a question of whether it runs, it won't even install.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
Wouldn't cost you anything to try it with a trial of codeweavers. I choose to run it in a W10 VM in Mint. Have not had any stability problems with many versions of Linux over the past 4 or 5 years. I have been using it for servers and my main system for some 10 years or more. Used to be problematic on my old Thinkpad but ran OK if I didn't suspend it. That was some 5 years ago. Run a MacBook now, with Fusion. VM's run fine.
I don't hate Windows... I don't hate anything. I am just a contrarian. 
Lou
If you can keep your head while those about you are losing theirs, perhaps you don't understand the situation.
|
|
|
|
|
I'll maybe try a VM since my latest machine has hardware support for it, but it doesn't have a lot of RAM which is why I wanted to avoid running one.
I don't think linux has stability issues, though some of the other commenters think that. I think it's more of an issue of the stability and interoperability of the packages you install. That's really what's at issue since linux is less of a "closed loop" than windows is, there's more opportunity to *#$%@# it up. That's not an issue though if you know what you're doing. It's a bit like coding in C++ vs. C#. I like both. C++ is more powerful but when you screw up you blow your whole leg off.
And for the record, i love windows. I even wrote a small part of it. (Click on "This PC" in a file explorer, and then Manage...)
But it can frustrate me. Linux is more my speed, but Visual Studio is my jam. Monodevelop and the various other IDEs on linux just don't cut it. I've been thinking of looking into JetBrains Ride to see how good it is, but I don't know if I want to buy yet another IDE after I've already invested so much into the VS stack, you know?
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
honey the codewitch wrote: though some of the other commenters think that.
Note some, just me, and you are allowed to name me.
honey the codewitch wrote: That's really what's at issue since linux is less of a "closed loop" than windows is, there's more opportunity to *#$%@# it up.
More dependencies that get changed then under Windows, where they are very reluctant to change anything that may break something, and where they can spend money on keeping stuff working.
honey the codewitch wrote: That's not an issue though if you know what you're doing.
So, the mismatch in the packages I install, and the updater killing itself is my own fault? 
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
> So, the mismatch in the packages I install, and the updater killing itself is my own fault?
As much as my VS2017 installation failing is mine
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
honey the codewitch wrote: And for the record, i love windows. I even wrote a small part of it. (Click on "This PC" in a file explorer, and then Manage...)
The MMC host?
|
|
|
|
|
no. the computer management snap-in for the MMC host
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Not possible, because VS is a WPF application.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
#realJSOP wrote: Not possible, because VS is a WPF application.
 "because shooting twice is just silly"
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
yeah... nah...
A year back looked at going this rout, didn't get as far as vs because first tried a few other simpler things (tools, utils I use) with wine that were gold rated and:
- at best I would say only "mostly" worked - sometimes caveats were noted, sometimes not.
- be prepared for at any time (even idle) for it to simply stop (freeze, spectacular or just disappear)
with a lot of tinkering I got things "more" mostly working but it was like per hour:
- 35 minutes tinkering,
- maybe 10 minutes actual work,
- 15 minutes preparing for the next inevitable fail
remember wine at best barely achieves stability at "windows xp" level, win 7? 
wine is not something I would daily use for anything I rely on: firstly there's always the feeling it's going to fail 60, 15, 1 minute later,
and that's only after making all the compromises and adjustments to your usage of the app to "achieve better stability" (i.e. they say "use these settings, avoid using X, ...
remember dev of wine itself is low, still years behind, poorly managed. In terms of dev towards app compatibility that's totally ad-hoc at best - only if someone needs something and they got the energy to do it does it happen, and only as far as their own needs for that app.
stick to VM's
- it works, it's easy, you can actually concentrate on getting work done
- not losing time tinkering the platform, installing / practising workarounds and compromises
- in case you're worried it's legal.
<< Signature removed due to multiple copyright violations >>
|
|
|
|
|
Thanks for the heads up. I wasn't going to assume WINE was still unstable - it has been years since I've played with it, and I had heard some good things, but I guess I should remain skeptical.
My only issue using the VM is I only have 8GB of ram in my dev machine for now (long sordid tale, i really need to upgrade this old i5)
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I need at least a bottle and a half, just to get started.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
I've been working on and off for the past year in trying to write a small mobile app to help my training at the gym. It's stupid simple what I'm after, but I've been trying to avoid going the easy (aka hard) route of separate iOS and Android apps. Because I like to complicate things and yet I'm lazy and super short on time.
I have a Windows Form version, which I ported to a Vue/Typescript web app. I'd like to stick to either .NET or TypeScript as my language of choice.
It seemed Xamarin would be sensible given that it allows me to stay in C#. Except my experience with Xamarin hasn't been great and it seems Microsoft is pushing React Native for UWP apps. So why not Reach Native? Porting Vue to React/RN is simple enough and I'll simply stick with TypeScript as my core language and I'm done.
Except that I cannot, for the life of me, get anything to work. I've forked GitHub repos that have "complete apps". I've walked through, line by line, the official docs on Facebook. I've spent the hour installing the Android SDKs, node, gradle, chocolatey, yarn, react native, etc etc etc. GB worth of installs. Endless command lines. Opening a powershell window in Admin mode to run the ps1 scripts. Manually adding the environment variables.
Insane.
I can't believe that 13 years after the iPhone was released we're still in the string and ducttape era of cross platform mobile development.
Once I get this working and boiled down to something sensible I'll write an article.
cheers
Chris Maunder
|
|
|
|
|
Chris Maunder wrote: Except my experience with Xamarin hasn't been great
Sorry to hear that. I've had nothing but success with Xamarin. Admittedly, I currently build native Android apps only.
/ravi
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
