Welcome to the Lounge

Mark_Wallace9-Jan-20 20:33

Mark_Wallace

9-Jan-20 20:33

Perfect code is about the same length as a piece of string.

I wanna be a eunuchs developer! Pass me a bread knife!

honey the codewitch9-Jan-20 21:10

9-Jan-20 21:10

I like that answer. Although in this case I have a big nasty tangled ball of yarn I suppose. 934K source file. woo

And that's not counting the 235k tokenizer it needs

hack everything.

KarstenK9-Jan-20 20:38

KarstenK

9-Jan-20 20:38

Normally is written less code as possible the best solution.

Important hint: this includes the usage of well tested libraries and frameworks. Cool | :cool:

Press F1 for help or google it.
Greetings from Germany

honey the codewitch9-Jan-20 21:09

9-Jan-20 21:09

Umm, all this code is generated.

So I've written zero code here.

hack everything.

Jon McKee9-Jan-20 20:44

9-Jan-20 20:44

If it works and meets performance and -ility expectations I wouldn't worry about it. It's always nice to find novel approaches that make the code smaller, cleaner, and/or faster but you could spend months on a solution that might not even exist and might mean absolutely nothing to anyone but yourself. That's time you could have poured into another project Smile | :)

Disclaimer: This advice is a little hypocritical as I have to actively stop myself from doing the same thing.

honey the codewitch9-Jan-20 21:08

9-Jan-20 21:08

Unfortunately I have no idea what is acceptable because this is a tool for other people to use, not for a specific project or client.

It has competition though so anything smaller and/or faster is going to be "better", ie more desirable.

So there's that.

I have my own concerns about using this parser i generated because my CodeDomGoKit.brick.cs file is already over 500K and that's with a parser that's hand written and under 100K. Not 934K or whatever it was!

It's not a show stopper, but it's almost embarrassing for me to have build tools that are 1MB executables.

hack everything.

Jon McKee9-Jan-20 22:03

9-Jan-20 22:03

If competing with products in the market is the goal I can see the problem. Usually I try to stick to my original requirements/goals to avoid the endless cycle of "I could probably make this better" but that doesn't seem to be an option here with active competition.

Considering that, my only input would be that generated code will never be as efficient as hand-rolled. I'm running into a similar issue with a project that generates a full type-tree constructor for a given type given the data to "fill" that type tree. There are always going to be edge cases that exist in the general problem space which prevent some optimizations. You can't solve a general problem as efficiently as a specific problem.

Best of luck though! I only understand a fraction of what you post but it sounds like a really cool project Thumbs Up | :thumbsup:

honey the codewitch9-Jan-20 22:31

9-Jan-20 22:31

Jon McKee wrote:
I only understand a fraction of what you post but it sounds like a really cool project

Yours too - although i can't quite wrap my head around it. Now I'm curious, are you trying to precompile something you'd have to reflect at runtime?

hack everything.

Jon McKee9-Jan-20 23:18

9-Jan-20 23:18

Something like that, yea. I'm creating a delegate for a type that constructs it and all of it's dependencies. I'm trying to write the data input/"parser" layer very generically so it can support a lot of cases, but the original idea related to parsing would be something like:

class User {
    [OverrideMethodBind(Method = "Add")]
    [OverrideParameterBind(Type = typeof(Address))]
    List<Address> addresses;
    [DataBind]
    string firstName;
    [DataBind(ID = "LN")]
    string lastName;
}
class Address {
    [DataBind(ID = "street")]
    string streetAddress;
    string city;
    [DataBind]
    StateAbbrev state;
    int zipCode;
}
enum StateAbbrev {
    AL,
    WA,
    MI,
    //etc....
}

In this example you want to create the object based on (state, streetAddress)+ (i.e. one or more times), firstName, and lastName from the data.

You can add metadata to the items of interest, supply "parsers" for various members (based on whatever condition you want - type, member, or ID), and it uses the data to then create (possibly many) Users based on that data. Removes the need for boilerplate related to loading a domain object with parsing results or whatever your data source is (another object, database, etc). So instead of writing something like:

MatchCollection results = regex.Matches(/* some input*/);
User u = new User(results["firstName"], results["LN"]);
//this also only deals with a single address
Address a = new Address(results["street"], results["state"]);
u.addresses.Add(a);

You'd write:

ObjectGenerator g = new ObjectGenerator<User>().AddParser(regex);
//or
ObjectGenerator g = new ObjectGenerator(typeof(User)).AddParser(regex);

User user = g.GenerateObject(/* some input*/);
//or
User[] users = g.GenerateObjects(/* some input*/);

Not a finalized interface but that's the jist of it.

honey the codewitch9-Jan-20 23:22

9-Jan-20 23:22

Very cool. I've been wanting to do something like that for a long time myself - a way to "bind" grammar constructs from say, an XBNF document to fields/properties in object models via metadata but i haven't found an expressive enough way to do it that isn't more work than just manually coding it. The upshot of doing that would be to get your parsed stuff directly into your final objects.

hack everything.

Jon McKee9-Jan-20 23:35

9-Jan-20 23:35

honey the codewitch wrote:
The upshot of doing that would be to get your parsed stuff directly into your final objects.

That's the goal! You nailed the main issue - that for simple single-use cases it's more work to do this than to just write the code manually. I'm attempting to make it expressive (and fast) enough to justify using with complex objects and/or multiple data source formats (XML, JSON, CSV, etc into the same object tree).

honey the codewitch10-Jan-20 2:57

10-Jan-20 2:57

That's a very tall order. I'd love to know what you come up with. My interests are closely related.

hack everything.

Jon McKee10-Jan-20 1:00

10-Jan-20 1:00

Since you seem to be the resident parsing expert, do you have any "must read" suggestions? I admittedly only know the basics but I've been interested in parsing for quite awhile since regular expressions in Perl were my introduction to programming. I got "Programming Perl" (a book I still own) as a gift over 18 years ago and after learning the basics of Perl, regular expressions were what cultivated my interest in programming.

honey the codewitch10-Jan-20 2:55

10-Jan-20 2:55

Honestly, I just picked it up along the way. What really broke me through where these two links.

LL(1) Parsing[^]

and LALR(1) Parsing[^]

Don't bother with LALR(1) until you can do LL(1)

I wrote "How to Make a Parser in 3 steps" here:

How to Make an LL(1) Parser: Lesson 1[^]

hack everything.

Daniel Pfeffer9-Jan-20 21:33

Daniel Pfeffer

9-Jan-20 21:33

Premature optimization is the root of all evil
-- D. Knuth

Make it work (check - Slang generator)
Automate it (check - Parsley generator)
Profile... profile... and profile again. Only then can you make intelligent decisions about what to improve.

Smaller and faster code is obviously better, but you eventually reach the point of diminishing returns. Is it really worth a few months of your life to improve code speed/size by (say) 1%?

Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.

honey the codewitch9-Jan-20 21:39

9-Jan-20 21:39

My hand written parser is 1/10th the size of the generated one, so hopefully there's room for improvement.

For now, I'm rewriting the handwritten parser, but I'm using Parsley to help me do it, using the /noparser option so I get its tokenizers from its grammars but it doesn't generated the parser for me.

I'm using the grammars as a guide to do it again, this time much more correctly and with some other foundational improvements.

I expect the code size to be within spitting distance of 100K, so that will cut my CodeDOM Go Kit! distro size down compared to using the generated code.

Later I'm going to go back and experiment with code synthesis in parsing, and see if i can't replace things like standard LL "loops" with actual while loops and such.

That should make it smaller. I just want to trim a lot of this boilerplate out of it, but it's not straightforward to do, especially in such a way as to keep the code readable.

hack everything.

jsc429-Jan-20 23:29

jsc42

9-Jan-20 23:29

Daniel Pfeffer wrote:
Smaller and faster code is obviously better

But unfortunately, they are often mutually exclusive. My first thought on seeing the code sample was ...

if ({ ExpressionParser.add, ExpressionParser.sub, ..., ExpressionParser.charType }.Contains(context.SymbolId))

This would be smaller, but Contains requires iteration over an array, much slower to execute than a simple short-circuiting Boolean expression. So, the old metric of Lines Of Code is flawed.

musefan9-Jan-20 23:59

musefan

9-Jan-20 23:59

A quick look at the brackets and I don't think a contains would do the same thing in that example.

But, your point still makes sense as an example.

jsc4210-Jan-20 6:46

jsc42

10-Jan-20 6:46

You're right! I should have checked the syntax first. The code should look like ...

ExpressionParser[] ep = { ExpressionParser.add, ExpressionParser.sub, /* ..., */ ExpressionParser.charType };
if (ep.Contains(context.SymbolId)) { }

if (new [] { ExpressionParser.add, ExpressionParser.sub, /* ..., */ ExpressionParser.charType }.Contains(context.SymbolId)) { }

assuming that ExpressionParser is an enum.
My delay in replying was that I hadn't realised that Contains is an extension method of System.Linq (but is easy to hand-code).