|
I'm very new to the source generator technology in C# so bear with me, as my information is probably incomplete or inaccurate but what I've discovered so far is amusing and annoying at the same time.
I tried to get a source generator referenced from one project for use in another project and I was getting weird results when crawling the syntax tree. I don't even know how to google about the results I'm getting back. It's hard to explain, but it breaks the thing.
Creating an incremental generator[^]
I followed this code precisely, and it didn't work on my system.
However, it will work when I invoke the generator to run specifically as part of a unit test. I haven't had as much luck invoking it outside a test framework. I have no idea why this is.
I *believe* (read: hope) it will work when packaged as a NuGet package as well, which is how it was designed to be deployed and used.
Of course, I don't want to deploy a NuGet package of code still in initial development.
The bottom line for all of this is that this leaves me in the position of being forced to perform Test Driven Development in order to make source generators.
I don't know if this is a weirdness with my machine, or perhaps a quirk in this relatively new technology or what, but it's just bizarre to be forced into a particular development paradigm like this.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
I will be working on a Source Generator this weekend, my first time too. I was looking at the same article series and a video linked below.
honey the codewitch wrote: I followed this code precisely, and it didn't work on my system.
Have a look at this: How to Organize Your .NET Minimal APIs with Source Generator - YouTube[^]
When I finish tackling this task, I will publish an article.
Graeme
"I fear not the man who has practiced ten thousand kicks one time, but I fear the man that has practiced one kick ten thousand times!" - Bruce Lee
|
|
|
|
|
Yeah, I just sat through that youtube video to tell me everything I already know, and that nevertheless does not work on my system as advertised.
I have to use a test framework to get the damned thing to run at all. Microsoft can get effed. 
Edit: IT WORKS! You just have to close VS and reopen it for it to actually take effect. Be nice if somebody said something.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
honey the codewitch wrote: Edit: IT WORKS! You just have to close VS and reopen it for it to actually take effect.
Congrats and thanks for the tip!
Graeme
"I fear not the man who has practiced ten thousand kicks one time, but I fear the man that has practiced one kick ten thousand times!" - Bruce Lee
|
|
|
|
|
void Main(string[] args) is broken.
The issue is this: It dequotes values but doesn't tell you which ones were quoted. So if you naively use these as arguments, you will interpret the literal string "/ifstale" as a command line switch /ifstale instead of a literal string. Yes, the problem is a bit contrived, but it's kind of scary to think about the possibilities where something like it can crop up. It flies in the face of application robustness.
My solution involves Environment.CommandLine which reports the raw command line argument, quotes and all.
The issue with that is sometimes your .NET assembly is hosted by another executable, like dotnet.exe and then your arguments are all off by two.
Here's what i have to do. Basically what I'm doing is making the switch -- normally, but / for windows, primarily because of file path issues causing ambiguities with /
After that I crack the command line arguments, getting the exe name separately. I didn't post that code but it's here if you're interested: Environment.CommandLine cracker - Pastebin.com[^]
var prefix = "--";
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
{
prefix = "/";
}
var cl = Environment.CommandLine;
string exename;
var clargs = CrackCommandLine(cl, out exename);
if (clargs.Count >= args.Length)
{
clargs = clargs.GetRange(clargs.Count - args.Length, args.Length);
}
else
{
clargs.Clear();
for (int i = 0; i < args.Length; i++)
{
clargs.Add(new KeyValuePair<bool, string>(false, args[i]));
}
}
Then - get this - I have to count the arguments backward from the end to match them up to your actual executable's arguments, and that's the only thing the args[] array is good for.
Now at the end of that code, you have keyvaluepair arguments where the Key is whether it's quoted or not, and the Value is the de-quoted, de-escaped value in any case. It's dirty, but it works.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
I agree, I don't do that. I parse Environment.CommandLine myself (using a rather complex Regular Expression).
Oh, thats right, I have an article on it...
ParsedCommandLine[^]
(It's not up-to-date, I added more features to it over the years.)
|
|
|
|
|
A regex is kind of complicated for that. I prefer using my looping dual state state machine (state is just represented by the isQuote bool.
The technique i presented above is used in the latest GitHub bits of the project here:
Program.Base: Drop In Command Line Application Functionality for Your Projects[^]
It does: Command line parsing, using screen generation, error handling, file and argument object lifetime management, word wrapping (necessary for using screen anyway so i exposed it) stale file checking and progress reporting.
It's a drop in file with no dependencies, and is targeted to .NET Framework as well as the newer stuff. (I have two files for it but the only difference between them is the .NET Framework (DNF) file does not have #nullable disable at the top, which shuts up nullable reference type warnings in the newer C#. You just pick the one you need.
If you can get it approved by your work you're more than welcome to it. I developed it because I got sick of writing essentially the same code over and over again.
This is less work than that System.CommandLine stuff, and does more.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
File this one under Weird (Weird Documentation)
I was looking for a way to read the thumbprint of a X509 Certificate and found a perfect example at:
X509Certificate2.Thumbprint Property (System.Security.Cryptography.X509Certificates) | Microsoft Learn[^]
It's a nice compact sample where you pass in the path to the Cert and it prints out properties of the cert.
However, there's a line in the code that I was curious about:
x509.Import(rawData);
Just Want To Read Properties of Cert
Just wanted to make sure it wasn't importing the cert into my local cert store.
I really don't want to do that. Just want to read properties.
Microsoft Docs On X509.Import
So I looked up the method in the docs (X509Certificate2.Import Method [^])
*Note, if you examine the URL you'll see that it includes view=net-8.0 (assuming that is referencing the latest .net 8.0 docs)
Import Method is Marked Obsolete
But then when you look at the docs the Import method is marked obsolete.
But there doesn't seem to be any mention why.
Since Import just imports the bytes of the certificate (doesn't import it into the cert store -- that is confusing naming) I went ahead and ran the sample.
Sample Ran Perfectly Fine
Of course the sample ran fine and served my needs.
But the "obsolete" thing is confusing.
Not sure why there isn't a pointer to more updated sample.
|
|
|
|
|
|
That's a good find and I appreciate you providing the link here.
Once I read that, I remembered having to make a code change related to it back in 2021/2 because this functionality change was being reported ahead of time.
It's too bad they didn't make that link more conspicuous in the Import method docs.
Thanks again
|
|
|
|
|
Update 3: Main-Lorentz isn't quite as slick as I hoped. Since it subdivides it will find foofoofoofoofoof and make it (foofoo){2}foof. Still it does alright, and I augmented it to find the extra foo in some cases like below:
var nfa = FA.Parse(@"(ba[rz])*(foo){3}f");
nfa.ToString("e") = @"(foofoofoof|ba[rz](ba[rz])*foofoofoof)"
nfa.ToString("r") = @"(ba[rz])*(foo){3}f"
RegexOrExpression: foofoofoof|ba[rz](ba[rz])*foofoofoof
RegexLiteralExpression: foofoofoof
RegexConcatExpression: ba[rz](ba[rz])*foofoofoof
RegexLiteralExpression: ba
RegexCharsetExpression: [rz]
RegexRepeatExpression: (ba[rz])*
RegexConcatExpression: ba[rz]
RegexLiteralExpression: ba
RegexCharsetExpression: [rz]
RegexLiteralExpression: foofoofoof
Reducing...
RegexConcatExpression: (ba[rz])*(foo){3}f
RegexRepeatExpression: (ba[rz])*
RegexConcatExpression: ba[rz]
RegexLiteralExpression: ba
RegexCharsetExpression: [rz]
RegexRepeatExpression: (foo){3}
RegexLiteralExpression: foo
RegexLiteralExpression: f
(ba[rz])*(foo){3}f
Update 2: I got the algorithm working. Now I just need to port away from my C++ comfort zone into C# and make it part of my lib.
Update: I still haven't gotten the algorithm to work but it's the final piece of the puzzle for my particular current test case:
var nfa = FA.Parse(@"(ba[rz])*(foo){3}f");
nfa.ToString("e") = @"(foofoofoof|ba[rz](ba[rz])*foofoofoof)"
nfa.ToString("r") = @"(ba[rz])*foofoofoof"
RegexOrExpression: foofoofoof|ba[rz](ba[rz])*foofoofoof
RegexLiteralExpression: foofoofoof
RegexConcatExpression: ba[rz](ba[rz])*foofoofoof
RegexLiteralExpression: ba
RegexCharsetExpression: [rz]
RegexRepeatExpression: (ba[rz])*
RegexConcatExpression: ba[rz]
RegexLiteralExpression: ba
RegexCharsetExpression: [rz]
RegexLiteralExpression: foofoofoof
Reducing...
RegexConcatExpression: (ba[rz])*foofoofoof
RegexRepeatExpression: (ba[rz])*
RegexConcatExpression: ba[rz]
RegexLiteralExpression: ba
RegexCharsetExpression: [rz]
RegexLiteralExpression: foofoofoof
Final reduce yields: (ba[rz])*foofoofoof
Oh boy, a bit of background first:
I have (soon to be past tense) a programming problem wherein I'm trying to reduce the complexity and/or redundancy of a regular expression. This is primarily because converting from a state machine to a regular expression yields a lot of redundancy and weird loop boundaries and stuff. It's correct, but even harder to read than a typical regex expression.
So I load this into an abstract syntax tree representing the expression through things like RegexConcatExpression and RegexLiteralExpression.
I then keep trying to reduce the whole tree until no more changes have occurred. Each expression class has it's own reduction process. Right now I'm working on RegexLiteralExpression
The big idea here is to reduce expressions like "bazfoofoofoobar" into "baz(foo){3}bar"
So i need an algorithm to find me "foofoofoo" in that string.
Enter this clever bit of mathiness:
Finding repetitions - Algorithms for Competitive Programming[^]
Pretty cool stuff. I ported it to C#.
Edit: And after trying a second implementation as well as the above, I don't understand the results I'm getting. 
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
modified 23-Jan-24 9:28am.
|
|
|
|
|
Quote: And after trying a second implementation as well as the above, I don't understand the results I'm getting. Cry | 
Welcome to my world   
M.D.V. 
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
|
honey the codewitch wrote: I don't understand the results I'm getting.
This reminded me of a comment that I read long ago:
quote: Abandon hope all ye who look at this code. Not even its author knows how it works, or why.
The code in question sent instructions to a remote virtual stack machine for which proper documentation no longer existed. It was partially written by trying various inputs and observing the behavior of the black box. 
|
|
|
|
|
[[:IsLetter:]_][[:IsLetterOrDigit:]_]*|0|-?[1-9][0-9]*(\\.[0-9]+([Ee]-?[1-9][0-9]*)?)?|[ \t\r\n]+
Runs over twice as slow as
[[:IsLetter:]_][[:IsLetterOrDigit:]_]*|0|-?[1-9][0-9]*(\\.[0-9]+([Ee]-?[1-9][0-9]*)?)?|[[:IsWhiteSpace:]]+
IsWhiteSpace is a Unicode charset. The [ \t\r\n] is simply ASCII.
Why is this weird? Because there are a lot more characters in the IsWhiteSpace character set than 4 ([ \t\r\n]) which should make the transitions slower due to having to search several character ranges.
Also 1552ms vs 629ms. That was what I found in my tests.
My regex lib searches and matches the test input in about 4ms regardless of which expression is used.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
honey the codewitch wrote: twice as slow
Please turn in your keyboard, you're done for the day.
But seriously, are those COLONs supposed to be BRACEs?
I've never used that syntax, and I'm trying to find it in the documentation.
modified 7-Jan-24 11:51am.
|
|
|
|
|
I can't speak to MS RE's but in POSIX, character classes use [:space:], which can be used within a bracket (i.e. []) expression.
"A little song, a little dance, a little seltzer down your pants"
Chuckles the clown
|
|
|
|
|
All I see in
Character Classes in .NET Regular Expressions - .NET | Microsoft Learn[^]
is \p{name} and \P{name} -- e.g. \p{IsCyrillic}.
And I just learned about -- Character class subtraction: [base_group - [excluded_group]] -- which may be a new feature.
E.g. [\p{IsBasicLatin}-[\x00-\x7F]] or [\u0000-\uFFFF-[\s\p{P}\p{IsGreek}\x85]] .
I note that both of these require the \p{name} notation, so maybe what CW is testing isn't doing what she thinks.
|
|
|
|
|
I just tested with .NET 8, and the :Whatever: syntax works perfectly fine.
What do you get when you cross a joke with a rhetorical question?
The metaphorical solid rear-end expulsions have impacted the metaphorical motorized bladed rotating air movement mechanism.
Do questions with multiple question marks annoy you???
|
|
|
|
|
OK, I've never seen it and I don't see it documented.
|
|
|
|
|
They are Unicode Character classes. They match the static methods on char in C#. They're supported on all major unicode regex engines i'm aware of as is [:characterclass:] on all posix engines
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
honey the codewitch wrote: They match the static methods on char in C#.
Then I hope they don't call those static methods on each character as they go.
P.S. Maybe try changing it to the [\p{name}] form and compare?
|
|
|
|
|
Probably not. I have a 688KB C# file with all of the supported character classes and codepoint ranges. Unicode is big.
I imagine they have something similar.
As far as [\p{name}] I am vague on that form of expression but isn't it Unicode? The unicode one is already faster. The curious bit is ascii. I suppose I could try [:alnum:]
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
Right, as far as I can tell [\p{name}] == [:name:] . But do they perform the same? They should.
|
|
|
|
|
I'll find out when I get a chance. Just based on the way I parse this stuff I'm assuming it will be the same.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
In the days of 7/8 bit chars, those class tests were often implemented in bitmaps (e.g. 8 classes in an array of 256 bytes)
A similar trick in, say, UTF-16 wouldn't be outrageous these days.
Software rusts. Simon Stephenson, ca 1994. So does this signature. me, 2012
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
