|
Quote: Possibly the hottest mess I've seen in some time
Word for word what I thought when I met my wife 
GCS/GE d--(d) s-/+ a C+++ U+++ P-- L+@ E-- W+++ N+ o+ K- w+++ O? M-- V? PS+ PE Y+ PGP t+ 5? X R+++ tv-- b+(+++) DI+++ D++ G e++ h--- r+++ y+++* Weapons extension: ma- k++ F+2 X
|
|
|
|
|
|
Don't you keep working on stuff like that just for the great feeling when you finally make it work?
Been there, done that, but not on that particulr board.
|
|
|
|
|
Don't know if this is what you're looking for but I bought one recently and it's a pretty decent device.
I was looking for an Alexa type device but not cloud based.
Audeme - Home[^]
|
|
|
|
|
Cool. I'm looking to avoid the Arduino ecosystem, as it's more bulky, pricey and less powerful than I'm used to as a general rule. I ordered a couple of different ESP32 based LyraT boards which are sort of gold standard boards for doing this sort of thing in the ESP32 world.
But yeah, that's basically what I'm doing. I intend to release a library for it, so maybe I'll end up supporting the Arduino boards down the road, but I'm really focused on the ESP32 line.
To err is human. Fortune favors the monsters.
modified 3-Apr-23 15:30pm.
|
|
|
|
|
I'm fixing to convert it to use with ARM, I going away from Arduino.
I'd eventually like to do some Home IoT stuff...if I ever get the time.
|
|
|
|
|
Mike Hankey wrote: if I ever get the time.
Time is such a precious commodity when you are retired ... sometimes I wonder how I found time to work 
Mircea
|
|
|
|
|
You got that right.
I work more now than I did when I worked full time.
Difference is there's no pressure to get them done.
Give me coffee to change the things I can and wine for those I can not!
PartsBin an Electronics Part Organizer - An updated version available! JaxCoder.com
Latest Article: Simon Says, A Child's Game
|
|
|
|
|
Mike Hankey wrote: I work more now than I did when I worked full time.
Difference is there's no pressure to get them done.
Yeah, Ain't that the truth. I left salaried employment to start my own shop, but
I worked twice as hard as I did when I had an actual job!
Then I retired, or at least I drew my retirement, so now I only spend about 8 or 9 hours in front of the keyboard tinkering with projects that interest me!
|
|
|
|
|
DrWalter PE wrote: I worked twice as hard as I did when I had an actual job!
Then I retired, or at least I drew my retirement, so now I only spend about 8 or 9 hours in front of the keyboard tinkering with projects that interest me!
Yep, but lately I've had to finish some woodworking projects so it's taken me away from the keyboard, temporarily anyway.
Give me coffee to change the things I can and wine for those I can not!
PartsBin an Electronics Part Organizer - An updated version available! JaxCoder.com
Latest Article: Simon Says, A Child's Game
|
|
|
|
|
Microsoft wrote:
You can use wildcard characters (* or ?), to represent one or more characters of a file name and to display a subset of files or subdirectories.
The asterisk wildcard always uses short file name mapping, so you might get unexpected results.
You can use the question mark (?) as a substitute for a single character in a name.
-- dir | Microsoft Learn[^]
The question mark does not actually substitute for a dot/period/full stop (.) in a file name.
C:\Users\PIEBALD\AppData\Roaming>copy con foo_bar.txt
^Z
1 file(s) copied.
C:\Users\PIEBALD\AppData\Roaming>copy con foo.bar.txt
^Z
1 file(s) copied.
C:\Users\PIEBALD\AppData\Roaming>dir foo*bar.txt
Volume in drive C is Windows8_OS
Volume Serial Number is 528E-277C
Directory of C:\Users\PIEBALD\AppData\Roaming
2023-03-31 08:06 0 foo.bar.txt
2023-03-31 08:06 0 foo_bar.txt
2 File(s) 0 bytes
0 Dir(s) 28,516,610,048 bytes free
C:\Users\PIEBALD\AppData\Roaming>dir foo?bar.txt
Volume in drive C is Windows8_OS
Volume Serial Number is 528E-277C
Directory of C:\Users\PIEBALD\AppData\Roaming
2023-03-31 08:06 0 foo_bar.txt
1 File(s) 0 bytes
0 Dir(s) 28,516,675,584 bytes free
C:\Users\PIEBALD\AppData\Roaming>del foo*bar.txt
C:\Users\PIEBALD\AppData\Roaming>
As a further note, I have 8dot3 naming turned off on my personal system and I'll have to try this there.
Tested on Win10 as well as this old Win8 junker.
|
|
|
|
|
To be fair, they do warn you that the * uses the 8.3 filename. I guess the ? does as well.
Use dir /X to get the 8.3 filenames.
|
|
|
|
|
Yet the 8dot3 names don't seem to make a difference -- it behaves the same whether 8dot3 names are enabled on the volume or not. Or maybe it generates 8dot3 names on-the-fly as needed but doesn't store them?
|
|
|
|
|
The documentation you cite was proofread poorly. They refer to "dir t97\*" several times rather than "dir t97.*".
Software Zen: delete this;
|
|
|
|
|
I'd just like to point out a gem I found underneath a semi-private portion of the ESP-IDF development framework.
The headers for it reside under a folder called "esp_private"
The magic sauce here is access to a very cool coprocessor of sorts. What it does is take a series of pins, which it then treats as bits - High = 1, Low = 0, like that.
Then you give it a clock frequency and a memory buffer, and that little coprocessor will traverse that memory buffer in the background flipping those pins off and on based on the data in the buffer you gave it. It all happens as rapidly as you need it to, and in the background.
You can use it to power things like RGB interfaced LCD controllers, Hub75 LED matrices, or anything else where you need a battery of digital signaling you can control with a byte buffer.
Very cool. Very undocumented. I'm busy hacking my way through it because while Espressif exposed a higher level RGB LCD panel interface that uses it under the covers, that doesn't work in my environment as the tech is very new. The older tech is incomplete and not functional in that capacity, but it's what I'm stuck with so I'm going to the metal myself. But that's not important.
The takeaway here is the amazing technology, and the fact that they're in no hurry to expose it to downstream consumers of the platform.
To err is human. Fortune favors the monsters.
|
|
|
|
|
Very interesting.
How did you discover this undocumented functionality?
I'm a hacker (the good kind) and I'm always curious about how people find the stuff that is hidden.
Also, I'm working on a ESP32 thing right now because you advised that it would be a good platform for my BLE work. It's going really well and I can't believe I've finally made a break-thru.
Anyways, while looking at a small ESP32 dev board at amazon, I stumbled upon this picture from amazon that is absolutely hilarious[^]. Can you advise on that use of the ESp32 with those tweezers? 
I believe he is trying to dip his ESP32 switches into the main board which will surely create a new computing platform.
|
|
|
|
|
I was trying to figure out if the ESP LCD Panel API in the ESP-IDF went underneath SPI Master actually (it doesn't) but while I was there I was looking at their i8080 bus code, and saw it used this thing called GDMA and gathered what it did from context, at least in broad strokes. I then did a bit of googling around and found other people hacking it, using it to drive LED matrices and such. 
I recommend the ESP32-S3 for all new development. At least until the ESP32-P4 is released. It has more pins, a more refined hardware architecture, including GDMA, and native USB 1.1 support, so you don't need to fab a serial UART USB bridge to program the damned thing. Just wire D- and D+ of your USB to pins 19 and 20 respectively.
To err is human. Fortune favors the monsters.
|
|
|
|
|
Very cool and thanks for the advice on the boards.
I'm just getting started and my first need is a device that supports BLE (i know many people hate Bluetooth, but every phone has it and this is an app that will be used by phones).
By the way, is there a good way to decide if a ESP32 supports BLE?
Or is that basically built-in/ always present like the wifi connection?
I know there are too many h/w companies to say for sure but is it basically an expectation of the platform? Just curious, I know BLE is not something you're interested in.
thanks
|
|
|
|
|
BLE is present on every ESP32 in the line, as far as I know. I'd definitely go with an S3 board, just so you're future proofing the experience you get.
To err is human. Fortune favors the monsters.
|
|
|
|
|
Running a process on one thread -- seven minutes.
Splitting the process onto two threads -- six minutes each.
Splitting the process onto four threads -- five minutes each.
Most of the time spent seems to be the threads waiting on a lock on a shared resource. 
Too much for a Friday afternoon, I'll get back on it on Monday. It must be something really stupid.
|
|
|
|
|
Yes, it looks like you do it wrong, sorry.
Ok, if the problem is a shared resource then you are not basicaly wrong.
But in case you try to use all cpu power from your machine, don't use threads, use parallel proceses.
|
|
|
|
|
A study mate of mine went to a job to implement, or rather port, a database system on the VAX. What they had for development and testing was a 730, probably the slowest VAX ever manufactured. They put a large, green "Turtle Wax" sticker on its front panel.
My good friend timed a couple operations: To blank fill an 80 character buffer before reading an input to it took, for blank filling alone, 20 milliseconds. No, not microseconds, but milliseconds. A process switch (I am quite sure they were running VAX VMS; I don't think any alternative was available in 1983) took 100 ms., i.e. 1/10 of a second.
This DBMS was designed as three processes with a lot of interaction between them. Or, rather, as much as was possible given that process switch time. That wasn't a lot! They had to chop one of the processes into pieces and distribute it on the two other processes, so that the interacting parts were (at least to a much higher degree) running in the same process with no process switch necessary. I don't know the details of how they did it, but suspect that they used a home-made coroutine switching, just loading the stack pointer and program counter of the communication partner to have the message processed. (The project leader was a professor of compiler construction, well versed in register and stack management.)
So the answer is sometimes not to increase the number of threads/processes, but exactly the opposite!
If you task is 100% CPU bound, and only 1 CPU is available, then introducing the overhead of thread switching will never speed up things. On the other hand, you rarely see 100% CPU bound tasks, and nowadays, you rarely see a machine with only 1 CPU available. For the VAX 730, it seemed like you had significantly less than 1 CPU available
|
|
|
|
|
I'm twenty years OpenVMS clean now. (Other than some minor dabbling, which I can't even do now that HP has killed the hobbyist program again.)
A similar process I have is IO-bound -- writing the results to disk -- so multi-threading won't help with that one.
But this one is writing to the database (SQL Server, SqlBulkCopy).
* Read 1000 objects
* Prepare each object to write to the database
* Write the objects to the database
* Repeat as needed (250000+ total objects)
I timed the reading and the writing portions, the total time for writing was very low.
The reading part is reading DNS data from domain controllers via LDAP, so I was expecting that the reading was taking the time, so I am astonished that I'm not seeing much time spent doing the actual reading.
Anyway, it can wait for Monday.
|
|
|
|
|
Obviously, your DNS server (the one you are reading from) can only handle a single request at a time. Somewhere behind the curtain, to do a read you must get hold of a semaphore. It may be so much behind the curtain that it is called a critical region or monitor, but in any case, it boils down to gaining control of a semaphore.
Maybe the semaphore handling (or region / monitor entering and leaving) really takes a lot of time, but your single-thread solution doesn't notice because it has obtained the data and is buy writing it to the database. Yet, the one(s) waiting in line wont obtain the semaphore until all the behind-the-scenes release work is completed.
If the waiting process has to poll for the semaphore, he won't get it immediately when it is freed. Maybe he comes a second or two later, and that time is lost. If it happens for every LDAP request, it adds up!
Note that everywhere but in the *nix community, programmers knew of queueing semaphore (so you didn't have to poll), regions and monitors based on queueing semaphores, from the mid/late 1970s. *nix had nothing like it, except "Let us create a file, whose existence indicates 'resource reserved'!" It is an extremely resource demanding semaphore, compared to OS implementations, and it is binary (queue-less), so you have to poll it. *nix programmers grew up without knowing anything about proper synchronization. When finally OS based semaphores where introduced to *nix, they were first binary; you still had to do the polling. And *nix programmers were very reluctant to start using them. Even today, it seems like young programmers know very little about proper use of semaphores, regions, and monitors, and regularly use binary semaphores as something like a voluntary Post-It: 'Please be so kind not to touch these data while I am working on them, will you?' Both putting up the Post-It and for others to read it is sort of voluntary. (I learned to build non-voluntary, efficient queueing semaphores, critical regions and monitors from the 1973 Brinch Hansen book 'Operating System Principles - it is all there, from 50 years ago. Obviously, not a single *nix programmer I have ever met has as much as heard of that book. Or any other describing the same solutions.)
So my guess is that you have come across a really poor implementation of resource protection, probably based on polling a binary semaphore. When the second process finally gets around to grabbing it, the first one is almost ready to enter his next round, a split second too late; #2 tool the semaphore. So now #1 is polling, at long intervals to keep the CPU load moderate, and comes maybe a second after the semaphore is freed. Wasted time. Wasted work. And the play repeats itself a few hundred or thousand times.
The process switching also takes resources: There is a whole lot to do when #2 takes over for #1 or the other way around, and it happens not only when the semaphore is grabbed, but every time it is polled. The timing loop for the polling takes time. Maybe not only your machine must do heavy context switches; it could be the same on the other side, the domain controller.
This semaphore polling is not necessarily in your code; it may lie in some library routine or in the LDAP implementation or somewhere else. After all, most LDAP accesses look up a single or a few objects; a short delay caused by the need for polling is acceptable. It just doesn't scale to 250000+ object reads. The problem could of course be in your code, but if you are unable to find anything wrong, you'll have to do some deeper debugging to catch the one who is guilty and put him up against the wall Or you may accept that it takes about seven minutes, whatever you do, and stick to a single process.
|
|
|
|
|
trønderen wrote: Obviously, your DNS server (the one you are reading from) can only handle a single request at a time
No, that isn't the case. The domain servers can serve concurrent requests.
And anyway, this is only one request -- which returns 250000+ objects in 1000 object pages.
I did expect that the bottle neck would be in retrieving each object (or page), but my tests haven't shown that -- probably invalid tests.
All I'm seeing is that lock ( thingy ) (in C#) seems to be taking a lot of time. I have even tried to reduce the number of locks by locking once and reading 1000 objects before releasing the lock, but that didn't help either.
Anyway, this post isn't about trying to find out what I did wrong.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
