Click here to Skip to main content
15,868,016 members

The Weird and The Wonderful

   

The Weird and The Wonderful forum is a place to post Coding Horrors, Worst Practices, and the occasional flash of brilliance.

We all come across code that simply boggles the mind. Lazy kludges, embarrassing mistakes, horrid workarounds and developers just not quite getting it. And then somedays we come across - or write - the truly sublime.

Post your Best, your worst, and your most interesting. But please - no programming questions . This forum is purely for amusement and discussions on code snippets. All actual programming questions will be removed.

 
GeneralRe: GDMA on the ESP32S3 Pin
honey the codewitch18-Mar-23 12:52
mvahoney the codewitch18-Mar-23 12:52 
GeneralRe: GDMA on the ESP32S3 Pin
raddevus18-Mar-23 12:59
mvaraddevus18-Mar-23 12:59 
GeneralRe: GDMA on the ESP32S3 Pin
honey the codewitch18-Mar-23 13:31
mvahoney the codewitch18-Mar-23 13:31 
GeneralI must be doing something wrong... Pin
PIEBALDconsult17-Mar-23 13:47
mvePIEBALDconsult17-Mar-23 13:47 
GeneralRe: I must be doing something wrong... Pin
0x01AA17-Mar-23 14:31
mve0x01AA17-Mar-23 14:31 
GeneralRe: I must be doing something wrong... Pin
trønderen17-Mar-23 14:57
trønderen17-Mar-23 14:57 
GeneralRe: I must be doing something wrong... Pin
PIEBALDconsult17-Mar-23 16:05
mvePIEBALDconsult17-Mar-23 16:05 
GeneralRe: I must be doing something wrong... Pin
trønderen17-Mar-23 17:17
trønderen17-Mar-23 17:17 
Obviously, your DNS server (the one you are reading from) can only handle a single request at a time. Somewhere behind the curtain, to do a read you must get hold of a semaphore. It may be so much behind the curtain that it is called a critical region or monitor, but in any case, it boils down to gaining control of a semaphore.

Maybe the semaphore handling (or region / monitor entering and leaving) really takes a lot of time, but your single-thread solution doesn't notice because it has obtained the data and is buy writing it to the database. Yet, the one(s) waiting in line wont obtain the semaphore until all the behind-the-scenes release work is completed.

If the waiting process has to poll for the semaphore, he won't get it immediately when it is freed. Maybe he comes a second or two later, and that time is lost. If it happens for every LDAP request, it adds up!

Note that everywhere but in the *nix community, programmers knew of queueing semaphore (so you didn't have to poll), regions and monitors based on queueing semaphores, from the mid/late 1970s. *nix had nothing like it, except "Let us create a file, whose existence indicates 'resource reserved'!" It is an extremely resource demanding semaphore, compared to OS implementations, and it is binary (queue-less), so you have to poll it. *nix programmers grew up without knowing anything about proper synchronization. When finally OS based semaphores where introduced to *nix, they were first binary; you still had to do the polling. And *nix programmers were very reluctant to start using them. Even today, it seems like young programmers know very little about proper use of semaphores, regions, and monitors, and regularly use binary semaphores as something like a voluntary Post-It: 'Please be so kind not to touch these data while I am working on them, will you?' Both putting up the Post-It and for others to read it is sort of voluntary. (I learned to build non-voluntary, efficient queueing semaphores, critical regions and monitors from the 1973 Brinch Hansen book 'Operating System Principles - it is all there, from 50 years ago. Obviously, not a single *nix programmer I have ever met has as much as heard of that book. Or any other describing the same solutions.)

So my guess is that you have come across a really poor implementation of resource protection, probably based on polling a binary semaphore. When the second process finally gets around to grabbing it, the first one is almost ready to enter his next round, a split second too late; #2 tool the semaphore. So now #1 is polling, at long intervals to keep the CPU load moderate, and comes maybe a second after the semaphore is freed. Wasted time. Wasted work. And the play repeats itself a few hundred or thousand times.

The process switching also takes resources: There is a whole lot to do when #2 takes over for #1 or the other way around, and it happens not only when the semaphore is grabbed, but every time it is polled. The timing loop for the polling takes time. Maybe not only your machine must do heavy context switches; it could be the same on the other side, the domain controller.

This semaphore polling is not necessarily in your code; it may lie in some library routine or in the LDAP implementation or somewhere else. After all, most LDAP accesses look up a single or a few objects; a short delay caused by the need for polling is acceptable. It just doesn't scale to 250000+ object reads. The problem could of course be in your code, but if you are unable to find anything wrong, you'll have to do some deeper debugging to catch the one who is guilty and put him up against the wall Smile | :) Or you may accept that it takes about seven minutes, whatever you do, and stick to a single process.
GeneralRe: I must be doing something wrong... Pin
PIEBALDconsult18-Mar-23 6:15
mvePIEBALDconsult18-Mar-23 6:15 
GeneralRe: I must be doing something wrong... Pin
GuyThiebaut19-Mar-23 22:02
professionalGuyThiebaut19-Mar-23 22:02 
GeneralRe: I must be doing something wrong... Pin
Daniel Pfeffer19-Mar-23 23:07
professionalDaniel Pfeffer19-Mar-23 23:07 
GeneralRe: I must be doing something wrong... Pin
PIEBALDconsult20-Mar-23 2:50
mvePIEBALDconsult20-Mar-23 2:50 
GeneralRe: I must be doing something wrong... Pin
Daniel Pfeffer20-Mar-23 4:26
professionalDaniel Pfeffer20-Mar-23 4:26 
GeneralRe: I must be doing something wrong... Pin
Bohdan Stupak19-Apr-23 4:31
professionalBohdan Stupak19-Apr-23 4:31 
GeneralChatGPT "solves" a riddle for me. Pin
GKP199212-Mar-23 22:15
professionalGKP199212-Mar-23 22:15 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
Jörgen Andersson13-Mar-23 0:13
professionalJörgen Andersson13-Mar-23 0:13 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
Ron Anders13-Mar-23 1:45
Ron Anders13-Mar-23 1:45 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
jschell20-Mar-23 7:03
jschell20-Mar-23 7:03 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
PIEBALDconsult13-Mar-23 5:34
mvePIEBALDconsult13-Mar-23 5:34 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
PIEBALDconsult13-Mar-23 13:34
mvePIEBALDconsult13-Mar-23 13:34 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
adriancs6-Apr-23 1:15
mvaadriancs6-Apr-23 1:15 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
adriancs6-Apr-23 1:26
mvaadriancs6-Apr-23 1:26 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
DrWalter PE26-Apr-23 18:38
professionalDrWalter PE26-Apr-23 18:38 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
GKP199226-Apr-23 20:26
professionalGKP199226-Apr-23 20:26 
GeneralRe: ChatGPT "solves" a riddle for me. Pin
Amarnath S18-May-23 1:08
professionalAmarnath S18-May-23 1:08 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.