|
No no no no no.
Read what I wrote. The time lag between the writefile in one thread, and the writefile in the other often exceeds 4 seconds, and the only thing happening between them is a setevent and a thread schedule.
By the way overlapped IO is not as quick as direct, and the fact I have threads obviates the need for it anyway.
|
|
|
|
|
You should show all applicable code (CreateEvent, thread initialization, any other code in the threads).
Munchies_Matt wrote: Er, ever used Windows 7?
I've been working with high-speed software for about a decade. In both Windows and Linux.
Munchies_Matt wrote: It just looks like really crap user mode thread scheduling (this is win 7 64, the app is win32).
No, it's not. You have another issue. As was already mentioned, it is likely one of the other blocking calls you're making. You can easily set up a test to see how fast events are triggered and release waits with a timer. You'll find it's not very long.
|
|
|
|
|
Albert Holguin wrote: You should show all applicable code (CreateEvent, thread initialization, any other code in the threads).
I gave you the CreateEvent code, there aren't any other threads (except for the main thread, the event is not waited on or used anywhere else than shown, and the thread initialisation is with the usual CreateThread() call:
trHR = CreateThread(NULL,0, &ThreadReadProc, this, 0, 0);
Albert Holguin wrote: You have another issue. As was already mentioned, it is likely one of the other blocking calls you're making
As I told the other guy, no, this isn't that case. As you can see from the code the read file completes, a write file is done, and an event signalled. The waiting thread only does a write file.
How much more simply can it be explained to you? You have seen the code, and moving the second write file into the same thread as the first write file results in no delay at all demonstrating that the fault lies in the thread scheduling.
I also don't need a timer to tell me anything, I have a USB bus analyser running and tracing from my driver thats telling me its taking over 4 seconds for the waiting thread to get scheduled.
|
|
|
|
|
Munchies_Matt wrote: How much more simply can it be explained to you? You have seen the code, and moving the second write file into the same thread as the first write file results in no delay at all demonstrating that the fault lies in the thread scheduling.
You're the one that needs help, not us. I suggest you listen to suggestions and think about them instead of being negative.
Munchies_Matt wrote: I have a USB bus analyser running and tracing from my driver thats telling me its taking over 4 seconds for the waiting thread to get scheduled
That's telling you there's problems in your code, not that there's problems with the thread handling in Windows.
|
|
|
|
|
Albert Holguin wrote: You're the one that needs help
No I don't. Just pointing out the crap that is Microsoft.
|
|
|
|
|
Then why are you posting this to Q&A?
I can tell you that signaling isn't slow, you have something wrong with your code... but you somehow refuse to listen. I suggest you set up a test scenario and figure out what's wrong with your code before blaming something that's been around and widely used for years.
|
|
|
|
|
Dot you think that well inside 4 to 8 seconds the thread with the read file should get de-scheduled and the one with the write file scheduled within a reasonable time of the wait being satisfied?
Clearly it isn't. That's the problem. I shouldn't have to put the read file thread to sleep, it should be descheduled automatically.
|
|
|
|
|
Oh, and I just put the code into one thread, which removes the wait and set event, and it runs without any delay at all, of course.
modified 24-Feb-15 6:11am.
|
|
|
|
|
Do you have the read and write calls reading/writing from the same place? ...you probably have a thread deadlock most of the time.
|
|
|
|
|
You have seen the code I supplied, it really is as simple as it looks:
There are two write files, in two different threads.
Both are synchronous write files.
When the first write file completes it and sets an event and goes into a read cycle.
The second write file is in a thread waiting for the event to get signalled.
The time between the two write files is often many seconds. What is happening during that time is a setevent and a wait being satisfied.
|
|
|
|
|
Where the system spend the delay time: before or after the WaitForSingleObject or SetEvent?
Looking to the data over USB from external analyzer doesn't tell you.
How can you be so sure that the delay come from the thread synch if you don't know when it gets out of it?
This is what people here are asking you.
|
|
|
|
|
I don't know, I only have the external events; the two write files, to go by.
Frankie-C wrote: How can you be so sure that the delay come from the thread synch if you don't know when it gets out of it
Why would a thread schedule, look at a while statement, and then either stall before the write file, or somewhere inside it?
You cant honestly suggest the thread timeslice is in the order of seconds; the thread doing the read file should be descheduled and the recently waiting thread should be run far quicker than that.
|
|
|
|
|
I've been not very clear. I didn't meant the thread scheduling, but the WaitForSingleObject.
I'll try to clarify my question: are you sure that your code is in the WaitForSingleObject waiting for the event, and when the event is set the code take a walk around and after some time remember that it was waiting for something and proceed? Or maybe the execution is wasted in different instructions (i.e. the file read or write that are in your code) while the event is set, then getting out few seconds later executes the WaitForSingleObject find the event set and jumps out immediatley (as is exepected from WaitForSingleObject), bat late anytime?
The point is that the wait for objects is the base of synch for all software in windows, should it behave as you say nothing should work. That is the reason why we believe that the source of delay should be somewhere else (driver delays, timeouts or else).
Then for the quality of MS code there is a lot of space for discussions... 
|
|
|
|
|
I understood, lets go back to the code:
As you say is the write thread getting stuck in the wait, or in the write.
The thing is, in either case, since both threads are of equal priority, the read thread Write thread:
while(pDlg->Go)
{
WriteFile(gh, buf, towrite, &written, 0);
WaitForSingleObject(pDlg->WriteTrigger, INFINITE);
}
Read thread:
while(pDlg->Go)
{
ReadFile(gh, buf, 1024 - readTot, &read, 0);
... process data, snipped
WriteFile(gh, &ACK, 1, &written, 0);
SetEvent(pDlg->WriteTrigger);
}
should get descheduled and the write thread scheduled in an even way, they should both get CPU time equally.
What looks to be happening is that the read thread is having a lot of CPU time (its a multi core CPU) and blocking the handle at an IO level thus stalling the write. What I find really surprising is that the read thread can run for up to 8 seconds.
The solution would be for the read thread to wait on the event immediately after it sets it, and then for the write thread to set it and wait immediately:
Write thread:
while(pDlg->Go)
{
WriteFile(gh, buf, towrite, &written, 0);
SetEvent(pDlg->WriteTrigger);
WaitForSingleObject(pDlg->WriteTrigger, INFINITE);
}
Read thread:
while(pDlg->Go)
{
ReadFile(gh, buf, 1024 - readTot, &read, 0);
... process data, snipped
WriteFile(gh, &ACK, 1, &written, 0);
SetEvent(pDlg->WriteTrigger);
WaitForSingleObject(pDlg->WriteTrigger, INFINITE);
}
But that is an ugly piece of code to look at! 
|
|
|
|
|
Yes ugly indeed, you could consider using Sleep() with a very short timeout to suspend current thread of execution. This should fix back the synchronization.
|
|
|
|
|
Sleeps arent performance optimal though, and this test app is designed to hammer some hardware hard for a day or so, so I need it to run as quick as it can.
Anyway, I put the second write into the same thread as the read and its running OK now of course.
|
|
|
|
|
I see, my suggestion was to put a minimal delay of 1ms to use the side effetct of suspending thread execution. Anyway the solution to avoid handle concurrency is the fastest per sure.
|
|
|
|
|
Yeah, triggering a small sleep might be enough to get the thread off the CPU for long enough.
|
|
|
|
|
I am using DeviceIoControl with IOCTL_DISK_GET_DRIVE_LAYOUT_EX to get drive information because I need physical offset of drive.
I am getting wrong result so I did some analyses and found that my all drives are of type 0x42 (Basic Dynamic Drive). It means this control code does not give correct information for Dynamic Drives.
Can you help me to get information about drives?
|
|
|
|
|
I try to implement classical sorting algorithm, I do not understand why my quicksort is slower than my mergesort. I spend a lot of time trying to understand why. Do you have any idea ?
For both :
#define ARRAY_INDEX(tab, i) ((void*)((char*)(tab) + (i) * type_size))
My mergesort :
void merge(void *tab, void *aux, int a, int b)
{
int i, j, k, mid;
ft_copy_tab(tab, aux, a, b);
mid = (a + b) / 2;
i = a;
j = mid + 1;
k = a;
while (k <= b)
{
if (i > mid)
ft_copy_index(tab, aux, k, j++);
else if (j > b)
ft_copy_index(tab, aux, k, i++);
else if (f_cmp(ARRAY_INDEX(aux, i), ARRAY_INDEX(aux, j)) > 0)
ft_copy_index(tab, aux, k, j++);
else
ft_copy_index(tab, aux, k, i++);
k++;
}
return ;
}
My quicksort :
void ft_quicksort(void *tab, int a, int b)
{
int i, lower, upper;
if (a >= b)
return ;
lower = a;
upper = b;
i = a;
while (i <= upper)
{
if (f_cmp(ARRAY_INDEX(tab, lower), ARRAY_INDEX(tab, i)) > 0)
{
ft_swap(tab, lower, i, type_size);
lower++;
i++;
}
else if (f_cmp(ARRAY_INDEX(tab, lower), ARRAY_INDEX(tab, i)) < 0)
{
ft_swap(tab, i, upper, type_size);
upper--;
}
else
++i;
}
ft_sort(tab, a, lower - 1);
ft_sort(tab, upper + 1, b);
}
|
|
|
|
|
Is your list already (partially) sorted? What element are you using for the pivot? Does your list have a lot of repeated elements?
As for the data structures themselves, both take O(n log n) comparisons in the average case. In the worst case, quicksort takes O(n^2) comparisons, whereas mergesort takes O(n log n) comparisons.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"You can easily judge the character of a man by how he treats those who can do nothing for him." - James D. Miles
|
|
|
|
|
My list is not sorted at all :
void *ft_init_array_int(int len)
{
int *tab;
srand(time(NULL));
if (!(tab = (int*)malloc(len * sizeof(int))))
return (NULL);
while (len--)
tab[len] = rand();
return ((void*)tab);
}
I have noticed that if I my list has a lot of duplicate keys ("tab[len] = rand()%10" instead of "tab[len] = rand()"), my quicksort is much faster than my mergesort.
I use the first element for the pivot. I wanted to have consistent result first and then improve my mergesort.
For instance, it takes 7 seconds or so for my mergesort to sort 10.000.000 integers and around 13sec for my quicksort (qsort from stdlib.h takes 4sec). Does that mean that my algo (without improvement) does not work well enough ?
|
|
|
|
|
If your implementation "only" takes about 3 times as long as the stdlib qsort, then you can be reasonably sure that, technically, you did it correctly.
As for why your program takes longer, there are quite a number of things that can go wrong in performance optimization. A good start would be avoiding shortcuts and explicit optimization, even if it may sound counter-intuitive. The compiler is exceptionally good at optimizing code without your help, but to do this efficiently, it needs to see the "real" code, not some byte manipulation that it can't make heads or tails of.
One example is your #define: all it does is accessing array members, but all the compiler sees is some weird pointer arithmetic. Do leave the compiler arithmetic to the compiler, and you'll find that at the very least you'll get more readable code at no performance loss, and maybe you'll even see an improvement. I think I understand what you use this define for, but if I'm right, all you really need is a typecast from void* to int* (or pointer to whatever element type you're using)
Another issue may be the various helper functions you're calling: without the actual type information, these functions may be very inefficient. E. g. a function to swap two ints only requires a handful of assembler instructions. A function to swap two arrays of n bytes, where n is a run time variable, requires about three to five times as much.
You could check your compiler's aqssembler output to see what it makes of your code. Try different variants to find out what the compiler makes of it. Maybe that can give you some insight into what is causing the performance issue.
GOTOs are a bit like wire coat hangers: they tend to breed in the darkness, such that where there once were few, eventually there are many, and the program's architecture collapses beneath them. (Fran Poretto)
|
|
|
|
|
Thank you for your exhaustive response, your answer has risen a few more interrogations. I will work on the subject and I will come back to you as soon as possible.
|
|
|
|
|
Problem: This is our first programming assignment. This assignment has two parts:
• Part 1: I ask you to write a program to build a decision tree using Gini impurity measurement to guide tree generation. The data set is the poker hand data set archived at UCI Machine Learning Repository: Poker Hand Data Set
Data Set
Characteristics: Multivariate Number of Instances: 1025010 Area: Game
Attribute
Characteristics: Categorical, Integer Number of Attributes: 11 Date Donated 2007-01-
01
Associated Tasks: Classification Missing Values? No Number of Web
Hits: 212827
You shall use the training data set to build your decision tree and then use the testing data set to evaluate your decision tree. You need to report classification accuracy using a bar chart and compare it with the distance based classification which is given in Part II.
• Part II: For this part, I ask you to use the same training data set in Part I to build a distance-based classification model. Here, you need to find a good distance metric and a parameter k that serves as the threshold to bound the nearest neighbors for any given data item (or point). Then, you need to apply your model to the testing data set to evaluate your classification model. You shall record the classification accuracy and compare it in a
bar chart with that of the decision tree model built in Part I.
Programming Language:
C++ or Java, but C++ is preferred.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
