|
I'm looking to add threading support to an application I'm working on, and am looking for advice on how to accomplish this.
I have a collection of objects which are updated through a foreach loop. Instead, I'd like to dedicate this to a series of threads, where each object is moved in for processing as the thread becomes available.
I've come up with 2 approached on how to do this.
Method 1: Using a WaitHandle
Give each thread an AutoResetEvent and rely on the WaitHandle.WaitAny() method to identify when a thread is finished so a new one can be created.
Link: MSDN[^]
Pseudocode:
void QueueThreads(IEnumerable myCollection, uint maxThreads)
{
uint usedThreads = 0;
AutoResetEvent[] autoEvents = new AutoResetEvent[Math.Min(maxThreads, myCollection.Count)];
foreach (T myObject in myCollection)
{
if (usedThreads < maxThreads)
{
autoEvents[usedThreads] = new AutoResetEvent(false);
usedThreads++;
}
else
{
uint freeThreadID = WaitHandle.WaitAny(autoEvents);
usedThreads--;
usedThreads++;
}
}
WaitHandle.WaitAll(autoEvents);
}
void DoWork(myObject, AutoResetEvent are)
{
are.Set()
}
Method 2: Sharing the enumerator across threads
Give each thread access to the enumerator used to iterate over the collection. Each thread stays alive until all items in the collection have been exhausted.
Pseudocode:
void QueueThreads(IEnumerable myCollection, uint maxThreads)
{
uint usedThreads = 0;
IEnumerator myCollectionEnum = myCollection.GetEnumerator();
for (uint i = 0; i <= Min(myCollection.Count,maxThreads); i++)
{
}
myCollectionEnum.Reset();
}
void DoWork(IEnumerator myColEn)
{
do
{
T myObject;
lock (myColEn)
{
if (myColEn.MoveNext())
{
myObject = myColEn.Current
}
else
{
break;
}
}
}
}
My experience with threading has been only experimental, so I'm hoping the CP threading vets can provide input on which approach is best, or if there's a third option worth considering.
As for starting a thread, I’ve seen a few different methods for doing this (ThreadPool.QueueUserWorkItem(), new Thread(new ThreadStart()), BackgroundWorker) , but don’t really know when to use which one.
Note that file/database IO would need to be synchronized between the threads, but aside from that, there shouldn't be any issued with shared resources as each object maintains its own state.
Also, I am restricted to using .Net 2.0.
modified on Thursday, September 15, 2011 2:37 PM
|
|
|
|
|
The question to ask would be why you are trying to use multiple threads, what are you expecting to gain from it? If it is increased performance you may not get what you are looking for since the bottle neck would most likely be at the file IO or database level.
I know the language. I've read a book. - _Madmatt
|
|
|
|
|
Essentially, this process would be to collect data from a list of networked machines. Most of the latency comes from locating the machine and then establishing a connection. The threading is so I could continue to process other machined while one is waiting to connect.
|
|
|
|
|
IIRC the number of concurrent TCP/IP connections you can establish is limited to a few tens, which wouldn't be a problem, unless you try and connect to a lot of machines that are absent or off, as those connection attempts would hit a time-out (possibly 1 minute).
BTW: using lots of threads for actions that are mostly blocking (waiting on something) rather than computing, is quite wasteful. A better approach could well be to apply asynchronous operations (similar to serving all WinForms Controls from a single thread).
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
Yes, I do expect that I will get a timeout for many of the connections. I'm starting with 10 threads, but will probably adjust that number once I get a better idea how the threads perform.
Luc Pattyn wrote: BTW: using lots of threads for actions that are mostly blocking (waiting on
something) rather than computing, is quite wasteful. A better approach could
well be to apply asynchronous operations (similar to serving all WinForms
Controls from a single thread).
There's a difference?  I've always understood "asynchronous programming" to be just a different term for threading.
|
|
|
|
|
Of course there is a difference, a rather big one.
In multi-threading you write code in sequential mode: lets do A, then wait for B, then do C, then wait for D, etc. That is easy to write as the program counter is your main state variable; now each thread needs a stack (so it can remember where it is when nesting functions/methods), and the operating system has to switch threads all the time.
In asynchronous operation, you could have just one thread; the workload is split in small jobs, and the thread executes these jobs one at the time, to completion. That is how Windows does a WinForms GUI: you have a button handler, a paint handler, etc. Now if you need to split a long-winding operation over multiple handlers, you need some state variables, you basically build a state machine, which is event-driven, not progressing based on the program counter.
Consider a serial port receiver: with its own thread, you can write like: now I expect this, then I wait, then I should get some acknowledge, then wait, then get a length, then some amount of data, etc. With a DataReceived handler, your handler has to reflect on "what am I getting know" and "where was I in the protocol", and make those match up.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
Yes. it seems that the asynchronous network access in the framework (Socket.BeginReceive etc) is the way forward here.
|
|
|
|
|
Hi,
here are my thoughts on the matter:
1.
I watched the amount of state variables you were using. Method #1 tries to keep track of the number of threads, and needs a number of AutoResetEvents. Method #2 uses an enumerator and a lock, to me that is much simpler to write, to understand, to debug, and hence the preferred way of them both. With the API you have chosen (taking an IEnumerable myCollection) it probably is the best one can do.
Not tested: If myCollection were an IList, you wouldn't need the enumerator and could simply use indexing, which also implies you wouldn't need the lock, a simple Interlocked.Increment() would suffice.
All the above is assuming your collection does not change while the jobs are being executed; if the workload is dynamic, I recommend using a real queue, and a lock of course.
2.
choosing the kind of thread should be easy: BackgroundWorker is based on ThreadPool; if the progress and completion features are appealing, prefer BGW over TP, otherwise don't. There is however one big caveat: ThreadPool has it's own mind about how many threads will actually be used: when you queue 20 jobs, you won't necessarily get 20 threads from the pool right away (other things may be going on in the ThreadPool); when the pool is too busy, it will get extended (within limits) and there is a complex algorithm in place that extends the pool with up to 2 threads per second if necessary; I don't now exactly what rule applies to retiring threads from the pool. This really boils down to your uint maxThreads parameter possibly be of limited value to what is going to really happen. Using instances of Thread puts you fully in charge. On top of that, whatever you do, determining what the optimal number of threads would be isn't very easy as they need to be mapped onto the number of cores in the system. I typically launch no more than N (and never more than 2*N) identical jobs on a system with N cores (see Environment.ProcessorCount).
3.
The value of multi-threading is greatest when the performance limiting factor is computational; threads add processing power to your app, as long as you don't exceed the processor count (i.e. typically Task Manager shows less than 99% CPU load). Quite often adding threads to an app moves the performance bottleneck to some other part of the system, maybe the cache efficiency goes way down and the memory bus bandwidth becomes critical; maybe the disk or network bandwidth becomes the major factor, etc. I recommend making the number of threads a variable, and experimenting with that number while watching total CPU load; as soon as it stops rising, adding threads won't help any further.
Final note: you may not need the actual optimum; there is the rule of diminishing returns, so going from 1 to 2 threads is the biggest step forward (or not!); going from 2 to 4 is a lot easier (you already have the code in place), however it may be much less rewarding. You could easily determine the empirical optimum on a given system; in my experience referring to Environment.ProcessorCount is a good predictive way to deal with varying hardware.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
I never really needed to so, I have never actually used a ListView in my professional life. Now I have a legacy control (legacy developer is still here) that uses a ListView control to allow users to easily select/deselect multiple rows using the checkbox feature. I am not required to add 2-3 additional, editable columns, one of which was requested to be a combobox or checkbox, the other an editable text cell. The ListView is not bound to data from a database, rows are added dynamically.
My question is, do you guys think it's worth it to keep it a ListView, or should I break legacy and use a DataGrid? It seems that adding these columns to do what I want will be easier in the control I'm more familiar with, but it is possible with both.
Craigslist Troll: litaly@comcast.net
"I have a theory that the truth is never told during the nine-to-five hours. "
— Hunter S. Thompson
|
|
|
|
|
based on my limited short experience , I have found that the users here are against changes that look and feel different to them, regardless of the functionality.
I would see who has final say on the design and put it past them to see what is said. Making unexpected changes to the GUI could result in heavy blowback from unhappy users who are kinda dim but are married to the big boss 
Programming is a race between programmers trying to build bigger and better idiot proof programs, and the universe trying to build bigger and better idiots, so far... the universe is winning.
|
|
|
|
|
If the users only have an option to view the ListView in detail view then I would think you have a good argument to change to a DataGrid. Not only for your current situation but it would also be more extensible. If the users can view the ListView in different Views I would tread carefully. Maybe it will be alright, maybe not.
"You get that on the big jobs."
|
|
|
|
|
wizardzz wrote: I am not required to add 2-3 additional, editable columns, one of which was requested to be a combobox or checkbox
That requirement just blew the listbox out of scope, you cannot host a combobox in a listbox, only a grid.
Listbox is an excellent read only tool for well lists. Use a datagridview for anything more complex.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
ListView does not support ComboBox columns, but you can create your own control based on ListView and add the features you need, but that would be too time consuming. In short, go for DataGridView.
|
|
|
|
|
Thank you everyone. After taking everything into account and discussing with the previous developer, I have decided to switch to drop the ListView. There wasn't really a right answer, but thanks for all the input.
Craigslist Troll: litaly@comcast.net
"I have a theory that the truth is never told during the nine-to-five hours. "
— Hunter S. Thompson
|
|
|
|
|
how to detect programmatically that mouse pointer is placed on windows title bar and also detect when move out from title bar with c#.
tbhattacharjee
|
|
|
|
|
The standard Form object doesn't expose any events for the non-client area of a form, except for border resizing. You'd have to create your own version of the Form class, override the WndProc method and parse any WM_NCHITTEST messages you get. From here, you can raise your own custom defined events in your new form class.
You can read more on the mouse messages here[^].
You can read up on more examples of this in these Google results[^].
|
|
|
|
|
How to extract from array items at specific inidces?
List<int> ints;
int[] indices;
List<int> slected = ints select items at 'indices';
Чесноков
|
|
|
|
|
I don't see a problem here.
List<int> ints;
int[] indices;
List<int> slected=new List<int>();
ints=...;
indices=...;
foreach(int index in indices) slected.Add(ints[index]);
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
I was thinking about 1 line of code as in LINQ
selected = ints.Select(i => i[ints]);
Чесноков
|
|
|
|
|
Once you got it I'll be curious to see it. In the mean time I just use foreach(int index in indices) slected.Add(ints[index]); which is one line of code anyway.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
You can use the Where clause ie
private void indices()
{
List<int> listInts = new List<int>() { 1, -23, 45, -723, 29, 49, -90, 628, 476, 39, -10, 30 };
int[] indices = new int[] { 3, 5, 7, 9 };
var selected = listInts.Where((i, index) => !indices.Contains(index));
}
That will do it.
...and I have extensive experience writing computer code, including OIC, BTW, BRB, IMHO, LMAO, ROFL, TTYL.....
|
|
|
|
|
It may be compact and correct, but I don't like it much as it is an expensive, quadratic, operation: it will look for each element in the list of indices.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
As I stated, it will work, whether it is efficient is a completely different story. I prefer your method to be honest, but I also like to keep my hand in with these new fangled things MS keeps bringing in. Some of them are actually plenty useful, sometimes, but like all these abstractions that are creeping into our job, supposedly to make our lives easier, you have to weigh up the various options.
...and I have extensive experience writing computer code, including OIC, BTW, BRB, IMHO, LMAO, ROFL, TTYL.....
|
|
|
|
|
thanks for the hint, would you be able to solve assignement in one statement?
there is array of ints and array of objects with public int MyValue property.
the number of elements in both arrays are equal.
How to assign those all ints to every object MyValue in one pass?
for (int i = 0; i < ints.Length; i++)
objects[i].MyValue = ints[i];
Чесноков
|
|
|
|
|
Why on earth are you trying to solve this in one statement? The more you try to achieve that, the harder it's going to be for you to remember what your code actually does and how it works.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
