Thread Creation/Destruction overhead

Question

0.00/5 (No votes)

See more:

Hi,

I am developing an application where application suppose to process the data of records (ranges from 1 - 500000). For each block i am creating threads,

If i am creating /destructing threads for at least 11k times, then .... is it the overhead ??

Thanks in advance.
-Prafulla

UPDATE :

I tried with creating thread at once. But it is adversaly affecting the performance. It is taking more time than creating threads at each time.

Is ther any thread specific memory related issue ???

NEED HELP

Posted 20-Jun-11 21:39pm

PrafullaVedante

Updated 22-Jun-11 21:16pm

v2

Add a Solution

3 solutions

Solution 2

Yes creating and destroying threads do have an overhead, perhaps the better option is to have one manager thread and then have worker thread to which you assign the work to from the manager thread.

hence you only create threads at the start of the program and have the wait on a message telling them which records to process.
Once done they send a message to the manager thread telling it they are done and want the next set of records to process.

Posted 21-Jun-11 3:03am

Guyverthree

Comments

KarstenK 21-Jun-11 9:59am

He needs a global queue which is thread safe to distribute the queued jobs.
The count of threads should be balanced which count is optimal to perform. Timeouts, parallel tasks and dead locking are problems.

Guyverthree 21-Jun-11 10:06am

Yes that is a fair point. The gobal queue is a must.

I was assuming that he must have thought about timeouts, parallel tasks and dead locking if he is creating 11K worth of threads already and wants the task to be completed successfully.

KarstenK 21-Jun-11 10:12am

my tip is that 10 threads are enough, so 100 wont be much faster if he isnt populating a server farm.

Threading isnt the solution, but intelligent and clever code ;-)

Sergey Alexandrovich Kryukov 21-Jun-11 16:35pm

Agree, my 5.
I added my 2 cents in my answer.
--SA

Solution 3

Very unreasonable overhead. You should re-think your design totally. There are could be different reasons for using threads. Do you need to more fully use your CPUs/cores (but then you hardly need more CPUs than cores) or thread represent logical entities of your application (such as in the famous problem of dining philosophers).

It is very likely that you can use one of a fixed small number of threads. You can also use the Thread Pool pattern, see http://en.wikipedia.org/wiki/Thread_pool_pattern[^].

—SA

Posted 21-Jun-11 10:35am

Sergey Alexandrovich Kryukov

Comments

Guyverthree 22-Jun-11 4:17am

Agree 5 :)

I didn't know about design patterns but they are obviously very very useful.

Sergey Alexandrovich Kryukov 22-Jun-11 4:18am

Thank you, Safar.
--SA

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CPallini · Accepted Answer · 2011-06-20T21:47:00

Solution 1

Probably yes. You should arrange a little test to verify execution times. You may also have a look at Thread Pool Pattern[^].

Posted 20-Jun-11 21:47pm

CPallini

Comments

Albert Holguin 21-Jun-11 9:19am

my 5, thread creation overhead can be significant, if there's a case where you have to create multiple threads, you're better off creating them once and tasking them.

Joan M 21-Jun-11 9:25am

5ed!

Guyverthree 22-Jun-11 4:21am

Agree 5

I didn't know about design patterns but they are obviously very very useful.