Click here to Skip to main content
15,867,308 members
Please Sign up or sign in to vote.
5.00/5 (1 vote)
See more:
I'm using C# 2010
I'm trying to figure out as much of this on my own as possible as I think that is the best way to learn something vs saying here is my code please fix it. What I'd like to be able to do (if possible) is to have multiple threads working on the same process. However in order to be able to do that I need to pass variables to that thread/process.
What I'm trying to do is write some files to disk, testfile1, testfile2, testfile3, etc... This runs ok on a single thread but as you get over 10 it starts to slow down and become noticable. I have the code working in a single thread fine, it's when I try to spread it out that I'm running into problems.
I'll keep trying things and searching around... thanks in advance!
Posted
Comments
MacRaider4 27-May-11 7:30am    
Update...
inside the main loop I have
ThreadWrapper tw = new ThreadWrapper(fullText, currentMail, ltrCount, totalMessages);

and then :
internal ThreadWrapper(string fullText, int currentMail, int ltrCount, int totalMessages)
{
Thread = new System.Threading.Thread(Body);
TextWriter tw = new StreamWriter("M:\\**Path**\\emailMessage" + currentMail + ".txt");
// write to the file
tw.WriteLine(Thread.CurrentThread.ManagedThreadId);
tw.WriteLine(fullText);

//close the stream
tw.Dispose();
} //ThreadWrapper

This is taking about 22 seconds to process all the files, doing it as part of the main loop takes about 22 seconds so no gain in performance.

Yes I realize not all the variables are being used, I just copied the call I was using which was using them all to make my test faster to run...
MacRaider4 27-May-11 8:57am    
Ok another update:
ThreadWrapper tw = new ThreadWrapper(fullText, currentMail, ltrCount, totalMessages);
//MailWrite(fullText, currentMail, ltrCount, totalMessages);
ltrCount++;

So I added a break point on the line with ltrCount++ to see what was going on to the best of my limited threading ability (we've all gotta learn some time). I'm noticing a "pause" with the longer writes as it does when I uncomment the call to mailWrite and comment out the top line (running on the main thread only). Maybe I'm expecting too much, but shouldn't it pass the work to the thread along with the variables at that point in time and then go on it's merry way (continuting on in the loop)? Or is there something else I need to do in order to get that functionality?
yesotaso 27-May-11 14:58pm    
It seems to me that you are shaking wrong tree... The IO itself is the bottle-neck, the idea of multi-threading is pipe-lining tasks so that with better usage of resources gaining performance, which would fail if one of tasks you try to pipeline takes %99.9 of time. Lets take an example, you copy 100MB of a file and copy it in ~10 secs, 2 consecutive copies would take ~20 secs but the fact is 2 concurrent copies would also take ~20 secs because of physical limitation which you can overcome only way : buy faster disk or buy motherboard with faster databridge DMA or whatever but it will cost dearly :)

Here you will find a complete source code showing how to do it. You create a thread wrapper where you can pass anything in or out and perform proper synchronization:

How to pass ref parameter to the thread[^].

If this is not quite clear for you how to apply the techniques to your particular problem, feel free to ask a follow-up question.

Just in case, please check the collection of my links to my past answers to the question if related topics:
How to get a keydown event to operate on a different thread in vb.net[^],
Control events not firing after enable disable + multithreading[^].

[EDIT]
Based or recent discussion: please take a look at the collection of links to my past answers on threading and see my recommendations:
How to get a keydown event to operate on a different thread in vb.net[^],
Control events not firing after enable disable + multithreading[^].

Hope they are useful.

—SA
 
Share this answer
 
v2
Comments
MacRaider4 26-May-11 14:35pm    
Ok, this is super simplified, however would something like this work on a "much larger scale"?
<pre>
for (int i = 0; i < 5; i++)
{
tw = new ThreadWrapper("Thread loop", i);
tw.Start();
}</pre>

tw has already been used at this point higher up in the code.
MacRaider4 26-May-11 15:12pm    
Alright, so I have checked and these are all running on the same thread...
This is in the ThreadWrapper:

writer.WriteLine(Thread.CurrentThread.ManagedThreadId);
writer.WriteLine(fullText);

And in each case it was a 9 in the first line for the files.
Sergey Alexandrovich Kryukov 27-May-11 14:32pm    
First, thanks for accepting this answer. Making a wrapper is really a robust way.
Why are you using ThreadId, just for experimental purposes? Best techniques are agnostic to what thread is doing what.

Also, about large scale: it would work perfectly, but the architecture should avoid unlimited number of threads. Even if number of threads is unknown during run-time, it should be defined pretty soon in the beginning of run-time (for example, it could be a parameter you put in some configuration file or pass to command line, etc.). Let me add one more link to my answer...
--SA
MacRaider4 27-May-11 14:37pm    
my reason for using ThreadId was just to see what "thread" was doing what. I actually figured out the problem I was having and now it's working as I would hope it to. The bottle neck was actually further up the chain, and unfortunatly one that I'll be stuck with, but not a big deal. I'll actually be able to use the information/knowledge gained here for a step a little further down the line in my current project.
Sergey Alexandrovich Kryukov 28-May-11 2:37am    
Well, good luck,
--SA
Seems this should give you the basics to create a thread method to write your file, then kick it off to create a seperate thread for each file passing what it needs in the object param. But, I think since your operation involved IO to physical disk, you may just run into a limitation based on the drives ability to handle write requests anyhow... ? Just a thought... good luck!

From msdn...

MSIL
using System;
using System.Threading;

public class Work
{
    public static void Main()
    {
        // To start a thread using a shared thread procedure, use
        // the class name and method name when you create the
        // ParameterizedThreadStart delegate. C# infers the
        // appropriate delegate creation syntax:
        //    new ParameterizedThreadStart(Work.DoWork)
        //
        Thread newThread = new Thread(Work.DoWork);

        // Use the overload of the Start method that has a
        // parameter of type Object. You can create an object that
        // contains several pieces of data, or you can pass any
        // reference type or value type. The following code passes
        // the integer value 42.
        //
        newThread.Start(42);

        // To start a thread using an instance method for the thread
        // procedure, use the instance variable and method name when
        // you create the ParameterizedThreadStart delegate. C# infers
        // the appropriate delegate creation syntax:
        //    new ParameterizedThreadStart(w.DoMoreWork)
        //
        Work w = new Work();
        newThread = new Thread(w.DoMoreWork);

        // Pass an object containing data for the thread.
        //
        newThread.Start("The answer.");
    }

    public static void DoWork(object data)
    {
        Console.WriteLine("Static thread procedure. Data='{0}'",
            data);
    }

    public void DoMoreWork(object data)
    {
        Console.WriteLine("Instance thread procedure. Data='{0}'",
            data);
    }
}

/* This code example produces the following output (the order
   of the lines might vary):

Static thread procedure. Data='42'
Instance thread procedure. Data='The answer'
*/
 
Share this answer
 
It's hard to tell if you have already got your process running in multiple threads or not from what you've said. You state it works fine in a single thread, but slows down after 10 or so... does that mean you are creating 10 files in a row on a single thread, or have managed to kick off 10 threads, each creating a file?
 
Share this answer
 
Comments
Sergey Alexandrovich Kryukov 26-May-11 13:58pm    
There are quite universal approaches to the problem.
Please see my answer and just the first reference to see how it's covered.
--SA
MacRaider4 26-May-11 14:06pm    
As of now it's just running in a single thread, writing 1 file, then the next, then the next.
Yvan Rodrigues 26-May-11 18:55pm    
Also keep in mind that more threads != faster. The increase in performance will be limited by the number of cores and the speed of your hard disk.
MacRaider4 27-May-11 7:26am    
It's got 2 Xeon X5650's at 2.67 and I think that drive is a RAID 10 (maybe 5, can't remember which drive is which off hand) with 15k RPM drives, should be able to handle a few writes at a time...

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900