How can we be sure that the number of iterations of commands in "parallel.for" matches the number of iterations that were set for it?

Question

0.00/5 (No votes)

See more:

, +

Hi guys,
In my article, I want to compare the speed of sequential processing to parallel processing in various conditions, so in one example, I want to convert the following sequential command to parallel, but after converting it to parallel, it seems that the loop is not repeated for the specified number of times. It may be necessary to use Concurrent Collection or a "lock" statement, but this usually reduces the speed of parallel processing compared to sequential processing.

I am aware that variables are shared among multiple threads. This means that multiple threads can read from and write to this variable concurrently, which may cause race conditions and other unpredictable behavior.

I want an example (preferably a simple example) for my article that compares for and Parallel.For. This example should have the following features:
• The execution of Parallel.For block codes should be faster than the execution of for block codes.
• Using a flag, a variable, or other mechanisms, ensure that the Parallel.For loop is executed the predefined number of times.

I use the following tools:

• .NET Framework 4.5

• Console app

• Windows 7

Thank you for your time.
Best regards

What I have tried:

C#

static long Counter = 0;
public static long Calcute()
{
    int Total = 0;
    for (int i = 0; i < 30; i++)
    {
        Total = 20 / 4;
        Total = 20 * 4;
        Total = 20 ^ 4;
        Total = 20 - 4;
        Total = 20 % 4;
    }
    Counter += 30;
    return Counter;
}

Sequential:

C#

static void Main(string[] args)
{
    Stopwatch SW = new Stopwatch();
    Console.ReadKey();
    SW.Start();
    for (int i = 0; i < 50000000; i++)
    {
        Calcute();
        Calcute();
        Calcute();
        Calcute();
        Calcute();
    }
    SW.Stop();
    Console.WriteLine("Total seconds: " + SW.Elapsed.TotalSeconds + ", Counter: " + Counter);
    Console.ReadKey();
}

Result (click here to see):
Counter value=50000000*30*5=7,500,000,000

Parallel:

C#

static void Main(string[] args)
{
    Stopwatch SW = new Stopwatch();
    Console.ReadKey();
    SW.Start();
    Parallel.For(0, 50000000, (int i) =>
    {
        Calcute();
        Calcute();
        Calcute();
        Calcute();
        Calcute();
    });
    SW.Stop();
    Console.WriteLine("Total seconds: " + SW.Elapsed.TotalSeconds + ", Counter: " + Counter);
    Console.ReadKey();
}

One of the results (click here to see):
Counter value=50000000*30*5= different values are displayed

Posted 16-Dec-22 3:56am

Reza jafery

Updated 21-Dec-22 6:33am

v7

Add a Solution

Comments

PIEBALDconsult 16-Dec-22 11:56am

0) I recommend against having your E-mail addresses in your profile.
1) A Stopwatch won't be very accurate for very short periods of time
2) Optimization may also be affecting your results -- be sure you have optimization turned off or change your code to avoid issues caused by optimization.
3) Are you saying that you are writing an article on a subject in which you do not feel that you are an expert?

2 solutions

Solution 1

Multithreading isn't a magic bullet: it can't just speed up your code and give you massive speed improvements, and it certainly won't give you repeatable results every single time.

The reason why is simple: you have N cores on your processor: 1, 2, 4, 8, 16 or possibly even 64.
Each thread needs a spare core to run on, and if a core isn't available then the thread goes into a wait queue until one is. The more threads you create, the more threads have to wait for a free core.
the rest of the applications (and OS) share those cores, so the load at any particular microsecond is dependant on all the cores and all the threads waiting for them.

And each thread needs a stack, normally 4MB for a 64 bit process.

To make matters worse, setting up and switching threads takes processor (and thus core) time! The shorter the thread runtime, the more noticeable this becomes.

Start loading up the thread count, and the total processing time for your app can rise, particularly if you start to get low on memory as it gets paged to disk (virtual memory) so that other threads can run.

All of this adds up to a nondeterministic system: you cannot predict how long it will take to complete because it depends to a huge degree on external factors beyond anyone's control (Windows is not and never has been a real-time OS).

And you are trying to start 50,000,000 threads? You aren't going to get consistent results no matter how hard you try!

Posted 16-Dec-22 4:25am

OriginalGriff

Comments

Reza jafery 16-Dec-22 10:33am

So you don't think there's any way to solve the following? So, why was the parallel loop created?
• The execution of Parallel.For block codes should be faster than the execution of for block codes.
• Using a flag, a variable, or other mechanisms, ensure that the Parallel.For loop is executed the predefined number of times.

Reza jafery 16-Dec-22 11:05am

Even increasing the speed is useless if the parallel loop commands are not repeated for the specified number of times; thus, I try to create an approach that guarantees the loop executes for the specified number of times.

OriginalGriff 16-Dec-22 11:16am

The loop is executed the same number of times, but the time that takes is nondeterministic when you start involving multithreading (technically, that's also the case for the single thread app, but unless the rest of your system gets busy you won't notice it)

You don't count the number of iterations, you just time the whole loop. And that lack of determinism shows up as a "range of values".

Reza jafery 16-Dec-22 14:27pm

Suppose the loop's range is 0 to 500; are you sure that each command (I mean all commands, methods, etc. in the parallel loop) will be repeated 499 times?

Dave Kreskowiak 16-Dec-22 17:45pm

This is where reading the documentation on Parallel.For Method (System.Threading.Tasks)[^] comes in.

Notice the method definition specified the parameters as such:

public static System.Threading.Tasks.ParallelLoopResult For (int fromInclusive, int toExclusive, Action<int,System.Threading.Tasks.ParallelLoopState> body);

The fromINCLUSIVE and toEXCLUSIVE means the range of values is going to be 0 to 499, in your example, which will execute the code block 500 times.

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Reza jafery · Accepted Answer · 2022-12-21T06:33:00

I found a good example:

C#

using System;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;

namespace ConsoleApp
{
    class Program
    {
        const long LOOP_COUNT = 900000;
        static long Calculate()
        {
            long Total = 0;
            for (long i = 0; i < 10000; i++) // add more iteration to make parallel faster than sequential
            {
                Total += 20 * i;
            }
            return Total;
        }

        static void SequentialCalculate()
        {
            var Counter = 0;
            long Total = 0;
            Stopwatch SW = new Stopwatch();
            SW.Start();
            for (long i = 0; i < LOOP_COUNT; i++)
            {
                Total = Calculate();
                ++Counter;
            }
            SW.Stop();
            Console.WriteLine("Sequential total seconds: " + SW.Elapsed.TotalSeconds + ", Counter: " + Counter + ", Total: " + Total);
        }

        static void ParallelCalculate()
        {
            var Counter = 0;
            long Total = 0;
            Stopwatch SW = new Stopwatch();
            SW.Start();
            Parallel.For(0, LOOP_COUNT, i =>
            {
                Total = Calculate();
                Interlocked.Add(ref Counter, 1);
            });
            SW.Stop();
            Console.WriteLine("Parallel total seconds: " + SW.Elapsed.TotalSeconds + ", Counter: " + Counter + ", Total: " + Total);
        }

        static void Main(string[] args)
        {
            Console.ReadKey();
            SequentialCalculate();
            ParallelCalculate();
            Console.ReadKey();
        }
    }
}

Result (by Intel Core i5-2430M, no debug mode):
Click this link to see