Click here to Skip to main content
15,880,796 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more: , +
i have a Datatable and i am adding data to 3 columns of the datatable by calling the function(which is necessary).it's taking around 15 min to process 7,000 records is there any way to use Multithreading and make it faster ?


foreach (DataRow row in dtTaskandBugs.Rows)
        {
            int id = Convert.ToInt32(row["Id"]);
            int storyid = GetStoryid(id);//68725; 70581//GetStoryid(Convert.ToInt32(row["Id"])); //get task or bug storyid

            int FeatureID = Fidname(storyid); //get Featureid of the the story
            string FeatureName = FeatureID == 0 ? "Anonymous" : fname(FeatureID);


            row["Storyid"] = storyid;
            row["FeatureID"] = FeatureID;
            row["FeatureName"] = FeatureName;
        }


What I have tried:

i have a Datatable and i am adding data to 3 columns of the datatable by calling the function(which is necessary).it's taking around 15 min to process 7,000 records is there any way to use Multithreading and make it faster ?
Posted
Updated 2-Mar-18 14:22pm

Try Parallel.ForEach

How to: Write a Simple Parallel.ForEach Loop | Microsoft Docs[^]

There is no guarantee it'll increase performance, there are many factors involved in that such as how many cores on your site's server, the SQL server architecture and all sorts. You're more likely to get better performance refactoring your function, or not using some other method to get the data in your database.
 
Share this answer
 
Comments
Afzaal Ahmad Zeeshan 3-Mar-18 18:16pm    
5ed. Totally right, that it does not guarantee any improvement. Only that you can sleep at night considering the fact, you played yours. :laugh:

I would use a SemaphoreSlim. This is a class that 'gates' the number of concurrent reads of a database to a preset level. Once that level has been reach, an existing read has to finish before a new read is allowed to start.


C#
//A method to asynchronously load data from the database
private async Task<Order> LoadOrderFromDbAsync(int orderId){//....}
 private SemaphoreSlim semaphoreSlim;
 private async Task<List<Order>> GatedConcurrentLoadOfOrdersAsync(IEnumerable<int> orderIds)
        {
           using (semaphoreSlim = new SemaphoreSlim(maxConcurrencyLevel))
          {
            var tasks = new List<Task<Order>>();
            foreach (int orderId in orderIds)
            {
                //start each task off but don't await it
                //tasks are started by simply invoking the async method
                Task<Order> task = GatedLoadOrderFromDatabaseAsync(orderId);
                tasks.Add(task);
            }

            //await them all to complete
            Order[] loadedOrders = await Task.WhenAll(tasks);
            return loadedOrders.ToList();
         }
        }

 private async Task<Order> GatedLoadOrderFromDatabaseAsync(int orderId)
        {
            //queue here if max number of threads are already active
            await semaphoreSlim.WaitAsync();
           try
            {
              return  await LoadOrderFromDbAsync(orderId);
                
            }
            finally
            {
                //open the gate to let a queued thread run
                semaphoreSlim.Release();
            }
        }
 
Share this answer
 
While ADDING rows to a DataTable? Not really.

From the MSDN documentation on the DataTable:
Quote:
This type is safe for multithreaded read operations. You must synchronize any write operations.

While it is possible to use multiple threads to add records to a DataTable object, adding rows to a DataTable must be synchronized. This means that only one thread can add a DataRow to the table at a time, essentially turning the operation into a single threaded operation. You will get little to no performance benefit from using multiple threads.

The real question should be why are you adding so many rows to a DataTable object in the first place? Chances are good you don't need all of those rows in the DataTable.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900