Click here to Skip to main content
15,890,690 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
I am having Application slowness when i suppose to get distinct out a DataTable which has 50k+ records. i am using the below code to do it

C#
/// <summary>
   /// Generate Distinct Data Table from the Data Table Generated from server.
   /// </summary>
   Func<CancellationTokenSource, DataGridCollectionView, DataTable, DataTable> GetDistinctTable =
       (cancellationToken, DqmDataGridCollectionView, ResultDataTable) =>
       {

           var distinctCol = DqmDataGridCollectionView.SortDescriptions.ToList();
           if (distinctCol.Any())
           {

               var _distinctColumns = from n in distinctCol
                                      select new { col = n.PropertyName.Split('_')[1] }.col;

               var dtLinq20 = new DataTable();

               foreach (var item in _distinctColumns)
               {
                   var existingColumn = ResultDataTable.Columns[item];
                   var _dColumn = new DataColumn { ColumnName = existingColumn.ColumnName, DataType = existingColumn.DataType, DefaultValue = existingColumn.DefaultValue, AllowDBNull = existingColumn.AllowDBNull, Caption = existingColumn.Caption };
                   dtLinq20.Columns.Add(_dColumn);
               }

               DataTable DistinctDataTable = null;

               try
               {
                   DistinctDataTable = ResultDataTable.AsEnumerable().AsParallel().WithCancellation(cancellationToken.Token)
                    .Select(row =>
                    {
                        var newRow = dtLinq20.NewRow();
                        foreach (var item in _distinctColumns)
                        {
                            newRow[item] = row[item];
                        }

                        return newRow;
                    })
                    .Distinct(DataRowComparer.Default).CopyToDataTable();
               }
               catch (Exception)
               {
                   return null;
               }


               DistinctDataTable.TableName = "Distinct_TBL";

               return DistinctDataTable;
           }
           return null;
       };



But for me it's taking too much of my applications load time. can any one suggest me a best solution?

What I have tried:

I have tried the above code and it looks like it was not working any way. though i am using async and calling async my application slows down.
Posted
Updated 2-Oct-16 22:45pm
Comments
Patrice T 3-Oct-16 0:47am    
define "But for me it's taking too much of my applications load time."
Gold$Coin 3-Oct-16 1:03am    
The code which i have pasted above was the work what i have done in my application. i need to know any thing i need to optimize with the code to improve my performance.
Patrice T 3-Oct-16 1:08am    
HOW MUCH TIME ARE YOU TALKING ABOUT ?
Gold$Coin 3-Oct-16 1:14am    
More than 4 Min.
Sinisa Hajnal 3-Oct-16 7:06am    
Change the query. Write stored procedure that will return exactly the data you need in a format you need it. Database knows how to work with big sets of data. Code uses loops which are slow and inefficient for this kind of tasks. Esp. since you have several loops both explicit and implicit through linq.

1 solution

Please make a seperate thread so it will run on the background. Load first the things you need the most and so on and on. Search for Proactor Pattern, this explains how to make sure that your program keeps running.

Quote:
Proactor Pattern
 
Share this answer
 
Comments
Gold$Coin 5-Oct-16 2:35am    
Ya i have tried it. if we do so the executing and all the underling calculation are doing good in reasonable time. But when we bind the data to the UI with all the stuffs in it (Coloring/Allignment/Font Size, Etc.,) it taking to much of time to complete so i decided to do it parallel and i have done with that. but only the Distinct part alone taking too much of time to complete.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900