Click here to Skip to main content
15,920,438 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I have a table with 15 column. Suppose I have 10000 records in the table.
I want to check the duplicate records in the table with all 15 column one by one .

When i am uploading the excel file with 10000 records the speed is vey slow.

please help me about the speed.how we can improve the speed?

the speed is very good at local computer

but the speed is slow on the server.
Posted
Updated 28-Apr-11 19:30pm
v3

If possible to attach a code what you are using it will really easy to go through the proceedings.

in the mean time get the excel rows extracted in Datatable so the u can easily apply rowfilter for duplication.
 
Share this answer
 
You should have posted your code in your question, Anyway apply these

Uploading large files with ASP.NET: what are the best practices?[^]

Uploading Files in ASP.NET[^]
 
Share this answer
 
Comments
Dalek Dave 29-Apr-11 1:35am    
Good links
Use the code below to read the data from excel.
C++
private OleDbConnection returnConnection(string fileName)
    {
        return new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + fileName + "; Jet OLEDB:Engine Type=5;Extended Properties=\"Excel 8.0;\"");
    }

    private DataTable loadSingleSheet(string fileName, string sheetName)
    {
        DataTable sheetData = new DataTable();
        using (OleDbConnection conn = this.returnConnection(fileName))
        {
           conn.Open();
           // retrieve the data using data adapter
           OleDbDataAdapter sheetAdapter = new OleDbDataAdapter("select * from [" + sheetName + "]", conn);
            sheetAdapter.Fill(sheetData);
        }
        return sheetData;
    }


Then save the DataTable to the database.
 
Share this answer
 
You will need to post some code here before someone can help you.

In general, uploading an excel sheet (or checking for duplicates) with a large number of records will take some time.
The best you can do is show a progress bar and inform the user to wait.

Maybe you could try to check for duplicates at the database level and not at the time when you start uploading
 
Share this answer
 
firstly u can save ur excel data to temporary table in database
then u can remove duplicate rows from sql using sqlquery which removes duplicate rows
AFTER that u can insert it in original table.
This might improve speed as it doesn't require looping.
Here is some link to remove duplicate rows
http://blog.sqlauthority.com/2007/03/01/sql-server-delete-duplicate-records-rows/[^]
http://www.orafaq.com/faq/how_does_one_eliminate_duplicates_rows_from_a_table[^]
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900