Click here to Skip to main content
15,887,135 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
Hello everyone,

I have to process file with data which contains more than 11 lacs (Million) rows.

Problem, Each row have data which I need to split and each row will make up 5 rows to Insert or update. Looking at performance point, Which one is suggestable:

1. Writing CTE to split each row into 5 and then CTE Result use merge statement.
2. Create table and dump entire data, Run logic/stored procedure which will generate 5 rows from each row into another table... and then use it with merge..

3. Create temp table and split data. Use that temp table in merge statement.

Any better idea than this.

Please note I am using SSIS to read file as we have multiple files....

Thanks,
Posted
Updated 29-May-15 0:37am
v2
Comments
Maciej Los 29-May-15 16:25pm    
Is this one-time job or do you have to use it periodically?
deepakdynamite 1-Jun-15 0:26am    
It's periodically... I guess daily or weekly

Each one has it's advantages and dis-advantages.
It totally depends upon in which situation you are using what ,
refer following link to clear all your confusion

http://www.dotnet-tricks.com/Tutorial/sqlserver/X517150913-Difference-between-CTE-and-Temp-Table-and-Table-Variable.html[^]

Table Variable V/S Temporary Table[^]
 
Share this answer
 
When the first approach seems to be OK, that second and third seems to be not-logical. Why? Imagine, you have already got data in a table and you want to "copy" it into temporary table, then you want to provide split/merge operation. Why? Note, each operation on huge portion of data must take some time. So, as many operation is being pending, as many time is needed. Got it? The most efficient way is to provide direct "copy" operation .
 
Share this answer
 
Comments
deepakdynamite 1-Jun-15 0:28am    
Thanks :)
Maciej Los 1-Jun-15 1:51am    
You're welcome ;)

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900