Click here to Skip to main content
15,867,999 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi,

I am making updates with source data to destination table.
Source file is excel and destination is sql server table.
In source file I do have one particular project related data where as in destination table i do have several projects. Data contains millions of rows in each project. So the source and destination size is very much.

I am using lookup transformation to compare the data and do the updations accordingly. The problem is it is taking much time to load the entire data in lookup cache as the destination table size is very much. so is there any way of loading the data of only the projects of source file like (select * from table where projectid=sourcetable.projectid)

Please advise

Thanks

What I have tried:

Both Source and destination data is huge then how to use lookup transformation
or is there any other way.
Posted
Comments
CHill60 3-Aug-18 3:45am    
I ususally have a "staging" table that I load stuff to - then move stuff into the final repository and empty the table

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900