How to create a new records for the imported records from different dump files that have repeated ID but different data?

Question

0.00/5 (No votes)

See more:

I have 3 dump files from 3 different machines pc1, pc2 and pc3. I am trying to sync the data from the 3 dump files with the database on my server but, unfortunately the dump files have similar unique ids for different records, because they are on different machines. When merging, the data from the dump file, overwrites the data on my server(if they have the same unique id). Is there a way for creating new records for the records that has the same unique id but different data fields. If no if there are other 3rd party tools for that?

What I have tried:

I have found a similar question but it gave one of 2 solutions:

    1. Either to replace the repeated rows

    2. Or to skip the repeated records

And no one of the above is suitable for me. I need to create a new record for the repeated rows with repeated unique ids but different other data field values
</pre>

Posted 12-Apr-18 23:06pm

Ahmed AE

Add a Solution

Comments

Richard MacCutchan 13-Apr-18 7:11am

Create a new unique id for each record as you read them in.

Ahmed AE 13-Apr-18 10:29am

What if there are the same record in 2 dump files but have different id's.

Richard MacCutchan 13-Apr-18 10:36am

Well you will have to find some method of identifying them and excluding the duplicate. It sounds like you first need to merge the three dumpfiles and discard any duplicate records. You can then process the remainder, changing the id of each record to a new unique id that does not conflict with your existing data.

Ahmed AE 13-Apr-18 11:32am

the problem that is found between the dump files and each other, can also be found between the resultant dump file (after merging) and the database on my server,i mean that there might be records that are found in the resultant dump file and also found in the database on my server, so what can i do in that case?

Richard MacCutchan 13-Apr-18 11:51am

Then follow the above suggestion.

Ahmed AE 13-Apr-18 11:36am

"Create a new unique id for each record as you read them in." does this will solve the problem ? this will only avoid overwriting the already found id's but will not solve the problem that there will be repeated records in the server database

Richard MacCutchan 13-Apr-18 11:54am

Try thinking about what you are trying to do here. If you have potential duplicate records then you need to find a way of discarding them. Whether they are in the dumpfiles or the database, the issue is exactly the same.
Process each dumpfile one at a time.
If the next record is a duplicate of the database then discard it.
If not then generate a unique id and enter it into the database.
Repeat for all records and then for the other dumpfiles.

Ahmed AE 14-Apr-18 7:10am

thanks

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)