Need to import a flat file with different section with different columns in it

Question

0.00/5 (No votes)

See more:

I have a '|' delimited Flat file with varying columns in 3 different sections.

Each section is identified by the value in the second column.

I have to read this file and load it in a table through SSIS.

It would be better if someone provides a solution for it.

Please find the below mentioned sample data

Sample data:
UPD|1|123|20180502|JOHN|20180502
UPD|1|345|20180502|PETER|20180502
UPD|2|456|20180502|X|A
UPD|2|912|20180502|Y|A
UPD|3| |257|20180502|X|A
UPD|3| |756|20180502|Y|A

If you notice the 3rd section has got an extra column. In section 3 alone we have got 7 columns but in rest of the section we have only 6 columns.

I need a output like this:

UPD 1 123 20180502 JOHN 20180502
UPD 1 345 20180502 PETER 20180502
UPD 2 456 20180502 X A
UPD 2 912 20180502 Y A
UPD 3 257 20180502 X A
UPD 3 756 20180502 Y A

What I have tried:

If i read this file through file_file_source in ssis the data is coming like this, the flat file is considering only the 6 columns and for the section 3, 7th column value is displayed in 6th column itself.

UPD 1 123 20180502 JOHN 20180502
UPD 1 345 20180502 PETER 20180502
UPD 2 456 20180502 X A
UPD 2 912 20180502 Y A
UPD 3 257 20180502 X|A
UPD 3 756 20180502 Y|A

For section 3 the last column value is displayed as "X|A".

Any help would be greatly appreciated. Thanks in advance.

Regards,
Gopinath

Posted 2-May-18 19:55pm

gopinathPalani

Updated 17-Jun-18 23:24pm

Add a Solution

Comments

W∴ Balboos, GHB 3-May-18 6:42am

Maybe, since you also seem to have c# available, you should read the flat file and use String.Split() on it? Then you can insert it in a preexisting table and be done with it.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Hiske Bekkering · Answer 1 · 2018-06-17T23:25:00

Hi there Gopinath,

It's an old question, so you might have already found an answer elsewhere. If not, take a look at my article on dynamic column mapping: Dynamic File Column mapping in SSIS. - SQLServerCentral[^]

My solution deals with variable columns in flat files where all rows have the same number of values as there are columns in the file. However, in the Script Component used in my solution, it's easy to implement code changes to handle any differences from row-to-row as the data is being read.

Regards,

Hiske