Parse Log file and import to database

Question

0.00/5 (No votes)

See more:

This is a sample of my log file in text document.

Jan 1 22:54:17 drop %LOGSOURCE% >eth1 rule: 7; rule_uid: {C1336766-9489-4049-9817-50584D83A245}; src: 70.77.116.190; dst: %DSTIP%; proto: tcp; product: VPN-1 & FireWall-1; service: 445; s_port: 2612;
Jan 1 22:54:22 drop %LOGSOURCE% >eth1 rule: 7; rule_uid: {C1336766-9489-4049-9817-50584D83A245}; src: 61.164.41.144; dst: %DSTIP%; proto: udp; product: VPN-1 & FireWall-1; service: 5060; s_port: 5069;

This are the two lines out of few thousand lines in the log file.

The main question is how do I parse this and then import it into database?

*I know the method is using SQL statements but I want to know the exact code for it.*

Posted 14-Jul-15 17:21pm

iambadatcoding

Add a Solution

Comments

PIEBALDconsult 14-Jul-15 23:38pm

"I want to know the exact code for it"
Only you know that.

As for parsing, I'd try Regular Expressions first.

iambadatcoding 15-Jul-15 21:33pm

I learnt a little bit of regular expressions in the past but Im not sure if log parser accepts regular expressions. Since my job is to separate them into different columns and input them into database.
One example is Jan 1 22:54:17, 7, {C1336766-9489-4049-9817-50584D83A245}, 70.77.116.190, %DSTIP%, tcp, VPN-1 & FireWall-1, 445, 2612
This is the output that I really need and I will need to convert it from a log file into csv file so that I can use my javascript to visualize it into graphs etc

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Wendelius · Answer 1 · 2015-07-14T17:44:00

Solution 1

The sample data looks like each item on a row is separated with a semiciolon. SO basically you can export this as a CSV file, make modifications / conversions to the data if needed and then add the data into the database.

Few articles to get you started:
- C# - CSV Import Export[^]
- Importing CSV Data and saving it in database[^]

Posted 14-Jul-15 17:44pm

Wendelius

Comments

iambadatcoding 15-Jul-15 21:30pm

Using CSV import/Export to make modifications?

Wendelius 15-Jul-15 23:36pm

Yes, you can treat the file as a CSV. just use semicolon ( ; ) as a delimiter and read data from the file. After reading, add relevant data to the database.