Click here to Skip to main content
15,890,527 members
Articles / Programming Languages / PowerShell
Tip/Trick

Parse Log4Net log files with PowerShell

Rate me:
Please Sign up or sign in to vote.
0.00/5 (No votes)
15 Feb 2013CPOL 19.6K   2   3
PowerShell in general can be used for searching text in files. The same fearture helps to get required debugging info from log files on the production server,

Introduction

Recently, I was needed to debug an issue in a deployed web application on production server in client premises. To get the logs on dev machine, I needed through go their approval systems and then somebody from their used to get me the logs. I cannot install other tools on production server to filter out the logs. There were too many log files to go through. So I have to live with what I have. So PowerShell came to rescue me here.

Background 

PowerShell is task-based command-line shell and scripting language built on top of the .NET Framework. So most of the basic functionality are available for our task at hand.

Using the code 

So the task is parse the text file with regular expression with PowerShell commandline. Here is the sample log file the Log4Net has generated.
C++
2013-02-12 01:31:25 - Trying for user: 986233
2013-02-12 01:31:25 - Failed.
2013-02-12 01:31:41 - Trying for user: : 956318
2013-02-12 01:31:41 - Success.
2013-02-12 01:31:56 - Trying for user: : 468533
2013-02-12 01:31:56 - Success.
2013-02-12 01:32:11 - Trying for user: : 496924
2013-02-12 01:32:11 - Failed. 
2013-02-12 01:32:26 - Trying for user: : 774469
2013-02-12 01:32:26 - Success.
2013-02-12 01:32:41 - Trying for user: : 799631
2013-02-12 01:32:41 - Success.

And now we need to write one or more command line statements to get failed user ids. Here is how I did it.

C++
PS D:\WorkSpace\Apps> [regex]::matches((get-content .\WebLog.txt | out-string ),
 '.*Trying.*\r\n.*Failed.*', "Multiline") | foreach { $_.Groups[0].Value } 

And the result was as expected 

C++
2013-02-12 01:31:25 - Trying for user: 986233
2013-02-12 01:31:25 - Failed.
2013-02-12 01:32:11 - Trying for user: : 496924
2013-02-12 01:32:11 - Failed. 

Points of Interest

We can get matching string easily from Select-String commandlet but It took some time for me to write multiline regular expression in PowerShell.

History

  • 02/15/2013 - v1.0.0 - Initial commit.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Technical Lead
India India
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
SuggestionGenerally, I don't like the use of a huge [regex]::matches call that slurps all the data Pin
Guss7718-Feb-13 21:41
Guss7718-Feb-13 21:41 
The treatment of the entire log as a single string is easier to comprehend but is the anti-thesis of stream processing and will break for very large log files.

Its important to understand how to do stream processing when its easy (i.e. small files that are easy to understand by eyeing them in an editor) so you won't lose all hope when dealing with terabytes worth of uncompressed data.

The above analyzer, assuming all you are interested in really is the failed user IDs, could be written like this:

get-content somelog.log | foreach { if ($_ -match "user: : (\d+)") { $user = $matches[1]; } $_ } | foreach { if ($_ -match "Failed") { echo $user; } }

Which reads like this:
Step 1: Read somelog.log, line by line
Step 2: For each line, if it has a user id, store the ID for later reference, and pass the unparsed line to the next stage
Step 3: For each line, if it has the word "Failed", retrieve the stored user id and print it out

Separating the steps like this means that for each step the memory requirement is minimal, and as a result comprehending huge logs becomes a trivial exercise of the IO bus and not a problem in memory management.

(In case you are really interested in showing the entire log line for each match, it should be trivial to extend the above script to allow for that).
GeneralRe: Generally, I don't like the use of a huge [regex]::matches call that slurps all the data Pin
Rumnha1-Mar-13 20:55
professionalRumnha1-Mar-13 20:55 
GeneralRe: Generally, I don't like the use of a huge [regex]::matches call that slurps all the data Pin
Guss7728-Apr-13 6:08
Guss7728-Apr-13 6:08 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.