 |
|
Home:
Perl Programming Help:
Regular Expressions:
Re: [namishtiwari] perl script help:
Edit Log
|
|

1arryb
User
May 29, 2009, 8:54 AM
Views: 9560
|
|
Re: [namishtiwari] perl script help
|
|
|
Hi namishtiwari, For parsing large files, you have to process them line-by-line, not read the whole thing into memory. Perhaps something like this:
#!/usr/bin/perl use strict; use warnings; no warnings "uninitialized"; my $logFile = $ARGV[0]; die "usage: $0 <logFile>" unless $logFile; die "Logfile $logFile doesn't exist" unless -f "$logFile"; open(my $log, "<", $logFile) or die "Can't open $logFile for reading."; print "Processing file $logFile...\n"; my $authenticates = {}; my $n = 0; while(my $line = <$log>) { # Outer loop. Look for an interesting part of the log file. $n++; $line =~ tr/\r\n//d; if ( $line =~ /Entering (\S+)::authenticate/ ) { # Aha! Start of the interesting part. my $authentication = $1; while ( $line = <$log> ) { # Inner loop: Find the UserName. $n++; $line =~ tr/\r\n//d; if ( $line =~ /userName :[[](\S*)[]]/ ) { my $userName = "$1" ? "$1" : "NULL"; # You have some null userNames in your log. my $nSessions = $authenticates->{$authentication}->{$userName}; $nSessions = $nSessions ? $nSessions += 1 : 1; # Tabulate the number of sessions for this authMethod/userName. $authenticates->{$authentication}->{"$userName"} = $nSessions; last; } elsif ( $line =~ /(?:Entering|Exiting) \S+::authenticate/ ) { print "ERROR: Misparse at line $n of $logFile\n"; last; } } } } close($log); # Output global results: print "AuthMethod\tUser\tSessions\n"; for my $authMethod (sort(keys(%$authenticates))) { for my $user (sort(keys(%{$authenticates->{$authMethod}}))) { print "$authMethod\t$user\t$authenticates->{$authMethod}->{$user}\n"; } } Running this code against your sample logfile.txt, produces the following:
Processing file logfile.txt... AuthMethod User Sessions QnAModule 01018603 3 QnAModule 01214225 2 QnAModule 01789302 1 QnAModule NULL 2 As you can see by your output, you have some data problems. This is a good example why parsing log files for data is a fairly weak interface: You may or may not get good or complete data. Cheers, Larry
(This post was edited by 1arryb on Jun 1, 2009, 1:00 PM)
|
|
|
Edit Log:
|
|
Post edited by 1arryb
(User) on May 29, 2009, 8:57 AM
|
|
Post edited by 1arryb
(User) on May 29, 2009, 9:27 AM
|
|
Post edited by 1arryb
(User) on Jun 1, 2009, 12:56 PM
|
|
Post edited by 1arryb
(User) on Jun 1, 2009, 1:00 PM
|
|
|  |