CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Regular Expressions: Re: [namishtiwari] perl script help: Edit Log



1arryb
User

May 29, 2009, 8:54 AM


Views: 14973
Re: [namishtiwari] perl script help

Hi namishtiwari,

For parsing large files, you have to process them line-by-line, not read the whole thing into memory. Perhaps something like this:

Code
#!/usr/bin/perl 

use strict;
use warnings;
no warnings "uninitialized";

my $logFile = $ARGV[0];
die "usage: $0 <logFile>" unless $logFile;
die "Logfile $logFile doesn't exist" unless -f "$logFile";
open(my $log, "<", $logFile) or die "Can't open $logFile for reading.";
print "Processing file $logFile...\n";
my $authenticates = {};
my $n = 0;
while(my $line = <$log>) {
# Outer loop. Look for an interesting part of the log file.
$n++;
$line =~ tr/\r\n//d;
if ( $line =~ /Entering (\S+)::authenticate/ ) {
# Aha! Start of the interesting part.
my $authentication = $1;
while ( $line = <$log> ) {
# Inner loop: Find the UserName.
$n++;
$line =~ tr/\r\n//d;
if ( $line =~ /userName :[[](\S*)[]]/ ) {
my $userName = "$1" ? "$1" : "NULL"; # You have some null userNames in your log.
my $nSessions = $authenticates->{$authentication}->{$userName};
$nSessions = $nSessions ? $nSessions += 1 : 1; # Tabulate the number of sessions for this authMethod/userName.
$authenticates->{$authentication}->{"$userName"} = $nSessions;
last;
} elsif ( $line =~ /(?:Entering|Exiting) \S+::authenticate/ ) {
print "ERROR: Misparse at line $n of $logFile\n";
last;
}
}
}
}
close($log);

# Output global results:
print "AuthMethod\tUser\tSessions\n";
for my $authMethod (sort(keys(%$authenticates))) {
for my $user (sort(keys(%{$authenticates->{$authMethod}}))) {
print "$authMethod\t$user\t$authenticates->{$authMethod}->{$user}\n";
}
}


Running this code against your sample logfile.txt, produces the following:

Code
Processing file logfile.txt... 
AuthMethod User Sessions
QnAModule 01018603 3
QnAModule 01214225 2
QnAModule 01789302 1
QnAModule NULL 2


As you can see by your output, you have some data problems. This is a good example why parsing log files for data is a fairly weak interface: You may or may not get good or complete data.

Cheers,

Larry


(This post was edited by 1arryb on Jun 1, 2009, 1:00 PM)


Edit Log:
Post edited by 1arryb (User) on May 29, 2009, 8:57 AM
Post edited by 1arryb (User) on May 29, 2009, 9:27 AM
Post edited by 1arryb (User) on Jun 1, 2009, 12:56 PM
Post edited by 1arryb (User) on Jun 1, 2009, 1:00 PM


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives