Home: Perl Programming Help: Intermediate:
Help in creating a perl script to parse web log file, access_log

New User

Jun 19, 2003, 4:16 PM

Views: 939
Help in creating a perl script to parse web log file, access_log

 Hi all,

I am new to this group. I need help regarding a perl script which parses the web log file, access_log.

The format of the access_log is: - - [15/Jun/2003:13:54:02 -0100] "GET /xxxx HTTP/1.1" 200 34906

The goal is to

1. Perfom a count of the pages for the given timestamp. It is possible that multiple pages exist with the same timestamp (As the timestamp I mentioned above).
2. Within a range of time interval, say, 15 minutes starting with the timestamp of the first line in the log file, I would like to compute the average of the number of pages, minimum and maximum number of pages in that interval.

3. I would like the output as below. Following is just an example.

Time Average Pages Min Pages Max Pages
--------------------------- ----------------- -----------------
15/Jun/2003:14:09:02 6.5 3 10
15/Jun/2003:14:24:02 5.5 4 7
Here is my Perl script:


###use strict;

use Getopt::Long;
use Time::Local;

my $file="access_log_modified";
my $begin_time = "";
my $end_time;
my @visual_pages = ();
my @final_visual_pages = ();
my %increment = ();
my ($datetime, $get_post, $Day, $Month, $Year, $Hour, $Minute, $Second);
my $interval = 60; #An interval of 1 minute


sub count_recs {

open (INFILE, "<$file") || die "Cannot read from $file";
($datetime,$get_post) = (split / /) [3,6];
$datetime =~ s/\[//;
($Day,$Month,$Year,$Hour,$Minute,$Second)= $datetime =~m#^(\d\d)/(\w\w\w)/(\d\d\d\d):(\d\d):(\d\d):(\d\d)#;

next WHILELOOP if ($get_post =~ /\.js$/ || $get_post =~ /\.gif$/ || $get_post =~ /\.css$/);

unless ($begin_time) {
$begin_time = $datetime;
push (@dates, $datetime);

} #outer while

foreach $dateproc (@dates) {
foreach $dateproc (sort keys %increment) {
push (@{$processed_visual_pages{$dateproc}}, $increment{$dateproc});
print "$dateproc @{$processed_visual_pages{$dateproc}}\n";



The output I get is:
25/Apr/2003:13:54:02 3
25/Apr/2003:13:54:19 2
25/Apr/2003:13:54:22 4
25/Apr/2003:13:54:34 3
25/Apr/2003:13:54:38 5
25/Apr/2003:13:54:41 3
25/Apr/2003:13:54:43 6
25/Apr/2003:13:54:44 3
25/Apr/2003:13:54:46 5
25/Apr/2003:13:54:47 2
25/Apr/2003:13:54:48 3
25/Apr/2003:13:54:50 7
25/Apr/2003:13:54:51 4
25/Apr/2003:13:54:53 2
25/Apr/2003:13:54:58 3
25/Apr/2003:13:55:01 2
25/Apr/2003:13:55:02 4
25/Apr/2003:13:55:05 5
25/Apr/2003:13:55:08 1
25/Apr/2003:13:55:14 3
25/Apr/2003:13:55:15 1
25/Apr/2003:13:56:13 6
25/Apr/2003:13:56:27 5
25/Apr/2003:13:56:35 4
25/Apr/2003:13:56:40 4
25/Apr/2003:13:56:45 1
25/Apr/2003:13:56:51 5
I am able to get the count for each of the timestamps. However, I am having trouble getting the records formatted in the interval range.

I shall appreciate an early help in solving this problem.

Thanks in Advance


Thaumaturge / Moderator

Jun 25, 2003, 2:43 AM

Views: 923
Re: [Andy61] Help in creating a perl script to parse web log file, access_log

Take a look at the Logfile::Apache module. Available from CPAN.

Dave Cross, Perl Hacker, Trainer and Writer
Get more help at Perl Monks