CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Help in creating a perl script to parse web log file, access_log

 



Andy61
New User

Jun 19, 2003, 4:16 PM

Post #1 of 2 (467 views)
Help in creating a perl script to parse web log file, access_log Can't Post

 Hi all,

I am new to this group. I need help regarding a perl script which parses the web log file, access_log.

The format of the access_log is:

127.0.0.1 - - [15/Jun/2003:13:54:02 -0100] "GET /xxxx HTTP/1.1" 200 34906

The goal is to

1. Perfom a count of the pages for the given timestamp. It is possible that multiple pages exist with the same timestamp (As the timestamp I mentioned above).
2. Within a range of time interval, say, 15 minutes starting with the timestamp of the first line in the log file, I would like to compute the average of the number of pages, minimum and maximum number of pages in that interval.

3. I would like the output as below. Following is just an example.

Time Average Pages Min Pages Max Pages
--------------------------- ----------------- -----------------
15/Jun/2003:14:09:02 6.5 3 10
15/Jun/2003:14:24:02 5.5 4 7
-------------
Here is my Perl script:

Code
 #!/usr/bin/perl   

###use strict;

use Getopt::Long;
use Time::Local;

my $file="access_log_modified";
my $begin_time = "";
my $end_time;
my @visual_pages = ();
my @final_visual_pages = ();
my %increment = ();
my ($datetime, $get_post, $Day, $Month, $Year, $Hour, $Minute, $Second);
my $interval = 60; #An interval of 1 minute

count_recs();

sub count_recs {

open (INFILE, "<$file") || die "Cannot read from $file";
WHILELOOP: while (<INFILE>) {
($datetime,$get_post) = (split / /) [3,6];
$datetime =~ s/\[//;
($Day,$Month,$Year,$Hour,$Minute,$Second)= $datetime =~m#^(\d\d)/(\w\w\w)/(\d\d\d\d):(\d\d):(\d\d):(\d\d)#;

next WHILELOOP if ($get_post =~ /\.js$/ || $get_post =~ /\.gif$/ || $get_post =~ /\.css$/);

unless ($begin_time) {
$begin_time = $datetime;
}
push (@dates, $datetime);

} #outer while

foreach $dateproc (@dates) {
$increment{$dateproc}++;
}
foreach $dateproc (sort keys %increment) {
push (@{$processed_visual_pages{$dateproc}}, $increment{$dateproc});
print "$dateproc @{$processed_visual_pages{$dateproc}}\n";
}

close(INFILE);
}

--------------

The output I get is:
25/Apr/2003:13:54:02 3
25/Apr/2003:13:54:19 2
25/Apr/2003:13:54:22 4
25/Apr/2003:13:54:34 3
25/Apr/2003:13:54:38 5
25/Apr/2003:13:54:41 3
25/Apr/2003:13:54:43 6
25/Apr/2003:13:54:44 3
25/Apr/2003:13:54:46 5
25/Apr/2003:13:54:47 2
25/Apr/2003:13:54:48 3
25/Apr/2003:13:54:50 7
25/Apr/2003:13:54:51 4
25/Apr/2003:13:54:53 2
25/Apr/2003:13:54:58 3
25/Apr/2003:13:55:01 2
25/Apr/2003:13:55:02 4
25/Apr/2003:13:55:05 5
25/Apr/2003:13:55:08 1
25/Apr/2003:13:55:14 3
25/Apr/2003:13:55:15 1
25/Apr/2003:13:56:13 6
25/Apr/2003:13:56:27 5
25/Apr/2003:13:56:35 4
25/Apr/2003:13:56:40 4
25/Apr/2003:13:56:45 1
25/Apr/2003:13:56:51 5
-------------------------
I am able to get the count for each of the timestamps. However, I am having trouble getting the records formatted in the interval range.

I shall appreciate an early help in solving this problem.

Thanks in Advance

Andy


davorg
Thaumaturge / Moderator

Jun 25, 2003, 2:43 AM

Post #2 of 2 (451 views)
Re: [Andy61] Help in creating a perl script to parse web log file, access_log [In reply to] Can't Post

Take a look at the Logfile::Apache module. Available from CPAN.

--
Dave Cross, Perl Hacker, Trainer and Writer
http://www.dave.org.uk/
Get more help at Perl Monks

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives