CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Need a Custom or Prewritten Perl Program?: I need a program that...:
Extract specific data (user ID) from log file

 



vdthoughts
New User

Nov 5, 2013, 2:36 AM

Post #1 of 6 (10099 views)
Extract specific data (user ID) from log file Can't Post

I would like to extract the user ID's only from the log file. I have copied the sample code below. All the user ID's are inbetween the text "ID principal: " & ", ". Later i would like to use the data to be copied into an excel sheet to create a chart. Can anybody help me to write a perl script for the same. I have never done some complex task on perl prog till now.
---------------------------------------------------------
Sample log file code:

Tue Nov 5 09:00:00 2013
SI Floating Licenses: 14
ID principal: IDB2COB, ID principal: kdk4oe, ID principal: oij7nde, ID principal: qay8plm, ID principal: wsx9okn, ID principal: LNA6COB, ID principal: lak3kor, ID principal: deveas
SI Seat Licenses: 7
ID principal: ISN1COB, ID principal: kao7wbe, ID principal: kor4sds, ID principal: HSC4WE, ID principal: usr4cint, ID principal: SUU4COB, ID principal: birr
IM Floating Licenses: 8
ID principal: tram2we, ID principal: IDB2COB, ID principal: kao7wbe, ID principal: HSC4WE, ID principal: VBN1KOR, ID principal: adf3rtz, ID principal: MAG4KOR, ID principal: ASI3COB
IM Seat Licenses: 2
ID principal: kor4sds, ID principal: ribbs


Tue Nov 5 09:30:00 2013
SI Floating Licenses: 10
ID principal: IDB2COB, ID principal: kdk4oe, ID principal: oij7nde, ID principal: qay8plm, ID principal: wsx9okn, ID principal: LNA6COB, ID principal: lak3kor, ID principal: deveas
SI Seat Licenses: 7
ID principal: ISN1COB, ID principal: kao7wbe, ID principal: kor4sds, ID principal: HSC4WE, ID principal: usr4cint, ID principal: SUU4COB, ID principal: birr
IM Floating Licenses: 8
ID principal: tram2we, ID principal: IDB2COB, ID principal: kao7wbe, ID principal: HSC4WE, ID principal: VBN1KOR, ID principal: adf3rtz, ID principal: MAG4KOR, ID principal: ASI3COB
IM Seat Licenses: 2
ID principal: kor4sds, ID principal: ribbs


Tue Nov 5 10:00:00 2013
SI Floating Licenses: 7
ID principal: IDB2COB, ID principal: kdk4oe, ID principal: oij7nde, ID principal: qay8plm, ID principal: wsx9okn, ID principal: LNA6COB, ID principal: lak3kor, ID principal: deveas
SI Seat Licenses: 7
ID principal: ISN1COB, ID principal: kao7wbe, ID principal: kor4sds, ID principal: HSC4WE, ID principal: usr4cint, ID principal: SUU4COB, ID principal: birr
IM Floating Licenses: 8
ID principal: tram2we, ID principal: IDB2COB, ID principal: kao7wbe, ID principal: HSC4WE, ID principal: VBN1KOR, ID principal: adf3rtz, ID principal: MAG4KOR, ID principal: ASI3COB
IM Seat Licenses: 2
ID principal: kor4sds, ID principal: ribbs
-----------------------------------------------------------


FishMonger
Veteran / Moderator

Nov 5, 2013, 6:13 AM

Post #2 of 6 (10092 views)
Re: [vdthoughts] Extract specific data (user ID) from log file [In reply to] Can't Post

What have you tried?

What errors/warnings are you getting?

Please post the code you're testing.


BillKSmith
Veteran

Nov 5, 2013, 8:19 AM

Post #3 of 6 (10080 views)
Re: [vdthoughts] Extract specific data (user ID) from log file [In reply to] Can't Post

You can meet your minimum requirements without a "complex task."

Read data by logical record. ( set $/ = q() refer: perldoc perlvar)

For each record:

Use one regex to extract the date (You will probably need it for your table)

Use a second regex to extract all the ID's in that record

Write both to one record of a .csv file (Which Excel can read.)


Use Excel to do anything else you want.

You would then be ready to attempt to use a module such as Excel::Writer::XLSL to create the spreadsheet with Perl.
Good Luck,
Bill


vdthoughts
New User

Nov 5, 2013, 8:21 AM

Post #4 of 6 (10079 views)
Re: [FishMonger] Extract specific data (user ID) from log file [In reply to] Can't Post

Code I have written till now.
Since I am very new to Perl am very sorry for my coding standards.

#Extract specific data (user ID) from log file
[perl] #!/usr/bin/perl

$LOGFILE = "C:\\Users\\vut2kor\\Desktop\\lic_usage.log";
open(LOGFILE) or die("Could not open log file.");
foreach $line (<LOGFILE>)
{
@matches =~ /ID principal:\s+(\w+),/g;
}
close(LOGFILE);
[/perl]

Compilation error:
C:\Users\vut2kor>perl -wc C:\Users\vut2kor\Desktop\Extract.pl
Applying pattern match (m//) to @array will act on scalar(@array) at C:\Users\vut2kor\Desktop\Extract.pl line 9.
Name "main::line" used only once: possible typo at C:\Users\vut2kor\Desktop\Extract.pl line 7.
Name "main::matches" used only once: possible typo at C:\Users\vut2kor\Desktop\Extract.pl line 9.
C:\Users\vut2kor\Desktop\Extract.pl syntax OK


BillKSmith
Veteran

Nov 5, 2013, 11:31 AM

Post #5 of 6 (10070 views)
Re: [vdthoughts] Extract specific data (user ID) from log file [In reply to] Can't Post

Try to understand your warning messages.


Quote
Applying pattern match (m//) to @array will act on scalar(@array) at C:\Users\vut2kor\Desktop\Extract.pl line 9.


The match operator is meant to operate on a string. You have specified an array (@matches). Perl will convert the array to a scalar (number of elements in the array) This is definitly not what you intend!


Quote
Name "main::line" used only once: possible typo at C:\Users\vut2kor\Desktop\Extract.pl line 7.


The variable $line (in package main) is used only once (on line 7)


Quote
Name "main::matches" used only once: possible typo at C:\Users\vut2kor\Desktop\Extract.pl line 9.


Same story for $matches on line 9.


Quote
C:\Users\vut2kor\Desktop\Extract.pl syntax OK


Your code is valid Perl, but given the warnings above, it probably will not do what you intend.


You should always use the following two statements at the beginning of your perl scripts:


Code
use strict; 
use warnings;


It may seem like an inconvenience that they require you to declare (with my) all your variables. The effort will be repaid with less debug time.

Your open statements do not do what you think. Always use the three argument form (Refer: perldoc -f open).

Always use lexical filehandles.


Code
open $LOGFILE, '<', "C:\\Users\\vut2kor\\Desktop\\lic_usage.log" 
or die "Could not open log file.";


Your regex will fail when an ID comes at the end of a line (no comma).



Note: [perl][/perl] tags do not work on this site. Use code instead.
Good Luck,
Bill

(This post was edited by BillKSmith on Nov 5, 2013, 9:22 PM)


BillKSmith
Veteran

Nov 6, 2013, 7:41 AM

Post #6 of 6 (10055 views)
Re: [BillKSmith] Extract specific data (user ID) from log file [In reply to] Can't Post

Here is the "front end" program that I described in post 3.
I have attached the input and output files from my test.

Code
use strict; 
use warnings;
use English;
my $logfile = "lic_usage.log"; # Changed for my environment
open(my $LOG_IN, '<', $logfile) or die("Could not open log file.");
open my $TABLE_OUT, '>', 'excel_table.csv' or die "cannot open excel_table\n";

{
local $INPUT_RECORD_SEPARATOR = q();
local $OUTPUT_FIELD_SEPARATOR = q(,);
local $LIST_SEPARATOR = q(,);
while (my $log_record = <$LOG_IN>) {
my ($date) = $log_record =~ m/^ ( .+? ) $/xms;
my @IDs = $log_record =~ m/(?<=ID\sprincipal:\s) [^,\n]+ /xmsg;
print {$TABLE_OUT} $date, "@IDs\n";
}
}
close $TABLE_OUT;
close $LOG_IN;

Good Luck,
Bill
Attachments: lic_usage.log (2.02 KB)
  excel_table.csv (0.64 KB)
  excel_table.xlsx (9.24 KB)

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives