CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: General Discussions: General Questions:
Fine tune perl script to make it faster(currently taking more than 30 minutes to process)

 



anujajoseph
New User

Nov 13, 2012, 6:11 PM

Post #1 of 1 (12221 views)
Fine tune perl script to make it faster(currently taking more than 30 minutes to process) Can't Post

Hi,

Need your expert advice on fine tuning the below perl script, which is currently taking more than 30 minutes to extract information from a raw file which has nearly 2million records. Thanks in advance for your help!!
Below are the requirements met in the script:
1)Script should scan through a log file and output the final result into a file called "test.txt"
2)It replaces some strings in the log file into more generic terms.
3)It finds the record number of the affected row from the log file and uses this record number to query the main source file for the particular record.
4) It generates a string which now contains information from the logfile and patches it along with the affected row from the source
5) the result is written into the output file test .txt

================================================

#!/usr/bin/perl

$read_file = "$ARGV[0]";

$read_source = "$ARGV[1]";



open(LOGFILE,$read_file) or die "An Error Occured : $!";

open(REPORT,">/retsit/systematics/test.txt");

$str1 = 'failed all WHEN clauses';

$str2= 'CUST SEGEMENT IS EMPTY';

$str3= 'unique constraint';

$str4= 'DUPLICATE RECORD';



while(<LOGFILE>)

{

if ($_ =~ /Record/)

{ $_ =~ s/$str1/$str2/g;

$_ =~ s/$str3/$str4/g;

$ind1 = index($_,'Record')+6;



$len2 =index($_, ':')-6;

$recnum = substr($_,$ind1,$len2);

$recnum =~ s/^\s+|\s+$//g ;

$strx = "sed -n '".$recnum."p' ".$read_source;

$str5 = `$strx`;

$_ .= '|'."$recnum"."$str5\n";



print REPORT $_;

}

}

close(LOGFILE);

close(REPORT);

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives