CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Optimizing performance for script to traverse on filesystem

 



gdan2000
Novice

Feb 1, 2012, 12:51 PM

Post #1 of 2 (1567 views)
Optimizing performance for script to traverse on filesystem Can't Post

I need to optimize performance for script that supposed to scan filesystem and collect info on *.msg files.

I made a script recursive but I'm not sure that this is the best way for implementation.

Additionally I look for top boxes with sort on hash keys

The directory structure is /test_vol/0/00/34567@test.com/1_sdr/34567.msg where range is 0-9 and 00-99 and number of mail accounts is huge

The code is:

sub ScanDirectory {
my ($workdir) = @_;

my($startdir) = &cwd; # keep track of where we began

chdir($workdir) or die "Unable to enter dir $workdir: $!\n";

opendir(DIR, ".") or die "Unable to open $workdir: $!\n";
my @names = readdir(DIR);
closedir(DIR);

foreach my $name (@names){

next if ($name eq ".");
next if ($name eq "..");
next if ($name =~ /\.dat$|\.mdb^|\.snapshot/);

if ( -d $name ) {
if ($name =~ /^\d+\@\w/ ) {
$all_mailbox_count++;
$box_size=0;
}

&ScanDirectory($name);
next;
}

if ( $name =~ /\.msg$/ ) {
my $msg_size=(stat($name))[7];
if ( $msg_size < 4096 ) {
$box_size+=4096;
} else {
$box_size+=$msg_size;
}
}
}

if ( $workdir =~ /(\d+)\@/ ) {
$msisdn=$1;
$all_mailbox_size+=$box_size;
if ( $box_size == 0 ) {
$empty_mailbox++;
} else {
&top_size_mailbox($msisdn,$box_size);
}
}

chdir($startdir) or die "Unable to change to dir $startdir: $!
+\n";
}


sub top_size_mailbox {

my ($msisdn,$box_size)=@_;

if ( keys( %top_size_mailbox ) < $num_top_size_box ) {
$top_size_mailbox{$box_size}=$msisdn;
} else {
my $min=(sort {$a <=> $b} keys %top_size_mailbox)[0];
if ( $box_size > $min ) {
delete $top_size_mailbox{$min};
$top_size_mailbox{$box_size}=$msisdn;
}
}

}

Code



      
    


rovf
Veteran

Feb 2, 2012, 1:07 AM

Post #2 of 2 (1566 views)
Re: [gdan2000] Optimizing performance for script to traverse on filesystem [In reply to] Can't Post

You were reinventing the wheel. Have a look at the File::Find module:

perldoc File::Find

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives