CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate: Perl Script Causing High CPU Usage: Edit Log



smart_amorist
Novice

Sep 19, 2013, 6:35 PM


Views: 5725
Perl Script Causing High CPU Usage

Perl Script Causing High CPU Usage

We have an application that generates huge number of files (3 - 4 lakh including subdirectories) under multiple directories. In order to archive, move or delete these files based on files age, I have written a simple perl script but sometime it is causing high CPU spikes(%CPU in Linux top outpu 60-100%). Just spikes it never continuously use high CPU.


Example:

/app/home/DIR_1/ABC_DEF.xml
/app/home/DIR_2/GHI_JKL.xml
/app/home/DIR_3/MNO_PQR.xml

or subdirectories

/app/home/DIR_1/SubDir_YYYYMMDD



$PatternToMatch ="???_???.xml";

or

$PatternToMatch = "SubDir_YYYYMMDD";

Now script check for each directory in loop and read matching files as below.

foreach(@dir_list){

chdir($_);

while(glob($PatternToMatch)) {
$age = -M $_;
if($age > 7) {
unlink $_;
or
qx(mv source destination);
or
gzip $_;
}
}

}


Noticed that when script is switching from one directory to other, it is causing CPU spikes. Strange thing is if are more files matched, then there is no high CPU spikes (10-15%), but if it only found around 30-60 files in each dir, i can see spikes. I have put sleep after each dir to bring down cpu.

What is the best way to match files based on age?
Any idea what is causing high CPU spikes.
%CPU shows high value sometime but at same time overall CPU usage given by top, mpstat shows hardly from 4-5%, this is confusing me.

If anyone has any best idea/script to achieve it please help me.


(This post was edited by smart_amorist on Sep 19, 2013, 6:38 PM)


Edit Log:
Post edited by smart_amorist (Novice) on Sep 19, 2013, 6:38 PM


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives