CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Understanding Parallell::ForkManager

 



gw1500se
New User

Oct 23, 2012, 5:54 AM

Post #1 of 2 (1121 views)
Understanding Parallell::ForkManager Can't Post

I have a large number of sub-tasks that I want to run in parallel. Some of the tasks are quite short while others are long (hours) so I set up a hash that contains the pid ($pm->start) and a name. When the task is 'start'ed I set the values in the hash then add that hash to an array. In theory the array should contain the information I need for each running task. When a task finishes I have a 'run_on_finish' defined that removes itself from the array. The problem is that it seems things are not happening in the way I understood the documentation. Within my loop, I thought the child executes everything between 'start and next' and 'finish'. In the meantime the parent continues executing everything after 'finish' to the end of the loop. Then the loop runs again creating a new child and the parent executes to the end of the loop again. Here is my code:

Code
foreach my $url (@urls) { 
unless ($seen{$url}++) {
my $proc=process->new();
$proc->pid($pm->start and next);
chomp($url);
trim($url);
print "*************Processing $url*******************\n";
system("$helper_utils/collect_user_metrics.pl", "$url",">$tempoutput/collect_user_metrics.txt");
system("$helper_utils/get_docroot_info.pl", "$pool", "$url",">$tempoutput/get_docroot_info.txt");
$pm->finish();
print("Adding new child to array\n");
$proc->name($url);
push(@processed,$proc);
}
}

My debug print ('adding new child') is never output but the 'Processing' is. Can someone clear up how this works for me? TIA.


wickedxter
User

Oct 23, 2012, 6:19 PM

Post #2 of 2 (1118 views)
Re: [gw1500se] Understanding Parallell::ForkManager [In reply to] Can't Post

are u workin on windows or linux? because fork on windows doesnt work as well at it does on linux

this is the basic way to use it..


Code
use strict; 
use warnings;
use Parallell::ForkManager;


my $fork = Parallell::ForkManager->new(4); #number of max forks


for(0..1000){
my $pid = $fork->start and next;

#work here for the forks to do;
print "PID: $pid ::: WORK: $_ \n";

$fork->finish;
}
$fork->wait_all_children;



(This post was edited by wickedxter on Oct 24, 2012, 7:27 AM)

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives