Perl Guru Forums: Perl Programming Help: Intermediate: Understanding Parallell::ForkManager

CGI/Perl Guide | Learning Center | Forums | Advertise | Login

Site Search: in


	MAIN INDEX	SEARCH POSTS	WHO'S ONLINE	LOG IN

Home: Perl Programming Help: Intermediate:
Understanding Parallell::ForkManager

gw1500se
New User

Oct 23, 2012, 5:54 AM

Post #1 of 2 (3001 views)

Understanding Parallell::ForkManager	Can't Post

I have a large number of sub-tasks that I want to run in parallel. Some of the tasks are quite short while others are long (hours) so I set up a hash that contains the pid ($pm->start) and a name. When the task is 'start'ed I set the values in the hash then add that hash to an array. In theory the array should contain the information I need for each running task. When a task finishes I have a 'run_on_finish' defined that removes itself from the array. The problem is that it seems things are not happening in the way I understood the documentation. Within my loop, I thought the child executes everything between 'start and next' and 'finish'. In the meantime the parent continues executing everything after 'finish' to the end of the loop. Then the loop runs again creating a new child and the parent executes to the end of the loop again. Here is my code:

Code

foreach my $url (@urls) { 
         unless ($seen{$url}++) { 
         	my $proc=process->new(); 
		$proc->pid($pm->start and next); 
                chomp($url); 
                trim($url); 
	        print "*************Processing $url*******************\n"; 
                system("$helper_utils/collect_user_metrics.pl", "$url",">$tempoutput/collect_user_metrics.txt"); 
               system("$helper_utils/get_docroot_info.pl", "$pool", "$url",">$tempoutput/get_docroot_info.txt"); 
              $pm->finish(); 
                print("Adding new child to array\n"); 
               $proc->name($url); 
		push(@processed,$proc); 
         } 
}

My debug print ('adding new child') is never output but the 'Processing' is. Can someone clear up how this works for me? TIA.

wickedxter
User

Oct 23, 2012, 6:19 PM

Post #2 of 2 (2998 views)

Re: [gw1500se] Understanding Parallell::ForkManager [In reply to]	Can't Post

are u workin on windows or linux? because fork on windows doesnt work as well at it does on linux

this is the basic way to use it..

Code

use strict; 
use warnings; 
use Parallell::ForkManager; 
 
 
my $fork = Parallell::ForkManager->new(4); #number of max forks 
 
 
for(0..1000){ 
  my $pid = $fork->start and next; 
 
  #work here for the forks to do; 
  print "PID: $pid ::: WORK: $_ \n"; 
 
  $fork->finish; 
} 
$fork->wait_all_children;

(This post was edited by wickedxter on Oct 24, 2012, 7:27 AM)

Search for (options)

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives