CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Advanced: Re: [KingNothing] Release memory from threads: Edit Log



budman
User

Feb 25, 2012, 7:27 PM


Views: 12229
Re: [KingNothing] Release memory from threads

The loop really didn't help much.
I created an example basing it on your loop contents.

Do you have loops that check if the threads are ok to join (is_joinable)
and ok to exit (is_running)?

About join, this means block or wait until the thread is complete
and then release the thread and do any OS cleanup thats needed.
Its important to join. Detach means to ignore the thread, which will just exit and release when its done.


I ran this on linux, adding the sleep after everything was done,
allowed the system time to free memory. In this run, it freed up ~80KB.

Amt Free Mem at Start : 2304404
Amt Free Mem w/Threads : 2299016 (-5388)
Amt Free Mem at End : 2387748 (83344)




Code
#!/usr/bin/perl 
use strict;
use warnings;

# example of the ext1.pm
{
package ext1;
use Exporter;
our @EXPORTS_OK=qw(submit reset);

sub submit {
my ($id, $file) = @_;
print "Thread $id: submit $file\n";
keys ( my %h ) = 1000;
sleep(2);
}

sub reset {
my ($id, $file) = @_;
print "Thread $id: reset $file\n";
keys ( my %h ) = 1000;
sleep(15);
}
1;
}

# would normally choose use ext1 if was separate file ext1.pm
# but since I have the package in the same file, I don't want to run require.
import ext1;
#use ext1;

use threads;
use Data::Dumper;
$|++;

my @files = (['file1',1],['file2',1],['file3',0],['file4',1],['file5',1]);
my $options = {'exit'=>'thread_only'};
my $bar = "=" x 40;

# open the threads

my $start_mem = free_mem();
print "$bar\nSubmit processes\n$bar\n";
my @results;
foreach my $i (0 .. $#files) {
my ($file,$choice) = @{$files[$i]};
if ( $choice == 1 ) {
$results[$i] = threads->create( $options, sub { &ext1::submit($i,$file) } );
} else {
$results[$i] = threads->create( $options, sub { &ext1::reset($i,$file) } );
}
}

# once threads are submitted, then need to either be joined or detached.

# wait for each thread to be ready to be joined.

sleep(2);
my $thread_mem = free_mem();
print "$bar\nJoining threads\n$bar\n";
foreach my $i (0..$#results) {
print "thread $i: ";
while ( !$results[$i]->is_joinable() ) { print "."; sleep(1) }
print "ready\n";
#$results[$i]->join; # can join here as well
}
# join all the threads
foreach my $r (@results) {
$r->join;
}

# check that all threads freed up

print "$bar\nThread results\n$bar\n";
foreach my $i (0 .. $#results) {
print "thread $i: ";
while ( $results[$i]->is_running() ) { print "."; sleep(1) }
print "finished\n";
#print Dumper($results[$i]);
}

sleep(10);

print "$bar\nMemory\n$bar\n";
my $finish_mem = free_mem();

printf "Amt Free Mem at Start : %d\n", $start_mem;
printf "Amt Free Mem w/Threads : %d (%d)\n", $thread_mem, $thread_mem - $start_mem;
printf "Amt Free Mem at End : %d (%d)\n", $finish_mem, $finish_mem - $start_mem;
print "done\n";
exit 0;

sub free_mem {
return (split /\s+/, qx(free | grep ^Mem:) )[3];
}




Output:

Code
======================================== 
Submit processes
========================================
Thread 0: submit file1
Thread 1: submit file2
Thread 2: reset file3
Thread 3: submit file4
Thread 4: submit file5
========================================
Joining threads
========================================
thread 0: ready
thread 1: ready
thread 2: .............ready
thread 3: ready
thread 4: ready
========================================
Thread results
========================================
thread 0: finished
thread 1: finished
thread 2: finished
thread 3: finished
thread 4: finished
========================================
Memory
========================================
Amt Free Mem at Start : 2304404
Amt Free Mem w/Threads : 2299016 (-5388)
Amt Free Mem at End : 2387748 (83344)
done



(This post was edited by budman on Feb 25, 2012, 7:43 PM)


Edit Log:
Post edited by budman (User) on Feb 25, 2012, 7:29 PM
Post edited by budman (User) on Feb 25, 2012, 7:31 PM
Post edited by budman (User) on Feb 25, 2012, 7:43 PM


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives