
budman
User
Feb 25, 2012, 7:27 PM
Post #4 of 11
(20943 views)
|
Re: [KingNothing] Release memory from threads
[In reply to]
|
Can't Post
|
|
The loop really didn't help much. I created an example basing it on your loop contents. Do you have loops that check if the threads are ok to join (is_joinable) and ok to exit (is_running)? About join, this means block or wait until the thread is complete and then release the thread and do any OS cleanup thats needed. Its important to join. Detach means to ignore the thread, which will just exit and release when its done. I ran this on linux, adding the sleep after everything was done, allowed the system time to free memory. In this run, it freed up ~80KB. Amt Free Mem at Start : 2304404 Amt Free Mem w/Threads : 2299016 (-5388) Amt Free Mem at End : 2387748 (83344)
#!/usr/bin/perl use strict; use warnings; # example of the ext1.pm { package ext1; use Exporter; our @EXPORTS_OK=qw(submit reset); sub submit { my ($id, $file) = @_; print "Thread $id: submit $file\n"; keys ( my %h ) = 1000; sleep(2); } sub reset { my ($id, $file) = @_; print "Thread $id: reset $file\n"; keys ( my %h ) = 1000; sleep(15); } 1; } # would normally choose use ext1 if was separate file ext1.pm # but since I have the package in the same file, I don't want to run require. import ext1; #use ext1; use threads; use Data::Dumper; $|++; my @files = (['file1',1],['file2',1],['file3',0],['file4',1],['file5',1]); my $options = {'exit'=>'thread_only'}; my $bar = "=" x 40; # open the threads my $start_mem = free_mem(); print "$bar\nSubmit processes\n$bar\n"; my @results; foreach my $i (0 .. $#files) { my ($file,$choice) = @{$files[$i]}; if ( $choice == 1 ) { $results[$i] = threads->create( $options, sub { &ext1::submit($i,$file) } ); } else { $results[$i] = threads->create( $options, sub { &ext1::reset($i,$file) } ); } } # once threads are submitted, then need to either be joined or detached. # wait for each thread to be ready to be joined. sleep(2); my $thread_mem = free_mem(); print "$bar\nJoining threads\n$bar\n"; foreach my $i (0..$#results) { print "thread $i: "; while ( !$results[$i]->is_joinable() ) { print "."; sleep(1) } print "ready\n"; #$results[$i]->join; # can join here as well } # join all the threads foreach my $r (@results) { $r->join; } # check that all threads freed up print "$bar\nThread results\n$bar\n"; foreach my $i (0 .. $#results) { print "thread $i: "; while ( $results[$i]->is_running() ) { print "."; sleep(1) } print "finished\n"; #print Dumper($results[$i]); } sleep(10); print "$bar\nMemory\n$bar\n"; my $finish_mem = free_mem(); printf "Amt Free Mem at Start : %d\n", $start_mem; printf "Amt Free Mem w/Threads : %d (%d)\n", $thread_mem, $thread_mem - $start_mem; printf "Amt Free Mem at End : %d (%d)\n", $finish_mem, $finish_mem - $start_mem; print "done\n"; exit 0; sub free_mem { return (split /\s+/, qx(free | grep ^Mem:) )[3]; } Output:
======================================== Submit processes ======================================== Thread 0: submit file1 Thread 1: submit file2 Thread 2: reset file3 Thread 3: submit file4 Thread 4: submit file5 ======================================== Joining threads ======================================== thread 0: ready thread 1: ready thread 2: .............ready thread 3: ready thread 4: ready ======================================== Thread results ======================================== thread 0: finished thread 1: finished thread 2: finished thread 3: finished thread 4: finished ======================================== Memory ======================================== Amt Free Mem at Start : 2304404 Amt Free Mem w/Threads : 2299016 (-5388) Amt Free Mem at End : 2387748 (83344) done
(This post was edited by budman on Feb 25, 2012, 7:43 PM)
|