CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
Search Posts SEARCH
Who's Online WHO'S
Log in LOG

Home: Perl Programming Help: Intermediate:
Memory Leak



Sep 30, 2009, 1:57 PM

Post #1 of 3 (1116 views)
Memory Leak Can't Post

I am working with a file of about 15 GB (I have 98 GB of RAM) taking the lines into a hash and checking to see if they are repeated so:

if ( defined ( $masterHash{$line} ) ) 
$masterHash{$line} = 1;

This continues on printing out the "$masterHash" periodically based on predefined parameters ie what percent of the data in the following fashion:

my $hereFile = $littleFile . "/" . "${FNUM}-$tmpFile"; 
$hereFile = "$hereFile"."$orgCount"."organisms_"."$percent"."percent_"."$count"."unique_motifs.txt";
push( @files, $hereFile );
open( HERE, ">$hereFile" )or warn ;

print HERE "#$orgCount organsisms\t$percent percent of total unique motifs\t$count total unique motifs\n";
foreach my $key ( keys %masterHash )
print HERE "$key\t$masterHash{$key}\n";
close( HERE );

When doing this the memory fills up really fast to the point that I crash my system, I figure I have a memory leak, but I am not quite sure how perl stores the memory nor am I sure how to get around this. I have tried using the module to autoflush the buffer, but that only slowed the program down and did not seem to free up any memory, any ideas?

Veteran / Moderator

Sep 30, 2009, 4:16 PM

Post #2 of 3 (1103 views)
Re: [adussaq] Memory Leak [In reply to] Can't Post

Here are a couple modules that will help you to troubleshoot your script.

Devel::ObjectTracker - Track object life to detect memory leaks

Devel::Monitor - Monitor your variables/objects for memory leaks

Devel::Size - Perl extension for finding the memory usage of Perl variables

The code you posted does have a couple issues, but it's too small of a sample to say if it's part of the problem you describe.


Sep 30, 2009, 4:24 PM

Post #3 of 3 (1099 views)
Re: [FishMonger] Memory Leak [In reply to] Can't Post

Thank you I will give that a try, if you have time can you tell me what the "issues" are with my code, I have been programming perl for a while, but I need to get better and I have a feeling that these issues arise in all of my code.....


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives