CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Out of memory error

 



jeff
Deleted

Nov 11, 2000, 8:29 PM

Post #1 of 3 (296 views)
Out of memory error Can't Post

Hi guys. I have a small problem and I am not sure what to do. I wrote a small program that loads some data into various arrays, manipulates it abit, and then writes out 5 html files.It runs fine here on my little 350mhz but I get an "out of memeory" error on the server. It wouldn't be suprising if it was loading a mountain of data, but I am only asking it to load up about 4,000 different chunks into 4 different arrays.

Does anyone have an idea on where I have sinned? I am somewhat new to this. Thanks
jeff


vpopper
Deleted

Nov 11, 2000, 9:58 PM

Post #2 of 3 (296 views)
Re: Out of memory error [In reply to] Can't Post

  <BLOCKQUOTE><font size="1" face="Arial,Helvetica,sans serif">quote:</font><HR>It runs fine here on my little 350mhz but I get an "out of memeory" error on the server<HR></BLOCKQUOTE>

The user running the process on the server is limited in several ways regarding the amount of memory and CPU power the user can use. If the perl process has reached the memory usage limit, for example, you'll receive an "Out of memory!" error.

For example, on some systems the file /etc/login.conf defines a login "class" that the user belongs to. The user may be limited with these sample attributes:

:cputime=1h30m:\
:datasize=8M:\
:stacksize=2M:\
:memorylocked=4M:\
:memoryuse=8M:\
:filesize=8M:\
:coredumpsize=8M:\
:openfiles=24:\
:maxproc=32:\

Regardless of the user'slogin class, the user may be limited in the server's httpd.conf config file:

RLimitCPU 60 60
RLimitMEM 16777216 16777216
RLimitNPROC 16 16

Talk to your sysadmin about changing your limits. Hope this helps!




[This message has been edited by vpopper (edited 11-11-2000).]


jeff
Deleted

Nov 11, 2000, 11:54 PM

Post #3 of 3 (296 views)
Re: Out of memory error [In reply to] Can't Post

Thanks vpopper. Yes, I was wondering if I was hitting the limits of user allocation and I am sure you are correct. I guess my suprise was that I couldn't load into memory some stuff that couldn't amount to more than 50k. Oh well, my answer probably lies in re-writing. I include the code. Thanks again. Jeff

bdir="c:/story";
$ddir="$bdir/data";
$utilitydir="$bdir/utility";

$title_and_key="$ddir/title_and_key.txt";
$alphabetical_titles="$ddir/alphatitles.txt";
$alphabetical_newtitles="$ddir/alphanewtitles.txt";
$database="$ddir/database.txt";

$siteindex="$bdir/archive.html";
$newsiteindex="$bdir/newfiles.html";
$subindex="$bdir/alpha";
$alphaheader="$utilitydir/header.htm";
$alphafooter="$utilitydir/footer.htm";
$newfileheader="$utilitydir/header.htm";
$newfilefooter="$utilitydir/footer.htm";
$subalphaheader="$utilitydir/header.htm";
$subalphafooter="$utilitydir/footer.htm";

$newdate=6;
$newdate*=24;

@breaks=qw(h n); #where to break the alphabet wherever desire. change as needed

#MAIN PROGRAM

&initialization; #read in all the data

#print alphabetical site index
$pagename=$siteindex; #set the name of the web page - a to z index
$header=$alphaheader; #set the name of the header
$flag=1; #flag tells program we want subindex headings printed
@titles=@alphatitles; #set the titles we will print
%key=%titles; #set the primary keys to our database
$footer=$alphafooter; #set the name of the footer
&print_webpage; #write the webpage

#print newfiles site index
$pagename=$newsiteindex; #set the name of the web page - new files index
$header=$newfileheader; #set the name of the header
$flag=(); #flag says we don't want subindex headings
@titles=@newalphatitles; #set the titles we will print
%key=%titles; #set the primary keys to our database
$footer=$newfilefooter; #set the name of the footer
&print_webpage; #write the webpage

#print sub-alphabetical site index
$len=@breaks; #how many breaks in the alphabet will we be taking?

for ($j=0;$j<=$len;++$j){

until (substr($k,0,1) eq uc $breaks[$j]){ #do until first character of title equals letter break

$k=shift(@alphatitles); #remove first title
push (@subtitles,$k); #add that title to a list of titles to print
}#until (substr($k,0,1) eq uc $breaks[$j])

unshift (@alphatitles,$k); #put last title back into the alphabetical titles
pop (@subtitles); #we are doing an alphabet break so remove last title

$pagename="$subindex".($j+1).".html"; #set title of page (alpha1.htm, alpha2.htm, etc)
$header=$subalphaheader; #set the name of the header
$flag=1; #flag tells program we want subindex headings printed
@titles=@subtitles; #set the titles we will print
%key=%titles; #set the primary keys to our database
$footer=$subalphafooter; #set the name of the footer
&print_webpage; #write the webpage
@subtitles=(); #clear the list of titles to print
}#for ($j=0;$j<=$len;++$j)

#END OF MAIN PROGRAM

#print whatever webpage we are told to
sub print_webpage{

open(OUTPUT,">$pagename"); #open the webpage to write
$inputfile=$header; #set the file to read the header html from
&print_misc; #read our header and print it to the html form
&print_body; #write the body of our html form
$inputfile=$footer; #set the file to read the footer html from
&print_misc; #read our footer and print it to the html form
close(OUTPUT); #close the webpage
}#subroutine

#print the main body of the webpage
sub print_body{

$length=@titles; #how many titles are we dealing with?

for ($i=0;$i<$length;++$i){ #cycle through all the files

if($flag){ #if we want subindex heading printed, check
if ($firstletter ne substr($titles[$i],0,1)){ #did last title have a different first letter?
&subindex_header; #if so, print new subindex heading
}#if ($firstletter ne substr($titles[$i],0,1))
}#if($flag)

$pointer=$key{$titles[$i]}; #read the primary key from the title/key hash
$url=$datab[$pointer][0]; #read the file url from the database using pk
$ctime=$datab[$pointer][1]; #read the file creation date from database using pk

print OUTPUT "<a href=http://$url>$titles[$i]</a>"; #write the url and title out nicely to webpage
$newtime=(time-$ctime)/3600;
if ($newtime<=$newdate && $flag){ #file is new and prints index headers (not newstory html)
print OUTPUT " <img border=0 src=$utilitydir/BulletCheck.gif align=middle width=34 height=15>";
} #print the newfile bullet
print OUTPUT "<br>\n";
}#for ($i=0;$i<$length;++$i)

return; #done writing body of webpage
}#subroutine

#read whatever header or footer file we pass to this
sub print_misc{

open(INPUT,"<$inputfile") #open file we recieved from caller
&#0124; &#0124;die "you mother****er $!"; #crash and burn

while(<INPUT> ){ #read the file and while there is something to read
print OUTPUT $_; #print the input to webpage
}#while(<INPUT> )

close(INPUT); #there is nothing more in file so close it up

return; #done writing header or footer
}#subroutine

#write subindex headers (index a, index b, etc)
sub subindex_header{
$firstletter=substr($titles[$i],0,1); #read the first letter of the title
# print OUTPUT "<center>\n"; #write to html body: center everything
# print OUTPUT "<h3>Index $firstletter</h3>\n"; # write out "Index" whatever (A,B,C,etc)
# print OUTPUT "<hr width=\"50%\">\n"; # write out a horizontal line
# print OUTPUT "</center>\n"; # uncenter everything

print OUTPUT "<p><br><table border=0 width=50% cellspacing=0 cellpadding=0><tr>";
print OUTPUT "<td><p><b><font size=4>Titles - Index $firstletter</font> </b></p></td>";
print OUTPUT "</tr><tr><td bgcolor=#000000 height=2><img border=0 src=1pixelspacer.gif width=1 height=2></td></tr>";
print OUTPUT "</table>";

return; #done writing subindex headers
}#subroutine

#read all variables in
sub initialization{

$inputfile=$alphabetical_titles; #read all file titles presorted in alphanumeric order
&read_file;
@alphatitles=@scratchpad;

$inputfile=$title_and_key; #read all titles and thier corresponding secondary key
&read_file;
%titles=@scratchpad;

$inputfile=$alphabetical_newtitles; #read all new file titles presorted in alphanumeric order
&read_file;
@newalphatitles=@scratchpad;

$inputfile=$database; #read all urls and thier corresponding primary key
&read_file;
while (@scratchpad){
for($i=0;$i<=2;++$i){
$i1=shift(@scratchpad);
$i2=shift(@scratchpad);
$i3=shift(@scratchpad);
$datab[$i1][0]=$i2;
$datab[$i1][1]=$i3;
}
}
# %datab=@scratchpad;

return; #done reading all variables
}#subroutine

#read a file without caring what it is
sub read_file{

@scratchpad=(); #initialize scratchpad variable to null

open(INPUT,"<$inputfile") #open whatever file is passed to this
&#0124; &#0124;die "you mother****er $!"; #crash and burn

while(<INPUT> ){ #while there is something in the file
chomp ($_); #remove cr and linefeed
push(@scratchpad,$_); #shove input onto scratchpad variable
}#while(<INPUT> )

close(INPUT); #there is nothing more to read so close file

return; #done reading whatever file
}#subroutine

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives