CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
File size limitation in perl ???

 



yaniv_av
Novice

Nov 11, 2002, 1:50 PM

Post #1 of 5 (476 views)
File size limitation in perl ??? Can't Post

I do some manipulation on a large file, then I'm writing the results (some format of index to the original file) to a new file, but the writing to the new file has stoped at some point and not all the lines I want to write acually writen to it...
(the reading from the file is ok because I have printed to the screen all the lines that I'm going to write to the new file and all the lines was printed. the writing itself wasn't ok)
When I check the file size it was only 2.82M. not too much...
Why does it happened ? And how can I fix it ?


podmaster
Novice

Nov 11, 2002, 11:54 PM

Post #2 of 5 (470 views)
Re: [yaniv_av] File size limitation in perl ??? [In reply to] Can't Post

Perl does potentially have file size limitations,
but we're talking about 2GB here (and it's usually a OS issue before it's a perl issue, and you can compile perl with large file support, meaning bigger than 2GB).

The reason your file gets truncated only you can know.
Post some code and someone will help you out.


yaniv_av
Novice

Nov 12, 2002, 6:14 AM

Post #3 of 5 (465 views)
Re: [podmaster] File size limitation in perl ??? [In reply to] Can't Post

Well, I'm doing some text manipulation on a large file and save the results in the array @file.
The I'm doing this code:

open(OUT, ">index.dat");
.
.
.
foreach (@file) {
$loc=O;
$i++;
$_=~s/(<body>)//g;
@record=split(/ /,join(" ",$_));
foreach (@record) {
if ( $_ =~ /[a-z]{3,13}/) {
$_=~s/[^a-z]//g;
printf ("%-15s : %-2d : %-4d \n",$_,$index_file [$i],$loc);
printf OUT ("%-15s : %-2d : %-4d \n",$_,$index_file[$i],$loc);
$count{$_}++;
}
$loc++;
}
}
close(OUT);

----------------
the first printf command prints all the lines I want to the screen (I added it only for debug) but when I check the output file which is created by the same command - It dosen't contain all the lines, but stop suddenly at some point.


Paul
Enthusiast

Nov 12, 2002, 6:17 AM

Post #4 of 5 (463 views)
Re: [yaniv_av] File size limitation in perl ??? [In reply to] Can't Post


Quote
Well, I'm doing some text manipulation on a large file and save the results in the array @file.


That could be your problem. Storing everything in an array uses up lots of memory and you are probably running out. Use a while() loop instead of saving everything in an array.


yaniv_av
Novice

Nov 13, 2002, 8:56 AM

Post #5 of 5 (452 views)
Re: [RedRum] File size limitation in perl ??? [In reply to] Can't Post

But if this is the problem, so why I succssed to print all the lines to the screen but not to write them to the file ?!?

If I was out of memory at some point then the printing to screen also ware failed, no ?

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives