CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
Serious memory leak using LWP

 



ajmcello
Novice

Mar 7, 2012, 7:14 PM

Post #1 of 3 (895 views)
Serious memory leak using LWP Can't Post

 
If I run the following code, it results in a memory leak. I monitor some internal websites, and after a few hours or days,the memory can become outrageous, sometimes running the machine out of the memory! I didn't put in a sleep or delay just so that you can see the problem, but when you run this code you'll see the memory usage just go insane. You can change the url to some bogus website without a properly formatted address, like $url="lksjfdlsjf" and you'll see the memory leak happen as well, and much quicker.

Is there a way to resolve this issue?

Thanks in advance!



Code
#!/usr/bin/perl 

use DBI;
use DBD::Pg;
use POSIX;

use HTML::Parse;
use LWP::Simple;
use URI::URL;


while(1) {

$url = "http://cnn.com";
$content = get $url;
$content = parse_html($content)->format;

print "$content\n";
}



wickedxter
User

Mar 8, 2012, 12:10 PM

Post #2 of 3 (882 views)
Re: [ajmcello] Serious memory leak using LWP [In reply to] Can't Post

This will result in memory consumption because your useing and infinite loop which is constantly sending request to the server and getting data back and holding everything in memory you can either do a cron job or schdule the script to run and close.

This works better, use system call so the other program doing the data retrieve is forked and returned to parent.

test2.pl

Code
 
while (1){

system ('c:/server/cgi-bin/test1.pl');
sleep(10);
}



file test1.pl

Code
use HTML::Parser;  
use LWP::Simple;

my $p = HTML::Parser->new();
my ($url, $content);

$url = "http://www.cnn.com";
$content = get $url;

print $p->parse($content);



(This post was edited by wickedxter on Mar 8, 2012, 12:11 PM)


ajmcello
Novice

Mar 8, 2012, 5:01 PM

Post #3 of 3 (873 views)
Re: [wickedxter] Serious memory leak using LWP [In reply to] Can't Post

 
Thanks. That's an interesting way of getting around the problem. I see what you're saying about why it continues to eat up memory. I sort of got around it by using fork and killing the parent, but it just seemed like there should be a better way of doing it without using a system for a fork call. I'm no experienced programmer, though.

I appreciate the response, thank you.

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives