Home: Perl Programming Help: Intermediate:
LWP timing problems


Apr 2, 2000, 1:34 PM

Views: 1870
LWP timing problems

Hi I've implemented LWP::UserAgent for my program. I've to verify a list of about 500 urls. I know there is a timeout function in that module and if I do not specify it, it will be the default one of 180 seconds. However I realised that there are some sites that takes longer than 3 minutes to verify. I've attached my code to this mail. Can anyone advice me if I made a mistake somewhere? I'm sorry that I can't indent my coding because the space to put my coding is too small. I'm sorry for the mess. Thanks in advance.

sub LinkVerifier1 {
use LWP::UserAgent;
require HTTP::Headers;
my $ua = new LWP::UserAgent;
$ua->agent("AgentName/0.1 " . $ua->agent);
foreach $KeyUrl (keys %ResultsArray) {
my $req = new HTTP::Request HEAD => $KeyUrl;
my $res = $ua->request($req);
if ($res->is_success) {
print "$KeyUrl-Success: ";
if ($res->last_modified) {
print "date updated ";
$TempLastModified = $res->last_modified;
if ($TempLastModified > $ResultsArray {$KeyUrl}{DATEMODIFIED})
{$ResultsArray{$KeyUrl}{DATEMODIFIED} = $TempLastModified;
} else {
if ($res->content_length) {
print "content updated ";
$TempContentLength = $res->content_length;
$ResultsArray{$KeyUrl}{FILESIZE} = $TempContentLength;
print "\n";
} else {
print "Broken link: $KeyUrl deleted!!!\n";
delete $ResultsArray{$KeyUrl};
push (@BrokenLinkArray, $KeyUrl);