CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX 		Search Posts SEARCH
POSTS 		Who's Online WHO'S
ONLINE 		Log in LOG
IN

Home: Perl Programming Help: Advanced:
The need for speed

 


Gregorio
User

Aug 10, 2000, 12:04 PM

Post #1 of 5 (802 views)
The need for speed Can't Post
I made a script that uses get commands with lwp::simple to get information from 5 different pages, i.e.

my $pageurl = "http://www.sitename.com";
my $page = get ($pageurl);

it also uses about 35-40 matches, ie.

if ($page =~ m/<TR><TD>([^<]+)<.TD><.TR>/) {
($information) = ($1);
}

It works, EXCEPT it takes about 2 minutes to load, not good, is there anyway to speed up the script or get faster? Any recommendations?


dws
Deleted

Aug 10, 2000, 1:40 PM

Post #2 of 5 (802 views)
Re: The need for speed [In reply to] Can't Post
Your first order of business it to measure where the time is going. Try something like the following to separate out the time it takes to fetch the page.
<BLOCKQUOTE><font size="1" face="Arial,Helvetica,sans serif">code:</font><HR>

my $start = time();
my $page = get($pageurl);
my $elapsed = time() - $start;
print "get($pageurl) took $elapsed second(s)\n";</pre><HR></BLOCKQUOTE>


Gregorio
User

Aug 10, 2000, 6:32 PM

Post #3 of 5 (802 views)
Re: The need for speed [In reply to] Can't Post
Ok, one of the sites takes 24 seconds to load! 2 take 11,13 and 2 take 4,5. So would it be possible to make the script get only the html from the page? Or any other way to speed up the time?

[This message has been edited by Gregorio (edited 08-10-2000).]


Gregorio
User

Aug 11, 2000, 11:38 AM

Post #4 of 5 (802 views)
Re: The need for speed [In reply to] Can't Post
Any way to do this without deleting the url?


Kanji
User / Moderator

Aug 11, 2000, 1:31 PM

Post #5 of 5 (802 views)
Re: The need for speed [In reply to] Can't Post
Hard to really comment when you provide so little information; the URLs you pull in only account for a minute of the two minutes you say it takes.

What in your script takes up that remaining minute?

Bear in mind also that if you do manage to speed up rest of your script, it *is* dependant on network connectivity and the response time of the web server you're getting from, so you'll still stuck at about a minute's processing unless you preemptively cache the URLs you get.

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives