CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX 		Search Posts SEARCH
POSTS 		Who's Online WHO'S
ONLINE 		Log in LOG
IN

Home: Perl Programming Help: Beginner:
WWW::Mechanize looping through site archive pages?

 


LordXenu
Novice

Feb 24, 2009, 6:43 PM

Post #1 of 3 (137 views)
WWW::Mechanize looping through site archive pages? Can't Post
Would it be possible to create a loop using WWW::Mechanize so as to allow my action to be performed on each page in sequence through a site's archives by clicking a "next page" link $browser->follow('Next Page'); ?

I'm mining data from each page in an online shopping cart, and need this action to be performed on each page as it arries through from page 1.

If ou coud point me in the right general direction i'm ready to stomp away on some trial and error...


1arryb
User

Feb 25, 2009, 9:56 AM

Post #2 of 3 (131 views)
Re: [LordXenu] WWW::Mechanize looping through site archive pages? [In reply to] Can't Post
Hi LordXenu,

You probably could do that, but it seems like overkill. WWW::Mechanize is really intended for sites that require you to interact with them via forms (e.g., login, accept license agreement, etc.). For simple site traversal, you can use HTML::TreeBuilder to look for an 'a' (anchor) tag with the text "Next Page", get the url from the link, and fetch the next page with LWP.

Cheers,

Larry


(This post was edited by 1arryb on Feb 25, 2009, 9:57 AM)


LordXenu
Novice

Feb 25, 2009, 11:52 AM

Post #3 of 3 (125 views)
Re: [LordXenu] WWW::Mechanize looping through site archive pages? [In reply to] Can't Post
the page i am working with has a very simple structure to their urls. so each page of the archive is page=1 page=2 etc etc. so i'm thinking that if i created an array of sorts i an loop through performing my action on each url from the array. just gotta figure out how i can create my array, should hopefully be able to figure that out. all i need to do is take an input url and, select the number of entries, and have it create an array adding each number up to my selected value to the end of the input url. then i should be able to take care of my action for a for each loop, correct?

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives