CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Logging in and exploring a website

 



taha
New User

Feb 24, 2013, 9:33 AM

Post #1 of 4 (853 views)
Logging in and exploring a website Can't Post

Hi,

I am a member of an educational site with hundreds of videos. The videos are unfortunately not downloadable, but it is easy to find their links in html source code. So I want to write a script that logs into the website as me and explores the website to find all links to lecture videos' sqf files. Logging in is the part I am stuck at. I already tried reading the browser cookies using the LWP and HTTP::Cookies to fool the site into thinking that the script is authenticated, but I failed. Now I am trying to log in by submitting a form. Here is the code I wrote :


use LWP ;

%form = ('name' => 'name',
'pass' => 'pass',
'form_build_id' => 'form-fcffc0dc01d85f31e59f6b23ab05e7b3',
'form_id' => 'user_login',
'op' => 'Log in' );
$url = 'https://login.php';
$browser = LWP::UserAgent -> new(file => 'browdat.txt', autosave =>1);
$browser -> cookie_jar({});
$l= $browser -> post($url , \%form);


Code
 
I thought that the browser would contain the session cookies and that it would save them to the autosave file. But it does not do that, since the browdat.txt is empty after I run the program. So can you tell me how I can submit the login form and get the authentication cookies that I can use to view the next pages?


Kenosis
User

Feb 24, 2013, 2:01 PM

Post #2 of 4 (846 views)
Re: [taha] Logging in and exploring a website [In reply to] Can't Post

Does the educational site offer an API for such an effort? If not, do their Terms of Service permit 'script access' to their resources?


taha
New User

Feb 24, 2013, 7:48 PM

Post #3 of 4 (843 views)
Re: [Kenosis] Logging in and exploring a website [In reply to] Can't Post

No, the site is a basic drupal website. It has a forum section and a section with all of video lectures. Their terms of service does not mention script access, but I didn't think that would matter. Can't LWP forge the actions of real browser? If no, is there a way to program the browser to this?


lightspd
Novice

Feb 25, 2013, 6:14 AM

Post #4 of 4 (831 views)
Re: [taha] Logging in and exploring a website [In reply to] Can't Post

have you looked into using
http://search.cpan.org/~mstrout/WWW-Mechanize/lib/WWW/Mechanize.pm

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives