CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX 		Search Posts SEARCH
POSTS 		Who's Online WHO'S
ONLINE 		Log in LOG
IN

Home: Perl Programming Help: Advanced:
a muti-threaded web client in perl

 


mozkill
Novice

Mar 14, 2003, 10:06 AM

Post #1 of 9 (2577 views)
a muti-threaded web client in perl Can't Post
i am trying to think up a way to do a multi threaded client application for load testing. i just bought the object oriented Perl book last night and i am sorta thinking that i am going to make a perl Class that handles one instance of a load test script and then run a whole bunch of them from one machine.

i want to do something similar to all the examples that exist of created a multi-threaded "forked" server listener, except i want to make a client version of it. i don't have a very good understanding of "forking" a process. perhaps i should do it that way instead of having a class object?

i have an already working script and running 80 instances of the perl interpreter at the same time with batch files uses over 500 mb of system memory. i need to instead launch one perl VM with mulitple script instances running within it so that it uses far less memory...

can anyone help me with a design recommendation, since i am about to attempt this and i want to do it correctly? i will share my code when i am done with it. there are no examples of this anywhere on the internet.


(This post was edited by mozkill on Mar 19, 2003, 8:53 AM)


mozkill
Novice

Mar 20, 2003, 3:29 PM

Post #2 of 9 (2555 views)
Re: [mozkill] a muti-threaded web client in perl [In reply to] Can't Post
my guess is that there probably isn't a single object oriented Perl programmer that belongs to this forum.


davorg
Thaumaturge / Moderator

Mar 24, 2003, 2:06 AM

Post #3 of 9 (2547 views)
Re: [mozkill] a muti-threaded web client in perl [In reply to] Can't Post
In Reply To
my guess is that there probably isn't a single object oriented Perl programmer that belongs to this forum.
Guess again :)

Personally, I haven't replied because I have no experience of multi-threaded applications.

--
Dave Cross, Perl Hacker, Trainer and Writer
http://www.dave.org.uk/
Get more help at Perl Monks


mozkill
Novice

Mar 28, 2003, 3:56 PM

Post #4 of 9 (2540 views)
Re: [davorg] a muti-threaded web client in perl [In reply to] Can't Post
well, here is what i have so far in a .pm file:

-------------------------

package LWPThread;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use strict;
use Carp;
use LWP::Debug qw(+conns);

sub new
{
my ($class) = @_;
my $ua_ref = LWP::UserAgent->new();
bless $ua_ref, $class;
testURL($ua_ref);
return 1;
}

sub testURL
{
my $ua_ref = shift;
$ua_ref->agent("perl-LWP\5.6.1");
my $url = 'http://web.pdx.edu/~jonw';
my $req = HTTP::Request->new(GET => $url);
my $res = $ua_ref->request($req);
my $code = $res->code;
my $content = $res->content;
if ($res->is_error()) {
printf "%s\n", $res->status_line;
} else {
my $content = $res->content();
}
printf $content;
}

1;
--------------------

and i call it from a perl file like so:

----------------

#! perl.exe -w
use lib "C:/Temp/mtlwp";
use LWPThread;
use strict;

LWPThread->new();


-----------------

in any case, i still cant get it to work. i am having a pass-by-reference issue that is hard to figure out. This code doesnt work "as is". I am still wrestling with it though. All i need is this simple little thing to work. :-(


(This post was edited by mozkill on Mar 28, 2003, 4:08 PM)


davorg
Thaumaturge / Moderator

Mar 31, 2003, 12:05 AM

Post #5 of 9 (2530 views)
Re: [mozkill] a muti-threaded web client in perl [In reply to] Can't Post
You should go back and read perlboot, perltoot and perlobj. You need to get a basic understanding of how Perl objects work before writing an object like this.

You create an object of type LWP::UserAgent but then re-bless the object as a LWPThread object. It's therefore no surprise that the object doesn't understand LWP::UserAgent methods when you try to call them. The LWP::UserAgent object should probably be an attribute of the LWPThread object.

Also you return 1 from the "new" method. This won't affect your tests as you're doing all the processing in the "new" method, but it's generally accepted that you should return the newly constructed object from a constructor method.

--
Dave Cross, Perl Hacker, Trainer and Writer
http://www.dave.org.uk/
Get more help at Perl Monks


mozkill
Novice

Mar 31, 2003, 9:12 AM

Post #6 of 9 (2528 views)
Re: [davorg] a muti-threaded web client in perl [In reply to] Can't Post
thanks. i didn't know about the perl object oriented tutorials. the object oriented Perl book is ok, but coming from a background with only 1st year C++, its not too clear. in looking at these tutorials, it looks like it will help me a lot. :-)


mozkill
Novice

Apr 1, 2003, 4:23 PM

Post #7 of 9 (2521 views)
Re: [mozkill] a muti-threaded web client in perl [In reply to] Can't Post
ok, here is the finished product
------

#!perl.exe

use LWP;
use HTML::Parse;
use HTML::Element;
use HTTP::Request;
use Thread qw(async);
use strict;

printf "\n-------\n";

my @children;
for (my $i=0;$i<4;$i++)
{
$children[$i] = async {
getIt();
};
}

for (my $i=0;$i<4;$i++)
{
$children[$i]->join();
}

sub getIt()
{
my $ua = LWP::UserAgent->new();
$ua->agent("MyApp/0.1 ");
my $url = 'http://web.pdx.edu/~jonw';
my $req = HTTP::Request->new(GET => $url);
my $res = $ua->request($req);
my $code = $res->code;
my $content = $res->content;
if ($res->is_error())
{
printf "%s\n", $res->status_line;
}
else
{
my $content = $res->content();
}
printf "$content\n-------\n";
}


(This post was edited by mozkill on Apr 2, 2003, 9:46 AM)


Paul
Enthusiast

Apr 2, 2003, 11:35 AM

Post #8 of 9 (2510 views)
Re: [mozkill] a muti-threaded web client in perl [In reply to] Can't Post
Quote
printf "%s\n", $res->status_line;
...why not just:

print $res->status_line . "\n";

...or the wonderful:

print "@{[$res->status_line]}\n";

Also your for() loop could be changed to:

for (1..4) {

Wink


(This post was edited by Paul on Apr 2, 2003, 11:37 AM)


mozkill
Novice

Apr 3, 2003, 10:16 AM

Post #9 of 9 (2502 views)
Re: [Paul] a muti-threaded web client in perl [In reply to] Can't Post
alright, now I am having a new issue. i dont know how do debug the following problem. can anyone provide me with any advice on debugging forked VM's that have been spawned with async??

if I run more than 10 threads, my script starts becoming more and more unstable... it will say "script crashed , Perl.exe generating error log". can anyone give any advice or hints.?


(This post was edited by mozkill on Apr 7, 2003, 2:07 PM)

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives