Perl Guru Forums: Perl Programming Help: Beginner: Perl Script in Multi User Environment

CGI/Perl Guide | Learning Center | Forums | Advertise | Login

Site Search: in


	MAIN INDEX	SEARCH POSTS	WHO'S ONLINE	LOG IN

Home: Perl Programming Help: Beginner:
Perl Script in Multi User Environment

Raj_hpking
Novice

Jul 23, 2014, 3:53 AM

Post #1 of 9 (704 views)

Perl Script in Multi User Environment	Can't Post

Hi,

I am using Perl script to process the csv file having millions of records. Scenario is,

1) Web Service will Trigger the Perl Script (Processfile.pl) and will pass the file Name & location where file is stored.
2) Processfile.pl will read the file name & locationwill process them.

Now, As being new in Perl Script world, I want to know, This web service will be part of multi user web application. Will there be any issue if web service (in turn, it will start Perl Script Processfile.pl) is being called by many users simultaneously.

Basically, I want to use same Processfile.pl among the users who calling web service. Parameter's value passed from web service to this processfile.pl will be different for each users. I don't want to use Threading in Processfile.pl.

(This post was edited by Raj_hpking on Jul 23, 2014, 5:41 AM)

FishMonger
Veteran / Moderator

Jul 23, 2014, 6:43 AM

Post #2 of 9 (692 views)

Re: [Raj_hpking] Perl Script in Multi User Environment [In reply to]	Can't Post

You haven't provided enough info about your Processfile.pl script for us to say if it will work in a web app environment.

How does your web app execute the script?

How long does it take for Processfile.pl to complete?

Does it need to return anything to the calling web app?

Can you post the script so that we can see what it does?

Raj_hpking
Novice

Jul 23, 2014, 7:10 AM

Post #3 of 9 (689 views)

Re: [FishMonger] Perl Script in Multi User Environment [In reply to]	Can't Post

Hi,

Processfile.pl is simple perl script file, it will read each record of CSV file (its path is provided by Web Service) and apply some logic like addition or deletion on the columns. Number of columns can be 10 (dynamic).

About time it will take, is depend on perl, i.e. how much time perl will take to complete say 10 millions of record.

I will create the batch file in web service and use below to trigger perl script,

System.Diagnostics.Process p =new System.Diagnostics.Process();
p.StartInfo.FileName=mybatchfilename;
p.Start();
p.WaitForExit();
p.Close();

It dont need to return anything to the web service, but need to generate one csv file after applying addition or deletion on each column. Web service will read that csv file.

Presently, I do not have that script ready, Once I get confident that we can use Perl Script in multi user scenario then I am thinking to start working on it.

(This post was edited by Raj_hpking on Jul 23, 2014, 7:17 AM)

FishMonger
Veteran / Moderator

Jul 23, 2014, 7:47 AM

Post #4 of 9 (682 views)

Re: [Raj_hpking] Perl Script in Multi User Environment [In reply to]	Can't Post

Perl scripts can and very often are used in multi user environments such as web apps, but "the devil is in the details". The ability to work in a multi user environment is not, for the most part, based on the choice of scripting language, it's in the details of what that script is doing.

If multiple users need to modify the same file, then no matter what language you use, you could have problems.

It's not clear, but it sounds like you're saying that the number of columns in each row of the csv file can vary. If that's the case, then you have a very poorly designed csv file.

Processing a file with millions of rows can take a couple seconds or hours/days. It all depends on what kind of processing you're doing with each row.

You have a web service that calls a batch file which in turn calls the perl script? That sounds like a very odd and inefficient setup.

Raj_hpking
Novice

Jul 23, 2014, 9:29 AM

Post #5 of 9 (678 views)

Re: [FishMonger] Perl Script in Multi User Environment [In reply to]	Can't Post

Thanks FishMonger for your detail reply,

1) As you said calling batch file from web service is very odd and inefficient setup, then is there any proper method which I can use? I dont want to execute perl from my web application, its some constraints that I cant do.

2) I will not modify Processfile.pl, this file is used to process dummy.csv file.

example: c:\> d:\Processfile.pl fewcommandlinearguments (dummy.csv file will be input to processfile.pl and will be separate for each users)

Here my question is, if one user is accessing Processfile.pl then can another user access processfile.pl in above scenario.

3) Yes, number of columns in dummy.csv file is not fixed. As user is uploading this file to ftp location. My other service will download that file. And that file will be used for processing. There are multiple factors due to which number of columns are dynamic. And even that will take care in future if any new column added to csv with only minimal changes to the application.

Please suggest any good logic/architecture if any.

(This post was edited by Raj_hpking on Jul 23, 2014, 9:35 AM)

FishMonger
Veteran / Moderator

Jul 23, 2014, 9:59 AM

Post #6 of 9 (673 views)

Re: [Raj_hpking] Perl Script in Multi User Environment [In reply to]	Can't Post

Quote

1) As you said calling batch file from web service is very odd and inefficient setup, then is there any proper method which I can use? I dont want to execute perl from my web application, its some constraints that I cant do.

Having the web app execute the perl script is the proper and best approach. What type of constraints do you have that prevent you from executing the perl script from the web app but doesn't prevent you from executing the batch script?

Quote

2) I will not modify Processfile.pl, this file is used to process dummy.csv file.

I haven't suggested that you should modify the script. I would need to know more details on what the script is doing before I could/would make that suggestion.

Quote

Here my question is, if one user is accessing Processfile.pl then can another user access processfile.pl in above scenario.

Yes..maybe..depends. That question can't be answered without knowing the details of the script. Do separate executions of the script need to have exclusive write access to the same file?

(This post was edited by FishMonger on Jul 23, 2014, 10:06 AM)

FishMonger
Veteran / Moderator

Jul 23, 2014, 10:11 AM

Post #7 of 9 (670 views)

Re: [Raj_hpking] Perl Script in Multi User Environment [In reply to]	Can't Post

If the csv file being parsed/processed by the script is being used as a template in the generation of a new csv file, then you shouldn't have any problems with multiple simultaneous script executions.

If the script is creating/modifying the exact same file with each execution, then you will have a problem no matter what language you're using.

Raj_hpking
Novice

Jul 24, 2014, 12:54 AM

Post #8 of 9 (662 views)

Re: [FishMonger] Perl Script in Multi User Environment [In reply to]	Can't Post

Quote

I am not executing Batch file from web app, I am executing batch file from WCF. As beginner in Perl, I don't know what is the proper way. I searched on net and found that Perl script is also executed same way as batch file. i.e. ProcessStartInfo perlStartInfo = new ProcessStartInfo(@"d:\Perl\bin\perl.exe");.

But I really dont know advantages of executing batch or Perl script in C#.net.

(This post was edited by Raj_hpking on Jul 24, 2014, 12:55 AM)

FishMonger
Veteran / Moderator

Jul 24, 2014, 6:13 AM

Post #9 of 9 (645 views)

Re: [Raj_hpking] Perl Script in Multi User Environment [In reply to]	Can't Post

I don't know anything about WCF or C#.net so I can't answer any questions related to them. I can answer Perl related questions and the answer to your question about being able to run Perl scripts in a multi user environment is yes.

Search for (options)

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives