Perl Guru Forums: Need a Custom or Prewritten Perl Program?: I Need a Programmer for Freelance Work: Text file parsing

CGI/Perl Guide | Learning Center | Forums | Advertise | Login

Site Search: in


	MAIN INDEX	SEARCH POSTS	WHO'S ONLINE	LOG IN

Home: Need a Custom or Prewritten Perl Program?: I Need a Programmer for Freelance Work:
Text file parsing

grant mccormack
New User

Oct 25, 2012, 4:41 PM

Post #1 of 4 (27927 views)

Text file parsing	Quote \| Reply \| Private Reply

I currently have a C# script that process's txt files for me,the script parses each file for segments of text, and creates html files based on each match found and then inserts this into a db.

when we run this on c# on a large volume of files, the time taken to complete can run into several hrs and i need to try and get the speed down and am being told by every1 that if its txt parsing then perl is the way to go.

if this is something you may be interested in then please drop me a line

many thanks

Grant

grant@pv-data-solutions.co.uk

FishMonger
Veteran / Moderator

Oct 25, 2012, 7:57 PM

Post #2 of 4 (27925 views)

Re: [grant mccormack] Text file parsing [In reply to]	Quote \| Reply \| Private Reply

A compiled program written in C# or any other similar low level language will always be faster than a Perl script IF it is written correctly. If the programmer didn't code it correctly, the program, no matter which language it was written in, would be efficient/fast.

Without knowing anything about your current program and parsing details I can't say for sure if a Perl script can process your files any faster, but several hours for a C# program does sound like it may not have been coded correctly.

grant mccormack
New User

Oct 26, 2012, 3:37 AM

Post #3 of 4 (27922 views)

Re: [FishMonger] Text file parsing [In reply to]	Quote \| Reply \| Private Reply

Hi Fishmonger

Many thanks for your reply

The txt files we recieve contains cases, there could be be up to 2000 cases per txt file, so our script parses out each case, then run a another parse on the extracted case for extra values, then creates a html and finally inserts all into a DB.

If we run 1 file, it will take around 2 mins, but whilst doing so the memory and cpu will be high, which isnt a problem in itself, but the issue comes in the volume of txt files we recieve, we can get around 2000 a week to process for each client

At present we are in setup so are processing for just test purposes but when we go live, we could recieve 500 a day for each client, so in effect could have multiple scripts all running at the same time, we are running a single thread so it process's one file at a time.

Does perl use less resource on the server than other languages ?

with the possibility of so many scripts running, i need to find the best way to process multiple files at the same time for various clients without placing a burden on the server

hope this makes sense

Grant

FishMonger
Veteran / Moderator

Oct 26, 2012, 6:23 AM

Post #4 of 4 (27915 views)

Re: [grant mccormack] Text file parsing [In reply to]	Quote \| Reply \| Private Reply

Quote

Does perl use less resource on the server than other languages ?

Questions like that are too generic and vague to be able to answer with any certainty. It all depends on the details of what the program is doing and how it's doing it.

Can you post one of your files and explain how it needs to be parsed and used in the creation of the html files and what needs to be added to the database?

Search for (options)

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives