Perl Guru Forums: Perl Programming Help: Beginner: script to create multiple files off multiple grep strings

CGI/Perl Guide | Learning Center | Forums | Advertise | Login

Site Search: in


	MAIN INDEX	SEARCH POSTS	WHO'S ONLINE	LOG IN

Home: Perl Programming Help: Beginner:
script to create multiple files off multiple grep strings

LDHB2012
New User

Dec 17, 2012, 12:17 PM

Post #1 of 4 (2923 views)

script to create multiple files off multiple grep strings	Can't Post

Hi everybody...

I'm completely new to the forums and extremely new to scripting. I've been through a few tutorials to get started on learning the perl language. I'm decent with linux CLI and I'm looking for ways to make life easier with some of the things I'll be doing.

I wanted to create a script that allows for multiple grep 'string' searches and then outputs the results to a file named 'grep'd string'.

Here's the scenario:
-One directory with multiple flat files which contain many rows of events which are delimited by |
-I want separate files created for each string I'm interested in and each file can be named after the string I specify through grep
-So I'll create a text file, I suppose, to contain all the strings I want to grep, where the script would point to that file as a reference for each grep command
-after every grep, a file should be created
-I want to run this weekly, as new files are added to the directory
-Upon running it weekly, I want the new info to be appended to it's respective file, but without creating duplicates from grep'ing the older files.

I think that pretty much sums it up. Please let me know if I can explain this better, or simply, more info is needed.

-David

Laurent_R
Veteran / Moderator

Dec 17, 2012, 2:05 PM

Post #2 of 4 (2919 views)

Re: [LDHB2012] script to create multiple files off multiple grep strings [In reply to]	Can't Post

Hi David,

In Reply To

-Upon running it weekly, I want the new info to be appended to it's respective file, but without creating duplicates from grep'ing the older files.

That might be the only slightly complicated thing, as you have to figure out how you will know which files to grep and which files not to grep. Is the file last modification date a sufficient indication for you? If so, it is rather simple (the -M operator gives you the elapsed time since a file last modification). If not, you might have to track the information, making the thing somewhat more complicated.

The rest of the requirement is basically straight forward.

Have you already started to write some code?

LDHB2012
New User

Dec 20, 2012, 7:02 AM

Post #3 of 4 (2884 views)

Re: [Laurent_R] script to create multiple files off multiple grep strings [In reply to]	Can't Post

Laurent:

Thanks for getting back so quick.

I have not yet started writing any code for this. My first goal was to find out if it could be done and if perl code could provide the most adequate solution. I can certainly grep the latest modded files, or maybe I can specify by date(each file is named in date format, based on it's contents)?

I'm more so looking for guidance on a starting point and what commands would allow this functionality. I'm pretty logical, and research well, but the foundation of perl, I lack very much.

-David

Laurent_R
Veteran / Moderator

Dec 20, 2012, 10:19 AM

Post #4 of 4 (2878 views)

Re: [LDHB2012] script to create multiple files off multiple grep strings [In reply to]	Can't Post

Yes, it can be done and Perl is a good candidate language to do it.

To find all the files in the directory and keep only those that are less than 7 days old, you could do something like this:

Code

my @file_list = glob "/data/2012_data/*.*"; 
@file_list = grep { -M $_ < 7} @file_list;

to read your config file:

Code

my $config = "config.txt"; 
open $CONF, "<", $config or die "cannot open $config $! \n"; 
my ($string, $file_name); 
while (my $line = <$CONF>) { 
     chomp $line; 
     ($string, $file_name) = split /;/, $line; # assumes string to be grepped and filename are separated by a semi-colon 
     # etc. 
}

I hope thse code snippets will help you starting, but you'll need to do some work of your own.

You will probably need to nested loops: one to loop opn the strings to be grepped, and another to loop through your data, and this last loop mlight itseld be two nested loops: one to loop through the lmist of files and one to loop though the lines of each file. You'll neeed careful polanning.

Search for (options)

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives