CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
Grepping a file in all the files in an array and storing them in a hash

 



iphone
User

Oct 22, 2010, 4:45 PM

Post #1 of 20 (1748 views)
Grepping a file in all the files in an array and storing them in a hash Can't Post

Currently the below code greps in files @file_1 and @file_2 based on user input of the filenames.

1.Now I have an array @files which contains lot of files.I need to open every file in @files and grep $file_name in all of these files.

2.Finally I need to output in the below way.

For example,$file_name matches a file X in @files.Output should be

Files matched by X:-
Matched files.

Please help


Code
 
print "Enter file1: ";
my $file1_name = <>;
chomp($file1_name);
open my $file1, '<', $file1_name or die "Cannot open file 1\n";
my @file_1 = <$file1>;
close $file1;

print "Enter file2: ";
my $file2_name = <>;
chomp($file2_name);
open my $file2, '<', $file2_name or die "Cannot open file 2\n";
my @file_2 = <$file2>;
close $file2;

print "enter file name";
my $file = <>;
open my $DATA, '<', $file or die "could not open '$file' $!";
while (my $line = <$DATA>){
(my $file_name) = $line =~ /
\\ #Match a backslash.
( #Capture the following match in $1.
[^\\]+? #Match anything not a backslash,
) #one or more times, non greedy.
; #Match a semicolon
/xmsg;
if ( ($file_name) and ( !$seen_file{$file_name}++ ) ) {
push @file1names, grep( (/\/\Q$file_name\E#/i && !/\.pl/), @file_1 ) ;
push @file2names, grep( /\/\Q$file_name\E#/i, @file_2 ) ;

{
local $, = "\n";

print @file1names, @file2names, "\n";

}



(This post was edited by iphone on Oct 22, 2010, 6:18 PM)


iphone
User

Oct 22, 2010, 5:33 PM

Post #2 of 20 (1741 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

Appreciate any help.Thanks a lot


winfinit
User

Oct 22, 2010, 6:06 PM

Post #3 of 20 (1739 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

can you post your complete code, some sample data and expected result, i am not exactly sure what you are trying to do here, as i see some magic variables in your code that you excluded from your explanation completely, such as
file_1 and file_2 along with file1names and file2names


iphone
User

Oct 22, 2010, 6:23 PM

Post #4 of 20 (1733 views)
Re: [winfinit] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

Hi Winfinit,

Thanks for the reply.I have modified the above code for the complete code.Basically am getting a file name($file_name) from every line in a file($file) and grepping for the file name in @file_1 and @file_2.

1.Now my problem is i have an array of files "@files" to grep,i have to open every file in @files and grep in every file instead of just 2 files (@file_1 and @file_2)
2.Also,my output has to be be like below


For example,$file_name matches a file X in @files.Output should be

Files matched by X:-
Matched files


(This post was edited by iphone on Oct 22, 2010, 6:28 PM)


BillKSmith
Veteran

Oct 23, 2010, 9:39 AM

Post #5 of 20 (1715 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

The code you posted does not do anything. Use of strict and warnings would show tell you about the missing brace and the missing declarations.

Tell me if my understanding of your requirements is correct. The array @files contains a list of file names. Each of these files contain a list of other file names. The user specifies still another file name. That file also contains a list of file names. You wish to print the intersection of user specified list with the union of all the other lists.

Implement the union by reading all the files specifed by @file into a single array (Remove duplicates as before). Implement the intersection the same as before (Of course you only need one grep statement instead of two).
Good Luck,
Bill


iphone
User

Oct 23, 2010, 11:48 AM

Post #6 of 20 (1712 views)
Re: [BillKSmith] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

Hi Bill,

Thanks alot for the reply.That's not my requirement.Let me make it simpler for you .I have added statements below what needs to be changed.

Code
#!/usr/bin/perl -w  
use strict;
use warnings;
use Cwd;

opendir(DIR, ".");
my @txt_files = grep(/\.txt$/,readdir(DIR));
print "TXT FILES\n";
print "@txt_files\n";
closedir(DIR);



print "FILE NAME: ";
my $file = <>;
chomp($file);

my $line;
open my $DATA, '<', $file or die "could not open '$file' $!";

my( %seen_file);
while (my $line = <$DATA>){
(my $file_name) = $line =~ /
\\ #Match a backslash.
( #Capture the following match in $1.
[^\\]+? #Match anything not a backslash,
) #one or more times, non greedy.
; #Match a semicolon
/xmsg;
if ( ($file_name) and ( !$seen_file{$file_name}++ ) ) {

Basically this if block needs to grep for the file $file_name in the files @txt_files
}

}
{
local $, = "\n";

OUTPUT SHOULD BE CHANGED AS FOLLOWS

/******FILES THAT MATCHED FILE1 IN @txt_files******/
LIST OF FILES MATCHED
/*******FILES THAT MATCHED FILE2 in @txt_files******/
LIST OF FILES MATCHED
IF there are not files matched for a fle in @txt_files we can ignore printing
}
close $DATA;


(This post was edited by iphone on Oct 23, 2010, 12:00 PM)


iphone
User

Oct 24, 2010, 2:24 PM

Post #7 of 20 (1683 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

Bill,is it clear?any pointers?pls help


BillKSmith
Veteran

Oct 25, 2010, 11:34 AM

Post #8 of 20 (1671 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

It now seems that you want to generalize the script in your original post to do its thing to each of the text files in the current directory rather than just file1 and file2. It appears that you DATA file is also in that directory. If it is a text file you must remove it from the list of text files.

It would be easier to get the array of text files by using glob (perldoc -f glob) than readdir and grep.

Once you have the list of filenames, you must open, read and close each one. I suggest reading the data into a hash of lists where the file name is the hash key and the hash value is a reference to a list of lines from that file.

You only need one grep statement. Put it in a loop which iterates over the keys of the hash. Store the results of grep in a hash. Again the key is the file name and the value is a reference to a list of names that were found in that file.

When done, print the second hash any way you would like.
Good Luck,
Bill


iphone
User

Oct 25, 2010, 5:04 PM

Post #9 of 20 (1661 views)
Re: [BillKSmith] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

Thanks a lot for the reply bill



It would be easier to get the array of text files by using glob (perldoc -f glob) than readdir and grep.

Code
[[iphone]]-->Please help if this is right 

$start_dir = "C\:\\Program Files";
my @plf_files = <"$start_dir/*.plf">;
print "PLF FILES\n";
print "@plf_files\n"; ---->Doesn't print anything.

Once you have the list of filenames, you must open, read and close each one. I suggest reading the data into a hash of lists where the file name is the hash key and the hash value is a reference to a list of lines from that file.

Code
[[iphone]] -- Does this look OK to you? 

foreach my $plf (@plf_files) {

open my $match, '<',$plf or die "could not open '$plf' $!"

$Hash_filenames{'filename')=$plf;
$Hash_filenames{'lines')=$match;
}


You only need one grep statement. Put it in a loop which iterates over the keys of the hash. Store the results of grep in a hash. Again the key is the file name and the value is a reference to a list of names that were found in that file.

When done, print the second hash any way you would like.

[[iphone]]Really sorry to say this.Very new to hashes.I am really confused on how to achieve this.Would you please provide some sample code?


(This post was edited by iphone on Oct 25, 2010, 5:20 PM)


BillKSmith
Veteran

Oct 26, 2010, 7:13 AM

Post #10 of 20 (1647 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

You seem to be using windows. File or directory names that contain whitespace need special treatment. Note: This is a windows, not a perl issue. It is possible to code your strings to satisfy the quote requirements of both perl and windows, but it is difficult. It is far easier to use the alias that windows provides for the troublesome name. To find the alias for 'Program Files', go to a command window and type dir c:\ /X. Find 'Program Files' and read the alias. It is probably 'progra~1'.

Otherwise, your code looked good to me, but it did not work until I used the glob function explicitly.

Refer to perldoc perldsc for lots of examples of data structures in perl.
Good Luck,
Bill

(This post was edited by BillKSmith on Oct 26, 2010, 1:05 PM)


iphone
User

Oct 26, 2010, 12:55 PM

Post #11 of 20 (1643 views)
Re: [BillKSmith] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

For some reason the below code just doesnt print anything.My $start_dir is on a server location,does that make any difference?


Code
$start_dir = \\server1\tools 
my @plf_files = glob("$start_dir/*.plf");
print "PLF FILES\n";
print "@plf_files\n";



(This post was edited by iphone on Oct 26, 2010, 12:56 PM)


BillKSmith
Veteran

Oct 26, 2010, 1:14 PM

Post #12 of 20 (1639 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

I would not expect that to matter, but who knows?

If you original code with 'readdir' works, use it. I only suggested glob because it is usually easier.
Good Luck,
Bill


iphone
User

Oct 26, 2010, 9:16 PM

Post #13 of 20 (1626 views)
Re: [BillKSmith] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

 
Thanks Bill.I have used read dir and moved ahead,however I am getting the following warning while running my code

Use of uninitialized value in pattern match (m//) at orphan_plf.pl line 37

and line 37 is

Code
$Hash_filematches{$filename}=  grep( /\/\Q$file_name\E#/i, $Hash_filenames{$plf} );


Complete code

Code
#!/usr/bin/perl -w  
use strict;
use warnings;
my %Hash_filenames=();
my %Hash_filematches=();
my $plf;
my $start_dir;
my $start_dir = \\server1\tools ;

my $file_name = "load.c";

opendir(DIR, "$start_dir");
my @plf_files = grep(/\.plf$/,readdir(DIR));
print "PLF FILES\n";
print "@plf_files\n";
foreach my $plf (@plf_files) {
chomp($plf);
open my $match, '<',"$start_dir\\$plf" or die "could not open '$plf' $!";
my @file_lines = <$match>;
#Save the filename as key and the lines in the file as value in a hash
$Hash_filenames{$plf}=@file_lines;
}
closedir(DIR);
if ($file_name) {

foreach my$filename(keys %Hash_filenames) {

#for every key(filename) in the above hash namely #$Hash_filenames
#grep for $file_name in the
#corresponding values (lines in the file)of the file
#and save the match as value and key as filename
#in another hash
$Hash_filematches{$filename}= grep( /\/\Q$file_name\E#/i, $Hash_filenames{$plf} );
}
}
{
local $, = "\n";
print "PRINTING MATCHED HASHES";
foreach my $key ( keys %Hash_filenames ) {
my $value = $Hash_filematches{$key};
print "$key => $value\n";
}
}



(This post was edited by iphone on Oct 26, 2010, 9:33 PM)


BillKSmith
Veteran

Oct 27, 2010, 6:29 AM

Post #14 of 20 (1611 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

A hash value must be a scalar. Assigning an array to a hash value saves the size of the array, not the array. The solution is to assign a reference to an array (Refer to perldoc perldsc for many examples).

Perl provides several ways to dereference references. The choice among them is largely a matter of preference. I usually use the syntax that I find easiest to write correctly. I admit that the alternates are almost always easier to read. My preference requires you to understand only one rule: "Any symbol can be replaced with a reference enclosed in curly braces." (e.g. anywhere that you would normally use $x, you can use ${$x_ref}). The rule is recursive. It can require many braces, some of which may not be necessary, but always allowed.

In you original code, you used an array @file_1 as an argument to grep. Replace the symbol (file_1) with a reference ($Hash_filenames{$plf}) and add braces to form:

@{$Hash_filenames{$plf}}



The comments apply to both of your hashes.
Good Luck,
Bill


iphone
User

Oct 27, 2010, 5:26 PM

Post #15 of 20 (1601 views)
Re: [BillKSmith] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

Thanks a lot Bill.You guidance is really helpful.


Code
changed 

$Hash_filenames{$plf}=@file_lines;

to

$Hash_filenames{$plf}=\@file_lines;



Code
 
changed
$Hash_filematches{$filename}= grep( /\/\Q$file_name\E#/i, $Hash_filenames{$plf} );

to

@{ $Hash_filematches{ $filename } } = grep( /\/\Q$file_name\E#/i, @{ $Hash_filenames{ $plf } });


I am getting "Use of uninitialized value in hash element at orphan_plf.pl line 40" and line 40 is
@{ $Hash_filematches{ $filename } } = grep( /\/\Q$file_name\E#/i, @{ $Hash_filenames{ $plf } });

and the output is
PRINTING MATCHED HASHES
apps.plf => ARRAY(0x1981c80)
modem.plf => ARRAY(0x1981ca4)


(This post was edited by iphone on Oct 27, 2010, 5:27 PM)


BillKSmith
Veteran

Oct 27, 2010, 6:02 PM

Post #16 of 20 (1595 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post


Quote
$Hash_filenames{$plf}=\@file_lines;



All the hash values are the same. They all refer to the same array. You want each one to refer to an different anonymous array. Note how this is done in the examples in the reference.



Line 40 appears to be fine.




Quote
apps.plf => ARRAY(0x1981c80)



You are printing a reference to an array. You must dereference the references in your print statements.





Good Luck,
Bill


iphone
User

Oct 28, 2010, 6:34 PM

Post #17 of 20 (1584 views)
Re: [BillKSmith] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post


Quote

All the hash values are the same. They all refer to the same array. You want each one to refer to an different anonymous array. Note how this is done in the examples in the reference.


1.why do they refer to same array?I have a loop in place for every plf(see below code)


Code
foreach my $plf (@plf_files) { 
chomp($plf);
open my $match, '<',"$start_dir\\$plf" or die "could not open '$plf' $!";
my @file_lines = <$match>;
$Hash_filenames{$plf}=\@file_lines;
}


2.I have question on the below line

Code
@{ $Hash_filematches{ $filename } } = grep( /\/\Q$file_name\E#/i, @{ $Hash_filenames{ $plf } });

I am grepping on @{ $Hash_filenames{ $plf } } which I believe gives size of array,how will this work grepping on size of array?


(This post was edited by iphone on Oct 28, 2010, 6:40 PM)


BillKSmith
Veteran

Oct 28, 2010, 9:15 PM

Post #18 of 20 (1572 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

1. There is only one array (@file_lines). All the hash values are references (\@file_lines) to that array. You are changing the contents of that array each time though the loop. When the loop ends, every key is associated with the data for the last line. You must create an anonymous array for each line and store a reference to that array in the hash.



2. That expression acts like a simple array. In list context, it returns the content. In scalar context, it returns the size. The second argument of grep is in list context.
Good Luck,
Bill


iphone
User

Oct 28, 2010, 11:12 PM

Post #19 of 20 (1564 views)
Re: [BillKSmith] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

In the below code snippet @file_1 has all the lines in file,I have replicated the same code ,how come @file_lines has only the last line.Please help me understand

Code
print "Enter file1: ";    
my $file1_name = <>;
chomp($file1_name);
open my $file1, '<', $file1_name or die "Cannot open file 1\n";
my @file_1 = <$file1>;
close $file1;



BillKSmith
Veteran

Oct 29, 2010, 6:39 AM

Post #20 of 20 (1555 views)
Re: [iphone] Grepping a file in all the files in an array and storing them in a hash [In reply to] Can't Post

Sorry, I did get confused in my detail explanation of item 1 above. I was trying to explain the erroneous behavior of the following code.


Quote

Code
foreach my $plf (@plf_files) {  
chomp($plf);
open my $match, '<',"$start_dir\\$plf" or die "could not open '$plf' $!";
my @file_lines = <$match>;
$Hash_filenames{$plf}=\@file_lines;
}


There is only one array (@file_lines). It is used by every plf_file.Every entry in the hash %Hash_filenames is assigned the same value, a reference to that array. At the end of the plf loop, every entry will refer to the data from the last file.You need an array for each file. You do not need a named array for each file. That was the crux of your problem at the start of this thread. For each file, you want an anonymous array that you can only access through a reference in the hash.
Good Luck,
Bill

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives