CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
A file parsing and 2D array/matrix problem.

 

First page Previous page 1 2 3 Next page Last page  View All


Chris Charley
User

Sep 10, 2012, 9:44 AM

Post #51 of 62 (2928 views)
Re: [rushadrena] A file parsing and 2D array/matrix problem. [In reply to] Can't Post


Quote
../xla./mgp./oaa./acs./spu./mdo./ssc./dre./xtr./gga

Your @file array has 11 entries beginning with '.'. So, you either need to remove that entry or access your files like below.

Code
print "Processing $file[$i+1] and $file[$j+1]\n";

I think thats your problem.


Laurent_R
Veteran / Moderator

Sep 10, 2012, 10:02 AM

Post #52 of 62 (2926 views)
Re: [rushadrena] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Quite obviously, there some values of $i and/or $j that you are using for which @file is not defined.

Check how you filled the @file array (is it from 0 to 9, or perhaps from 1 to 10)?

I don't want to check in the code you posted, because you posted it quite a while ago and I assume it has changed quite a bit since.

Edit: I had not seen the answer by Chris when I typed the above. Maybe it is 0 to 10 or 1 to 11, after all. You should probably filter out the extra entry as soon as you read your directory.


(This post was edited by Laurent_R on Sep 10, 2012, 10:18 AM)


rushadrena
Novice

Sep 10, 2012, 2:42 PM

Post #53 of 62 (2910 views)
Re: [Laurent_R] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Hi Laurent and Chris I didnt see that coming. I have fixed that error by introducing this

Code
splice(@dirs, 0, 1);

Now I have this huge file matrix.txt(I have snipped the file so as to contain only 2x5 matrices for a concise view) which I have to further process : Simultaneously one by one splitting and processing

Code
Processing  
10011
01010
Processing
11110
01010
Processing
11100
00110
Processing
10010
11001
Processing
11110
00001
Processing
11010
00110
Processing
10101
01010
Processing
00111
11100
Processing
11010
11111


What I want is to go through this list in a selection of three.

1. After the first occurrence pattern "Processing" i have 2 lines.Save these 2 lines in a separate file.save the file size of this file in variable "a"

2. After 2nd occurrence of "Processing", I have 2 lines.Save these 2 lines in a separate file.save the file size of this file in variable "b"

3. After 3rd occurrence of "Processing", I have 2 lines.Save these 2 lines in a separate file.save the file size of this file in variable "c"
Now I'd perform an operation $a+$b/$c >> RESULT.txt.

Then select other triplet i.e. 4th,5th and 6th occurence of "Processing" save the calculation to RESULT.txt and so on.

[EDIT:] I did tried these two codes, but they dont cater to the triplet selection

Code
 perl -ne 'BEGIN{ $/="Processing"; } if(/^\s*(\S+)/){ open(F,">$1.out")||warn"$1 write failed:$!\n";chomp;print F "gi", $_ }'



Code
awk -vRS="Processing" '{ print $0 > "file"t++".out" }' matrix.txt



(This post was edited by rushadrena on Sep 10, 2012, 2:58 PM)


rushadrena
Novice

Sep 11, 2012, 6:10 AM

Post #54 of 62 (2855 views)
Re: [Chris Charley] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Hi all,

Im still getting this irritating error after 10 minutes of running the code. And it keeps popping up continuously. :-

Code
$ perl code2.pl >> 10x10 
Use of uninitialized value within @file in concatenation (.) or string at code2.pl line 61.
Use of uninitialized value in concatenation (.) or string at code2.pl line 61.
Use of uninitialized value within @file in concatenation (.) or string at code2.pl line 61.
Use of uninitialized value in concatenation (.) or string at code2.pl line 61.


The relevant debug code

Code
  DB<1> print @dirs 
./acs./dre./gga./mdo./mgp./oaa./spu./ssc./xla./xtr
DB<2> print $#dirs
9

BTW, here's the code

Code
#!/usr/bin/perl 
use strict;
use warnings;
use List::Util qw/ max /;

open my $fh, "<", 'SUPERLIST_PRODUCT' or die $!;
my $spr_prod = do {local $/; <$fh>};
close $fh or die $!;
my @spr_prod = $spr_prod =~ /\d+/g;

open $fh, "<", 'SUPERLIST_SUBSTRATE' or die $!;
my $spr_substr = do {local $/; <$fh>};
close $fh or die $!;
my @spr_substrate = $spr_substr =~ /\d+/g;
my $xx=0;
my $zz=0;
my @matrix;
my @file;
my @dirs = split /\n/,`find -maxdepth 1 -type d`;
splice(@dirs, 0, 1);
for ($xx=0;$xx<=9;$xx++)
{
for ($zz=0;$zz<=9;$zz++)

{
my $path = '.'; # (current directory - '.') or path to data files
my @file = ("$dirs[$xx]/irrev_rev_revdup", "$dirs[$zz]/irrev_rev_revdup" );
for my $file ( @file ) {
my %data;
my @substrate;

print "#########################$file###################################";
open my $fh, "<", "$file"
or die "Unable to open $file for reading. $!";

while (<$fh>) {
if (/^substrate/) {
@substrate = /\d+/g;
}
elsif (/^product/) {
while (/(\d+)/g) {
for my $sub (@substrate) {
$data{$sub}{$1} = 1 ;
}
}
}
else {
die "Unknown format $file. $!";
}
}
close $fh or die "Unable to close $file. $!";

print "Processing file: $file\n";
process(\@spr_prod, \@spr_substrate, %data);

push @matrix, \%data;
}
}}
for my $i (0 .. $#matrix) {
for my $j ($i+1 .. $#matrix) {
print "Processing $file[$i] and $file[$j]\n";
my %data = combine($matrix[$i], $matrix[$j]);
process(\@spr_prod, \@spr_substrate, %data);
}
}

sub process {
my ($spr_prod, $spr_subst, %data) = @_;
my %seen;

my @product = sort {$a <=> $b}
grep ! $seen{$_}++,
@$spr_prod, map keys %$_, values %data;

# to get column width for print
my $wid = 1 + max map length, @product;

printf "%7s" . "%${wid}s" x @product . "\n", 'prod->', @product;

undef %seen;
my @substrate = sort {$a <=> $b}
grep ! $seen{$_}++,
@$spr_subst, keys %data;

for my $substrate (@substrate) {
printf "%7s", $substrate;
printf "%${wid}s", $data{$substrate}{$_} || '-' for @product;
print "\n";
}
printf "\n%5s\n%5s\n%s\n\n", '^', '|', 'substrate';
}


sub combine {
my ($matrix1, $matrix2) = @_;
my %new_hash = %$matrix1;

for my $substrate (keys %$matrix2) {
$new_hash{$substrate}{$_} = 1 for keys %{ $matrix2->{$substrate} };
}
return %new_hash;
}



rushadrena
Novice

Sep 11, 2012, 8:31 AM

Post #55 of 62 (2846 views)
Re: [Laurent_R] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Laurent there is another problem with the code. I tried to test the code on a smaller instance of directories and file size. For example I created 10 directories a,b,c,d,e,f,g,h,i,j inside a directory named test . Each of these directories has a file named "irrev_rev_revdup".
The contents of each of "irrev_rev_revdup" is same (for checking purpose) --

substrate[s]: 1 9
product[s]: 10
substrate[s]: 2 7
product[s]: 8 3
substrate[s]: 3
product[s]: 4
substrate[s]: 2
product[s]: 5
substrate[s]: 6
product[s]: 7
substrate[s]: 9
product[s]: 3 8
substrate[s]: 3 5
product[s]: 3 9
substrate[s]: 10
product[s]: 9 1
substrate[s]: 7
product[s]: 2 6
substrate[s]: 3
product[s]: 8 7

Now inside the test directory I created two files SUPERLIST_PRODUCT and SUPERLIST_SUBSTRATE, the contents of both these files ----

1
2
3
4
5
6
7
8
9
10

So the test directory has these files and directories:-
a b c d e f g h i j SUPERLIST_PRODUCT SUPERLIST_SUBSTRATE code2.pl

Now after running the code2.pl, logically there should be 100 concatenated (combined) matrices because there are 10 directories.But Im getting 19900 combined matrices.

Can you point out what could be the flaw ?


Chris Charley
User

Sep 11, 2012, 8:49 AM

Post #56 of 62 (2845 views)
Re: [rushadrena] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Do you want a matrix of each individual file with the combined pairs of matrices OR just the combined pairs (without each individual file matrix).


rushadrena
Novice

Sep 11, 2012, 9:08 AM

Post #57 of 62 (2842 views)
Re: [Chris Charley] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Chris yes I want a matrix of each individual file with the combined pairs of matrices .
Suppose i have 5 files then i would like to have a matrix each for these files( a total of 5 that is), and also for the combined pairs (thus 5x5 =a total of 25 that is).


Laurent_R
Veteran / Moderator

Sep 11, 2012, 10:27 AM

Post #58 of 62 (2835 views)
Re: [rushadrena] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Hi,

I think your problem is probably there:


Code
for my $i (0 .. $#matrix) {  
for my $j ($i+1 .. $#matrix) {


If I under(stand what you do, it should proably be :


Code
for my $i (0 .. $#dirs) {  
for my $j ($i+1 .. $#dirs) {



rushadrena
Novice

Sep 11, 2012, 1:14 PM

Post #59 of 62 (2824 views)
Re: [Laurent_R] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Laurent,
I implemented the corrections suggested, and now there are 45 combined matrices generated ((9+8+7+....1).
But the combined matrices are not getting correctly represented,

Code
#########################./a/irrev_rev_revdup#######################Processing file: ./a/irrev_rev_revdup 
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - - - - - 1 -
6 - - - - - - 1 - - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

^
|
substrate

#########################./d/irrev_rev_revdup######################Processing file: ./d/irrev_rev_revdup
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - 1 - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 1 - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - 1 - - - 1 -
6 - - - 1 1 - 1 1 - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -


Now the combined matrix of both is here and its incorrect(It should be displaying "Processing ./a and ./b" rather than "Processing ./a and ./d")

Code
 Processing ./a and ./d 
prod-> 1 2 3 4 5 6 7 8 9 10
1 1 - 1 - - - - 1 - 1
2 1 1 1 - 1 - 1 1 - -
3 1 1 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 1 1 1 1 - - - - 1 -
6 - 1 - - - - 1 - - -
7 - 1 1 - - 1 1 1 - -
8 - - - - - - - - - -
9 1 - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

^
|
substrate


The above combined matrix should be for these two ("a" and "b" rather than "a" & "d"):-

Code
#########################./a/irrev_rev_revdup######################Processing file: ./a/irrev_rev_revdup 
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - - - - - 1 -
6 - - - - - - 1 - - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

^
|
substrate

#########################./b/irrev_rev_revdup#####################Processing file: ./b/irrev_rev_revdup
prod-> 1 2 3 4 5 6 7 8 9 10
1 1 - 1 - - - - 1 - -
2 1 1 - - 1 - 1 - - -
3 1 1 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 1 1 1 1 - - - - 1 -
6 - 1 - - - - 1 - - -
7 - 1 - - - 1 1 - - -
8 - - - - - - - - - -
9 1 - 1 - - - - 1 - -
10 1 - - - - - - - 1 -

Similarly none of the 45 matrices are getting displayed with proper name combinations.
Relevant code for displaying combination names:-

Code
for my $i (0 .. $#dirs) { 
for my $j ($i+1 .. $#dirs) {
print "Processing $dirs[$i] and $dirs[$j]\n";
my %data = combine($matrix[$i], $matrix[$j]);
process(\@spr_prod, \@spr_substrate, %data);
}
}


EDIT***** :- It would be really helpful if each of the concatenated matrices are printed after the two matrices from which it has been combined.


(This post was edited by rushadrena on Sep 11, 2012, 10:13 PM)


rushadrena
Novice

Sep 12, 2012, 12:16 AM

Post #60 of 62 (2787 views)
Re: [Chris Charley] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Hi Chris,
I modified your code to cater to my requirements (see parts in itallics).:-

Code
#!/usr/bin/perl 
use strict;
use warnings;
use List::Util qw/ max /;

open my $fh, "<", 'SUPERLIST_PRODUCT' or die $!;
my $spr_prod = do {local $/; <$fh>};
close $fh or die $!;
my @spr_prod = $spr_prod =~ /\d+/g;

open $fh, "<", 'SUPERLIST_SUBSTRATE' or die $!;
my $spr_substr = do {local $/; <$fh>};
close $fh or die $!;
my @spr_substrate = $spr_substr =~ /\d+/g;
my $xx=0;
my $zz=0;
my $i=0;
my $j=0;

my @matrix;
my @file;
my @dirs = split /\n/,`find -maxdepth 1 -type d`;
splice(@dirs, 0, 1);
for ($xx=0;$xx<=9;$xx++)
{
for ($zz=0;$zz<=9;$zz++)

{
my $path = '.'; # (current directory - '.') or path to data files
my @file = ("$dirs[$xx]/irrev_rev_revdup", "$dirs[$zz]/irrev_rev_revdup" );
for my $file ( @file ) {
my %data;
my @substrate;

print "#########################$file###################################";
open my $fh, "<", "$file"
or die "Unable to open $file for reading. $!";

while (<$fh>) {
if (/^substrate/) {
@substrate = /\d+/g;
}
elsif (/^product/) {
while (/(\d+)/g) {
for my $sub (@substrate) {
$data{$sub}{$1} = 1 ;
}
}
}
else {
die "Unknown format $file. $!";
}
}
close $fh or die "Unable to close $file. $!";

print "Processing file: $file\n";
process(\@spr_prod, \@spr_substrate, %data);

push @matrix, \%data;

} my %dat = combine($matrix[$i], $matrix[$j]);
print "Combined matrix $file[$i] and $file[$j]"
process(\@spr_prod, \@spr_substrate, %dat);

}}


sub process {
my ($spr_prod, $spr_subst, %data) = @_;
my %seen;

my @product = sort {$a <=> $b}
grep ! $seen{$_}++,
@$spr_prod, map keys %$_, values %data;

# to get column width for print
my $wid = 1 + max map length, @product;

printf "%7s" . "%${wid}s" x @product . "\n", 'prod->', @product;

undef %seen;
my @substrate = sort {$a <=> $b}
grep ! $seen{$_}++,
@$spr_subst, keys %data;

for my $substrate (@substrate) {
printf "%7s", $substrate;
printf "%${wid}s", $data{$substrate}{$_} || '-' for @product;
print "\n";
}
printf "\n%5s\n%5s\n%s\n\n", '^', '|', 'substrate';
}


sub combine {
my ($matrix1, $matrix2) = @_;
my %new_hash = %$matrix1;

for my $substrate (keys %$matrix2) {
$new_hash{$substrate}{$_} = 1 for keys %{ $matrix2->{$substrate} };
}
return %new_hash;
}

And here's a snippet from the output :-

Code
#########################./a/irrev_rev_revdup######################Processing file: ./a/irrev_rev_revdup 
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - - - - - 1 -
6 - - - - - - 1 - - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

^
|
substrate

#########################./b/irrev_rev_revdup#####################Processing file: ./b/irrev_rev_revdup
prod-> 1 2 3 4 5 6 7 8 9 10
1 1 - 1 - - - - 1 - -
2 1 1 - - 1 - 1 - - -
3 1 1 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 1 1 1 1 - - - - 1 -
6 - 1 - - - - 1 - - -
7 - 1 - - - 1 1 - - -
8 - - - - - - - - - -
9 1 - 1 - - - - 1 - -
10 1 - - - - - - - 1 -

^
|
substrate
Combined matrix ./a/irrev_rev_revdup and ./a/irrev_rev_revdup
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - - - - - 1 -
6 - - - - - - 1 - - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

^
|
substrate

#########################./a/irrev_rev_revdup####################Processing file: ./a/irrev_rev_revdup
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - - - - - 1 -
6 - - - - - - 1 - - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

^
|
substrate

#########################./c/irrev_rev_revdup###################Processing file: ./c/irrev_rev_revdup
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 1 - 1 1 1 - 1 1 1 -
4 - - - - - - - - - -
5 1 - 1 - 1 - 1 1 1 -
6 - - - - - - 1 - - -
7 1 1 1 - 1 1 1 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 1 - 1 1 - - - 1 -

^
|
substrate
Combined matrix ./a/irrev_rev_revdup and ./a/irrev_rev_revdup
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - - - - - 1 -
6 - - - - - - 1 - - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

As you can see the concatenated matrix isn't correct in any of the cases.
I hope you are getting me, the output should have been like this:-
Processing file: ./a/irrev_rev_revdup
Processing file: ./b/irrev_rev_revdup
Combined matrix ./a/irrev_rev_revdup and ./b/irrev_rev_revdup (Along with correct concatenation)


Laurent_R
Veteran / Moderator

Sep 12, 2012, 4:33 AM

Post #61 of 62 (2777 views)
Re: [rushadrena] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Granted:


Code
 Processing ./a and ./d   
pro -> 1 2 3 4 5 6 7 8 9 10
1 1 - 1 - - - - 1 - 1
2 1 1 1 - 1 - 1 1 - -
3 1 1 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 1 1 1 1 - - - - 1 -
6 - 1 - - - - 1 - - -
7 - 1 1 - - 1 1 1 - -
8 - - - - - - - - - -
9 1 - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -


is not a correct combination the matrices a and b you have shown. But is it a proper combination of a and d?


Laurent_R
Veteran / Moderator

Sep 12, 2012, 1:46 PM

Post #62 of 62 (2769 views)
Re: [rushadrena] A file parsing and 2D array/matrix problem. [In reply to] Can't Post

Hi Rushadrena,

I am taking data from your post #59. I took a bit more time to look at your data.

I looked at your matrices A and A combined with (B or D).



Code
#########################./a/irrev_rev_revdup#######################Processing file: ./a/irrev_rev_revdup  
prod-> 1 2 3 4 5 6 7 8 9 10
1 - - - - - - - - - 1
2 - - 1 - 1 - - 1 - -
3 - - 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 - - 1 - - - - - 1 -
6 - - - - - - 1 - - -
7 - 1 1 - - 1 - 1 - -
8 - - - - - - - - - -
9 - - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -


Processing ./a and ./d
prod-> 1 2 3 4 5 6 7 8 9 10
1 1 - 1 - - - - 1 - 1
2 1 1 1 - 1 - 1 1 - -
3 1 1 1 1 - - 1 1 1 -
4 - - - - - - - - - -
5 1 1 1 1 - - - - 1 -
6 - 1 - - - - 1 - - -
7 - 1 1 - - 1 1 1 - -
8 - - - - - - - - - -
9 1 - 1 - - - - 1 - 1
10 1 - - - - - - - 1 -

^
|
substrate


The result looks consistent: any place where you have a 1 in matrix A, you also have a 1 in the matrix supposed to be a combination of A and (B or D).

In other words, your combination actually looks like a correct combination of Matrix A with some other matrix (possibly B or D).

If the caption says A and D, why do you claim it is in fact A and B and that the result is false? Maybe it is actually A and D, and maybe it is after all correct. What makes you think that this combination is actually A and B?

Since you haven't given us Matrix D, I can't go much further into the analysis, but I do not see at this point anything that can lead to the conclusion that the combination matrix is wrong.

Please provide matrix D of your test, so that I can figure out myself whether this is right or not. For the time being, all I can say is that the combinbation matrix definitely looks like a combination of matrix A with another matrix.

First page Previous page 1 2 3 Next page Last page  View All
 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives