CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
Compare unchanged files in two arrays

 

First page Previous page 1 2 Next page Last page  View All


StarkRavingCalm
User

Feb 14, 2013, 12:30 PM

Post #26 of 47 (3012 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

Ok, I needed to switch from Net:SFTP to Net:SFTP:Foreign to trap errors.
But now the intial 'ls' I am putting into a hash complains:
Not a HASH reference at ./test-perl-sftp-get.pl line 103.
( next if $entry->{filename} =~ /^\./;)

I am guessing the attributes are different from SFTP to SFTP:Foreign

What would the SFTP:Foreign equivalent of this be:?


Code
my %file; 
foreach my $entry ( $sftp->ls('/home/ftptest/inbound') )
{
next if $entry->{filename} =~ /^\./;
my $size = (split(' ', $entry->{longname}))[4];
$file{$entry->{filename}} = $size;
}

print Dumper \%file;

Code
 

Thanks in advance!


(This post was edited by StarkRavingCalm on Feb 14, 2013, 1:11 PM)


StarkRavingCalm
User

Feb 18, 2013, 1:29 PM

Post #27 of 47 (2997 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

 



using SFTP-Foreign, I can get a list this way:


Code
$files = $sftp->ls('/home/ftptest/inbound'); 

Code
 
Shouldn't I just be able to change my code to this:


Code
my %file->$sftp->ls('/home/ftptest/inbound'); 

foreach my $entry ( %file )
{
next if $entry->{filename} =~ /^\./;
my $size = (split(' ', $entry->{longname}))[4];
$file{$entry->{filename}} = $size;
}

print Dumper \%file;

Code
 
But doing that throws this error:

Can't call method "Net::SFTP::Foreign=HASH(0x8ff5310)" without a package or object reference at line 88.


FishMonger
Veteran / Moderator

Feb 19, 2013, 1:20 PM

Post #28 of 47 (2982 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post


Quote
Shouldn't I just be able to change my code to this:

Code
my %file->$sftp->ls('/home/ftptest/inbound');  

foreach my $entry ( %file )



No, you can't do it that way.

$sftp->ls(...) returns a reference to an array of hash refs.

You could de reference that array ref like this:

Code
my @hash_refs = @{ $sftp->ls('/home/ftptest/inbound') };


You could also de reference the hash refs in a similar manor as you loop over the array, but I think it's better/cleaner to stay with the reference and use the arrow notation when de referencing the hash.


StarkRavingCalm
User

Feb 19, 2013, 1:33 PM

Post #29 of 47 (2980 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

Ok, seem to be closer... Thank You!!!


Code
my @hash_refs = @{ $sftp->ls('/home/ftptest/inbound') }; 

foreach my $entry ( @hash_refs )
{
next if $entry->{filename} =~ /^\./;
my $size = (split(' ', $entry->{longname}))[4];
$file{$entry->{filename}} = $size;
}

print Dumper \@hash_refs;

I think I need to mess with the fields it's picking up:

$VAR1 = [
{
'longname' => '-rw-rw-r-- 1 ftptest ftptest 171 Feb 18 11:23 newfile',
'a' => bless( {
'uid' => 541,
'flags' => 15,
'mtime' => 1361204601,
'atime' => 1361199077,
'perm' => 33204,
'gid' => 541,
'size' => 171
}, 'Net::SFTP::Foreign::Attributes' ),
'filename' => 'newfile'
},
{
'longname' => '-rw-rw-r-- 1 ftptest ftptest 171 Feb 18 11:23 newfile3',
'a' => bless( {
'uid' => 541,
'flags' => 15,
'mtime' => 1361204604,
'atime' => 1361204604,
'perm' => 33204,
'gid' => 541,
'size' => 171
}, 'Net::SFTP::Foreign::Attributes' ),
'filename' => 'newfile3'
},
{
'longname' => 'drwxrwxr-x 2 ftptest ftptest 4096 Feb 18 11:23 .',
'a' => bless( {
'uid' => 541,
'flags' => 15,
'mtime' => 1361204604,
'atime' => 1361309822,
'perm' => 16893,
'gid' => 541,
'size' => 4096
}, 'Net::SFTP::Foreign::Attributes' ),
'filename' => '.'
},
{
'longname' => 'drwx------ 4 ftptest ftptest 4096 Jan 22 15:46 ..',
'a' => bless( {
'uid' => 541,
'flags' => 15,
'mtime' => 1358887570,
'atime' => 1361199071,
'perm' => 16832,
'gid' => 541,
'size' => 4096
}, 'Net::SFTP::Foreign::Attributes' ),
'filename' => '..'
},
{
'longname' => '-rw-rw-r-- 1 ftptest ftptest 171 Feb 18 11:23 newfile2',
'a' => bless( {
'uid' => 541,
'flags' => 15,
'mtime' => 1361204603,
'atime' => 1361204603,
'perm' => 33204,
'gid' => 541,
'size' => 171
}, 'Net::SFTP::Foreign::Attributes' ),
'filename' => 'newfile2'
}
];


(This post was edited by StarkRavingCalm on Feb 19, 2013, 1:39 PM)


FishMonger
Veteran / Moderator

Feb 19, 2013, 1:38 PM

Post #30 of 47 (2978 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

Dump out $entry or @hash_refs to verify that they hold what you expect.


StarkRavingCalm
User

Feb 19, 2013, 1:39 PM

Post #31 of 47 (2975 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

Just edited my post, must have caught that while you were replying....

It seems like its ignoring everything in the for each loop and adding all these additional attributes (atime, mtime, etc...)


(This post was edited by StarkRavingCalm on Feb 19, 2013, 1:47 PM)


StarkRavingCalm
User

Feb 21, 2013, 7:56 AM

Post #32 of 47 (2929 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

Ok, so I found a way to get this to work:

Code
my $file = $sftp->ls('/home/ftptest/inbound', no_wanted => qr/^\./ ); 
my %file = map { $_->{filename} => $_->{a}->size } @$file;


But now the logic does not seem to work. Will the code below need to change?

Code
 for ( keys %file ) 
{
if ( $file{$_} eq $file2{$_} )
{
push(@equal, $_);
next;
}
else
{
push(@notEqual, $_);
next;
}

}


FishMonger
Veteran / Moderator

Feb 21, 2013, 8:04 AM

Post #33 of 47 (2928 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

Have you tried dumping both hashes to see if they have the same structure?

There is no need for either of those next statements, so I'd remove them.


StarkRavingCalm
User

Feb 21, 2013, 8:08 AM

Post #34 of 47 (2926 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

Yup both give expected results:
$VAR1 = {
'file2' => 7,
'file1' => 4,
'file3' => 10,
'file4' => 13
};


$VAR1 = {
'thisisanewfile' => 0,
'file1' => 47,
'file3' => 10,
'file4' => 13,
'upfile1' => 0,
'upfile2' => 0,
'upfile3' => 0,
'upfile4' => 312
};


FishMonger
Veteran / Moderator

Feb 21, 2013, 8:25 AM

Post #35 of 47 (2925 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

Ok, you have 2 matches, file3 and file4.

When you say "now the logic does not seem to work", which is a very poor problem description, are you saying that those filenames weren't added to the @equal array?

One issue I see is that you're doing a string comparison, but the values being compared are numbers. Change the test to:

Code
if ( $file{$_} == $file2{$_} )



StarkRavingCalm
User

Feb 21, 2013, 8:39 AM

Post #36 of 47 (2921 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

Sorry for the poor description.... Blush

Use of uninitialized value in string eq at
Which is:
if ( $file{$_} eq $file2{$_} )


Now with what you suggested above, I get:
Use of uninitialized value in numeric eq (==) at


FishMonger
Veteran / Moderator

Feb 21, 2013, 8:43 AM

Post #37 of 47 (2920 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

You need to add another condition to verify that the file is actually in both hashes, then test if their values are equal.


StarkRavingCalm
User

Feb 21, 2013, 8:46 AM

Post #38 of 47 (2918 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

Here is what I have:


Code
for ( keys %file2 ) 
{
if ( exists $file{$_} ) { next; }
push(@newfiles, $_);
}


for ( keys %file )
{
if ( exists $file2{$_} ) { next; }
push(@removedfiles, $_);

}


for ( keys %file )
{
if ( $file{$_} eq $file2{$_} )
#if ( $file{$_} == $file2{$_} )
{
push(@equal, $_);
next;
}
else
{
push(@notEqual, $_);
next;
}

}


StarkRavingCalm
User

Feb 21, 2013, 12:52 PM

Post #39 of 47 (2904 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

This is working now (again) Wink
I just had to correct some syntax errors....
Thank you!


StarkRavingCalm
User

Mar 1, 2013, 10:52 AM

Post #40 of 47 (2866 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

I am trying to extend this out to use an ftp server where applicable but am not getting the expected results.
I am trying to use the same logic against an ftp server and I always come up with these undefined entries for output.
How should this be sorted if I want the filename to be the key and the size to be the value?
What can be used to strip out the hidden files?
In the SFTP:: Foreign example I am using no_wanted => qr/^\./ but I dont think that applies here.



Code
foreach my $file ( $ftp->ls('-la') ) { $files{basename($file)} = (stat($file))[7]; 

}

print Dumper \%files;

Code
Here are the results: 
$VAR1 = {
'-rw-r--r-- 1 541 541 2739312 Mar 01 18:45 upfile2' => undef,
'-rw-rw-r-- 1 541 541 19 Mar 01 18:46 upfile3' => undef,
'drwxrwxr-x 2 541 541 4096 Mar 01 18:46 .' => undef,
'-rw-r--r-- 1 541 541 152 Mar 01 18:45 upfile1' => undef,
'drwxrwxr-x 2 541 541 4096 Mar 01 18:46 ..' => undef,
'-rw-rw-r-- 1 541 541 185 Mar 01 18:46 upfile4' => undef
};


FishMonger
Veteran / Moderator

Mar 1, 2013, 5:49 PM

Post #41 of 47 (2861 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

Why are you using basename()?

You need to parse $file with a regex or the split function to retrieve the filename.


Code
foreach my $file ( $ftp->ls('-la') ) { 
my $filename = (split ' ', $file)[-1];
next if $filename =~ /^\.\.?$/;
$files{$filename} = (stat($filename))[7];
}



StarkRavingCalm
User

Mar 2, 2013, 6:50 AM

Post #42 of 47 (2830 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

Still getting undef and hidden files but much closer:

$VAR1 = {
'upfile4' => undef,
'upfile1' => undef,
'.bash_history' => undef,
'upfile2' => undef,
'upfile3' => undef
};


FishMonger
Veteran / Moderator

Mar 2, 2013, 7:07 AM

Post #43 of 47 (2828 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

Sorry, that was my fault for only looking at the first part of the problem. The stat function is not needed.


Code
foreach my $file ($ftp->ls('-la')) { 
my ($size, $filename) = (split ' ', $file)[4,-1];
next if $filename =~ /^\./;
$files{$filename} = $size;
}



(This post was edited by FishMonger on Mar 2, 2013, 7:09 AM)


StarkRavingCalm
User

Mar 2, 2013, 7:41 AM

Post #44 of 47 (2821 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

That works thanks!
I'll need to spend some time to get VMS correct but this will definitely help...


StarkRavingCalm
User

Mar 4, 2013, 12:30 PM

Post #45 of 47 (2796 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

ok, sftp and ftp work great.
but running against vms gives weird results which seems to just be the way that the output is presented in vms.

Example output:
$VAR1 = {
'upfile_20130226_13221660.TXT' => undef,
'upfile_20130220145458.TXT' => undef,
'upfile_20130206112304.TXT' => undef,
'TMP.DIR' => undef,
'RPT.DIR' => undef,
'FTP_SERVER.LOG' => undef,
'SRT.DIR' => undef,
'PRM.DIR' => undef,
};

Here is what a 'dir' (in unix emulation mode) on the server looks like:

-rwxrwxr-x 1 * ras 980 Feb 5 16:13 upfile_20130226_13221660.TXT
-rwxrwxr-x 1 * ras 2450 Feb 5 16:20 upfile_20130220145458.TXT
-rwxrwxr-x 1 * ras 1960 Feb 6 11:23 upfile_20130206112304.TXT


No matter what field I try to set $size to in the example code above, it always comes back as undef.
Is this a problem with the separators?


NOTE: If I copy the results into a text file, I can get the fields I want with awk.
So does that mean that the fields separators are correctly formated?
What am I missing here?


(This post was edited by StarkRavingCalm on Mar 4, 2013, 12:55 PM)


FishMonger
Veteran / Moderator

Mar 4, 2013, 2:26 PM

Post #46 of 47 (2789 views)
Re: [StarkRavingCalm] Compare unchanged files in two arrays [In reply to] Can't Post

The size is the 5th field, which is index 4. Did you try that?

You can use /\s+/ in the split instead of ' '.
i.e.,

Code
my ($size, $filename) = (split /\s+/, $file)[4,-1];



StarkRavingCalm
User

Mar 4, 2013, 5:35 PM

Post #47 of 47 (2784 views)
Re: [FishMonger] Compare unchanged files in two arrays [In reply to] Can't Post

using dir instead or ls gives the correct results.

Thanks again for the help with this!


(This post was edited by StarkRavingCalm on Mar 4, 2013, 5:48 PM)

First page Previous page 1 2 Next page Last page  View All
 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives