CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
Search Posts SEARCH
Who's Online WHO'S
Log in LOG

Home: Perl Programming Help: Beginner:
Compare two flat file via sorting first


New User

Nov 18, 2011, 1:58 AM

Post #1 of 1 (316 views)
Compare two flat file via sorting first Can't Post

Hello Guys,

This is my first post to this wonderful forum

I am trying to compare two flat file based on third column(file separator is '|')For low volume file its working fine but for huge file its taking large amount of time

So I am trying to sort the data first in two flat file and then trying to compare them

My questions are

1)Will this approach work if not please suggest any

2)I am unable to put the sorted array into file or trying to compare the two arrays

I am sending you both the script .Please let me now how to code it through

Script for comparison

use strict;
use warnings;
use POSIX qw(strftime);

my $stmp = strftime "%Y%m%d_%H%M%S", localtime;
print "$stmp\n";

my $lx_path = '/opt/grpwhse/stage/';
my $file_prv = $ARGV[1];
my $file_prv1 = substr($file_prv,0,index($file_prv,'.'));
my $file_last = $ARGV[0];
my $file_curr = $ARGV[1];
my $delta = '_delta_';
my $file_ext = '.dat';

my ($file1, $file2, $file3) = ($file_last,$lx_path.$file_curr,$file_prv1.$delta.$stmp.$file_ext);
open my $fh1, '<', $file1 or die "Can't open $file1: $!";
open my $fh2, '<', $file2 or die "Can't open $file2: $!";
open my $fh3, '>', $file3 or die "Can't open $file3: $!";

my %save; #Hash of hashes to store records from file1 for comparison with file2
while (<$fh1>){
my @rec = split /\|/;
my $key = $rec[2];
$save{$key}->{'data'} = $_; #Save current record in hash
$save{$key}->{'flag'} = 'D';

while (<$fh2>){
my @rec = split /\|/;
my $key = $rec[2];
#$save{$key}->{'data_2'} = $_;

if (not exists $save{$key}){
$save{$key}->{'data'} = $_;
$save{$key}->{'flag'} = 'I';
}elsif (string_to_compare($_) ne string_to_compare($save{$key}->{'data'})){
$save{$key}->{'data'} = $_;
$save{$key}->{'flag'} = 'U';
delete $save{$key};

foreach (sort keys %save){
my $data_flag = string_to_compare($save{$_}->{'data'});
my $data_key = string_to_key($save{$_}->{'data'});
print $fh3 "$data_key|$save{$_}->{'flag'}|$data_flag\n";

sub string_to_compare{
my $line = shift;
my ($skip1, $skip2, $key, $remainder) = split /\|/, $line,4;
return $remainder;

sub string_to_key{
my $line = shift;
my ($id1, $id2, $key, $remainder) = split /\|/, $line,4;
return "$id1|$id2|$key";


Script for sorting

use strict;
use warnings;

open (my $data , '<', $ARGV[0])|| die "could not open $ARGV[0]:\n$!";
my @array=(<$data>);
my @sorted=sort {(split(/\|/,$a))[2]<=>(split(/\|/,$b))[2]} @array;
print @sorted;

I want to use the above sort functionality in the file comparison script

But I am not able to

Please help!!!Thanks a lot for your time!!!!!!!!!


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives