CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: General Discussions: General Questions:
sort in perl

 



pnovice
New User

May 18, 2011, 10:57 PM

Post #1 of 6 (8733 views)
sort in perl Can't Post

Hi,

I have an array whose values are always a string of 4 numbers as follows

$array[0] = "23 25 2 -0.08"

$array[1] = "13 12 1 0.07"

... upto $array[500000]

I have to sort this array based on the last number in the string, i.e. -0.08 0.07 etc

Can anyone help?

Thanks,


miller
User

May 19, 2011, 1:17 AM

Post #2 of 6 (8730 views)
Re: [pnovice] sort in perl [In reply to] Can't Post


Code
 
use strict;
use warnings;

my @array = (
"23 25 2 -0.08",
"13 12 1 0.07",
"23 25 2 0.01",
"13 12 1 0.03",
);

my @sorted = map {$_->[1]}
sort {$a->[0] <=> $b->[0]}
map {[/([-.\d]+)$/, $_]} @array;

print "$_\n" for @sorted;


- Miller


pnovice
New User

May 19, 2011, 6:47 AM

Post #3 of 6 (8717 views)
Re: [miller] sort in perl [In reply to] Can't Post

Thanks miller for your response.

Last night after lot of reading up I came up with below

#sorting data
print "\tSorting data ...\n" ;
@sorted_data_array = reverse sort custom @data_array ;
sub custom {
$a =~ /(\S+)$/ ; $x1 = $1 ;
$b =~ /(\S+)$/ ; $y1 = $1 ;
abs($x1) <=> abs($y1) ; #need abs for my application
}


It is working. However, I have to run this on 15-50 million array sizes, and runtime is an issue.

I do not have good machines to test which method is faster.

Can you comment on that? If so, I can use yours

Thanks,


miller
User

May 19, 2011, 9:35 AM

Post #4 of 6 (8715 views)
Re: [pnovice] sort in perl [In reply to] Can't Post

Yours are mine are functionally the same thing (except for my missing abs). The main difference is that I'm doing a Schwartzian Transform to cache the key that you need to sort by. In yours it's recalculated every time which is going to slow down processing time, especially for so many records.

Memory might be an issue though for both methods. How large are your strings? How much processing power do you actually have? Don't worry, they're just rhetorical questions.

- Miller


pnovice
New User

May 19, 2011, 2:48 PM

Post #5 of 6 (8710 views)
Re: [miller] sort in perl [In reply to] Can't Post

Thanks Miller, i have verified that your method is about 5X faster. I had to eliminate several IO operation to measure just this piece of code.

Thanks.

Here is my machine info

OS: Linux 2.6.9-89.ELsmp x86_64
Model: Dual-Core AMD Opteron(tm) Processor 8220 SE
CPU: 8 @ 2813.195MHz
Memory: Ram: 31GB Swap: 78GB /tmp: 175GB


The script will go out to several other users and its hard to predict what they will run on, so I had to put the best code out.

Can you help me, if there is a speed up possibility in these other pieces of code? They all work correctly.

Section 1

#reading input data (Could be well over couple of GB) memory not an issue.
print "\tReading input file $fil ...\n" ;
$cnt = 0 ;
while (<INP>) {

if ($_ =~ /(\S+)\s+(\S+)/) {

$ref = sprintf("%10.6f", $1) ;
$val = sprintf("%10.6f", $2) ;
$err = $val-$ref ;
$err = sprintf("%10.6f", $err) ;
if ($ref!=0) { $pct = ($err/$ref)*100.0 ; $pct = sprintf("%.2f", $pct) ; } else { $pct = 0 ; }
$data_array[$cnt++] = "$ref $val $err $pct" ;
}

}
close(INP) ;


Section 2

#cumulative plot
print "\tWriting data file for cumulative plot...\n" ; #huge files being written out.
@sorted_p = sort { $a <=> $b } @p ;
$i = 0 ;
open (FIL, ">${o}data_cum") ;
foreach (@sorted_p) { $i++ ; print FIL "$_ $i\n" ;}
close(FIL) ;


Section 3

#write a part of an array into a file
print "\tWriting data file for scatter plot...\n" ;
open (FIL, ">${o}data") ;
for ($i=$remove; $i<=$#sorted_data_array; $i++) {
if ($sorted_data_array[$i] =~ /(\S+)$/) {
push(@p,$1) ;
print FIL "$sorted_data_array[$i]\n" ;
}
}
close(FIL) ;


miller
User

May 19, 2011, 3:38 PM

Post #6 of 6 (8708 views)
Re: [pnovice] sort in perl [In reply to] Can't Post

There's no obvious places to increase efficiency in the additional code that you've shown. Sorts are kind of obvious cases where algorithmic efficency steps can be made.

- M

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives