Perl Guru Forums: General Discussions: General Questions: sort in perl

CGI/Perl Guide | Learning Center | Forums | Advertise | Login

Site Search: in


	MAIN INDEX	SEARCH POSTS	WHO'S ONLINE	LOG IN

Home: General Discussions: General Questions:
sort in perl

pnovice
New User

May 18, 2011, 10:57 PM

Post #1 of 6 (9546 views)

sort in perl	Can't Post

Hi,

I have an array whose values are always a string of 4 numbers as follows

$array[0] = "23 25 2 -0.08"

$array[1] = "13 12 1 0.07"

... upto $array[500000]

I have to sort this array based on the last number in the string, i.e. -0.08 0.07 etc

Can anyone help?

Thanks,

miller
User

May 19, 2011, 1:17 AM

Post #2 of 6 (9543 views)

Re: [pnovice] sort in perl [In reply to]	Can't Post

Code

 
use strict; 
use warnings; 
 
my @array = ( 
	"23 25 2 -0.08", 
	"13 12 1 0.07", 
	"23 25 2 0.01", 
	"13 12 1 0.03", 
); 
 
my @sorted = map {$_->[1]} 
	sort {$a->[0] <=> $b->[0]} 
	map {[/([-.\d]+)$/, $_]} @array; 
	 
print "$_\n" for @sorted;

- Miller

pnovice
New User

May 19, 2011, 6:47 AM

Post #3 of 6 (9530 views)

Re: [miller] sort in perl [In reply to]	Can't Post

Thanks miller for your response.

Last night after lot of reading up I came up with below

#sorting data
print "\tSorting data ...\n" ;
@sorted_data_array = reverse sort custom @data_array ;
sub custom {
$a =~ /(\S+)$/ ; $x1 = $1 ;
$b =~ /(\S+)$/ ; $y1 = $1 ;
abs($x1) <=> abs($y1) ; #need abs for my application
}

It is working. However, I have to run this on 15-50 million array sizes, and runtime is an issue.

I do not have good machines to test which method is faster.

Can you comment on that? If so, I can use yours

Thanks,

miller
User

May 19, 2011, 9:35 AM

Post #4 of 6 (9528 views)

Re: [pnovice] sort in perl [In reply to]	Can't Post

Yours are mine are functionally the same thing (except for my missing abs). The main difference is that I'm doing a Schwartzian Transform to cache the key that you need to sort by. In yours it's recalculated every time which is going to slow down processing time, especially for so many records.

Memory might be an issue though for both methods. How large are your strings? How much processing power do you actually have? Don't worry, they're just rhetorical questions.

- Miller

pnovice
New User

May 19, 2011, 2:48 PM

Post #5 of 6 (9523 views)

Re: [miller] sort in perl [In reply to]	Can't Post

Thanks Miller, i have verified that your method is about 5X faster. I had to eliminate several IO operation to measure just this piece of code.

Thanks.

Here is my machine info

OS: Linux 2.6.9-89.ELsmp x86_64
Model: Dual-Core AMD Opteron(tm) Processor 8220 SE
CPU: 8 @ 2813.195MHz
Memory: Ram: 31GB Swap: 78GB /tmp: 175GB

The script will go out to several other users and its hard to predict what they will run on, so I had to put the best code out.

Can you help me, if there is a speed up possibility in these other pieces of code? They all work correctly.

Section 1

#reading input data (Could be well over couple of GB) memory not an issue.
print "\tReading input file $fil ...\n" ;
$cnt = 0 ;
while (<INP>) {

if ($_ =~ /(\S+)\s+(\S+)/) {

$ref = sprintf("%10.6f", $1) ;
$val = sprintf("%10.6f", $2) ;
$err = $val-$ref ;
$err = sprintf("%10.6f", $err) ;
if ($ref!=0) { $pct = ($err/$ref)*100.0 ; $pct = sprintf("%.2f", $pct) ; } else { $pct = 0 ; }
$data_array[$cnt++] = "$ref $val $err $pct" ;
}

}
close(INP) ;

Section 2

#cumulative plot
print "\tWriting data file for cumulative plot...\n" ; #huge files being written out.
@sorted_p = sort { $a <=> $b } @p ;
$i = 0 ;
open (FIL, ">${o}data_cum") ;
foreach (@sorted_p) { $i++ ; print FIL "$_ $i\n" ;}
close(FIL) ;

Section 3

#write a part of an array into a file
print "\tWriting data file for scatter plot...\n" ;
open (FIL, ">${o}data") ;
for ($i=$remove; $i<=$#sorted_data_array; $i++) {
if ($sorted_data_array[$i] =~ /(\S+)$/) {
push(@p,$1) ;
print FIL "$sorted_data_array[$i]\n" ;
}
}
close(FIL) ;

miller
User

May 19, 2011, 3:38 PM

Post #6 of 6 (9521 views)

Re: [pnovice] sort in perl [In reply to]	Can't Post

There's no obvious places to increase efficiency in the additional code that you've shown. Sorts are kind of obvious cases where algorithmic efficency steps can be made.

- M

Search for (options)

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives