mmcw2201
User
Feb 10, 2002, 5:26 AM
Post #1 of 20
(5398 views)
|
|
Sorting big files
|
Can't Post
|
|
I am using this code to sort:
$datafile = "path/to/flat/database/file";
The Datafile contains a flat database like this:
ID|Name|Category|Title|Image|Description|Price|Taxable
0028|KINKS|10|Candy From Mr.Dandy.|geenidee.gif|label:<br>tracklist:<br>|30.00|1|
0050|BISHOPS|1|Live!|geenidee.gif|label:<br>tracklist:<br>|40.00|1|
0051|CHURCH|2|Temperature Drop In Downtown|geenidee.gif|label:<br>tracklist:<br>|40.00|1|
But actual contains about 9000 lines!
open(DATA,"$datafile")) {
@{$r_data} = <DATA>;
close(DATA);
# Set values if values are empty
$r_in->{'row'} = 0 unless ($r_in->{'row'});
if ($r_in->{'row'}) {
$r_in->{'type'} = "a" unless ($r_in->{'type'});
$r_in->{'order'} = "a" unless ($r_in->{'order'});
}
else {
$r_in->{'type'} = "n" unless ($r_in->{'type'});
$r_in->{'order'} = "a" unless ($r_in->{'order'});
}
$r_data = sort_data($r_in->{'row'},$r_in->{'type'},$r_in->{'order'},$r_data);
#########################################################################
# #
# subroutine sort_data #
# Subroutine that does the actual sort. #
# Accepts 4 params viz. #
# 1. The column number to sort. Column no. start from 0. #
# 2. The type of sort numeric or alphabetic. Default is Alphabetic. #
# 3. The order of sort. Default order is ascending #
# 4. The referrence of the array that needs to be sorted. #
#########################################################################
sub sort_data {
my ($row,$type,$sort_order,$r_data) = @_;
my (@array);
if ($type eq "n") {
if ($sort_order eq "d") {
@array = map { (split ('<->', $_))[1] }
reverse sort {$a <=> $b}
map { join ('<->', lc ((split('\|', $_))[$row]) , $_) }
@{$r_data};
}
else {
@array = map { (split ('<->', $_))[1] }
sort {$a <=> $b}
map { join ('<->', lc ((split('\|', $_))[$row]) , $_) }
@{$r_data};
}
}
else {
if ($sort_order eq "d") {
@array = map { (split ('<->', $_))[1] }
reverse sort {$a cmp $b}
map { join ('<->', lc ((split('\|', $_))[$row]) , $_) }
@{$r_data};
}
else {
@array = map { (split ('<->', $_))[1] }
sort {$a cmp $b}
map { join ('<->', lc ((split('\|', $_))[$row]) , $_) }
@{$r_data};
}
}
return (\@array)
}
This code will work with small flat databases.
Using big flat databases it will not work because perl won't let it. You will get an timeout error.
I searched the internet and found a module called: file::sort. This module should work?? Or is there an other way to sort easily big flat databases! Has to work on NT and UNIX!!!
How to implement the file::sort module to the like the script I used above!
And is it possible to use this module without installing it. I can not install modules myself and my provider won't do it for me!! Some modules can be uploaded to a certain directory and you can link to it using something like this:
# Set module
use lib 'Path/to/Modules/Directory';
use file::Sort;
Can someone help me??
|
MAIN
SEARCH
WHO'S 
LOG 
Re: [mmcw2201] Sorting big files
[
You can probably stop the timeouts by sending dummy HTML every 500 lines or so (comments should work). I found that after about 4000 lines or so, the sorts can time out the browser. Writing your own manual database based on flatfiles, is kinda like re-inventing the wheel. MySQL is probably the best way to go, and well worth looking into.