CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
optimize script to sorting

 



unreal
Novice

Nov 2, 2013, 5:21 AM

Post #1 of 12 (967 views)
optimize script to sorting Can't Post

Hi,

I'm writing a script to sort the links but it works very slowly.

4links - working good
100- not working or slow working
Must be working with 1000 links.

Please something advise or give me ready solutions.
And if anyone can explain to me why such a simple script working so slow?

Please help.
Sorry for my english.

unreal


Code
#!/usr/bin/perl 

use strict;
use warnings;

open(PLIK_1, "<", "cat1.txt") or die "Nie można otworzyć pliku: $!";
my @scalar = <PLIK_1>;
print "- plik cat1.txt wczytany\n";
close PLIK_1;
open(PLIK_2, "<", "rap.txt") or die "Nie można otworzyć pliku: $!";
my @scalar2 = <PLIK_2>;
print "- plik rap.txt wczytany\n";
close PLIK_2;
open(PLIK_3, "<", "ego1.txt") or die "Nie można otworzyć pliku: $!";
my @scalar3 = <PLIK_3>;
print "- plik ego1.txt wczytany\n";
close PLIK_3;
my $a;
my $b;
my $c;


my $i = 0;
my $k =0;
my $e = 0;
while($i <= $#scalar)
{
if($scalar[$i] =~ m/xxxx.net\/(.*)\/(.*)$/){
$b = $2;
}
if($scalar2[$i] =~ m/xxxx2.net\/(.*)(.*)\/(.*)\.html/){
$a = $3;
}


if($a eq $b)
{
open(PLIK_cat, ">>", "cat.txt") or die "Nie można otworzyć pliku: $!";
print PLIK_cat $scalar[$k++];
close PLIK_cat;
$i++;
}
else
{
if($k == $#scalar2)
{
$k = 0;
}
else
{
$k++;
}
}



}
my $s = 0;
my $a1;
my $c2;
while($s <= $#scalar)
{
if($scalar2[$s] =~ m/xxxx2.net\/(.*)(.*)\/(.*)\.html/){
$a1 = $3;
}
if($scalar3[$s] =~ m/xxxx3.com\/(.*)\/(.*)$/){
$c2 = $2;
}
if($a1 eq $c2)
{
open(PLIK_ego, ">>", "ego.txt") or die "Nie można otworzyć pliku: $!";
print PLIK_ego $scalar3[$e++];
close PLIK_ego;
$s++;
}
else
{
if($e == $#scalar3)
{
$e = 0;
}
else
{
$e++;
}
}
}



(This post was edited by unreal on Nov 2, 2013, 5:24 AM)


FishMonger
Veteran / Moderator

Nov 2, 2013, 6:17 AM

Post #2 of 12 (960 views)
Re: [unreal] optimize script to sorting [In reply to] Can't Post

Part of the reason the script is slow is because you're opening/reopening cat.txt and ego.txt inside a loop. Open those files prior to the loop and close them after the loop.


FishMonger
Veteran / Moderator

Nov 2, 2013, 7:05 AM

Post #3 of 12 (955 views)
Re: [unreal] optimize script to sorting [In reply to] Can't Post

If you post a sample of each of the input files and explain what you need to accomplish, we could probably show you a better approach.

Are all 3 input files the same size?

Some thinks to consider.
1) Var names should describe the data they hold, which yours don't.

2) Vars should be declared in the smallest scope they require and as reasonably close as possible to where they are first used.

3) $a and $b are built-in global vars used in sort routines and it would be best to not use them outside of that context.

4) Using proper indentation and vertical whitespace is important because it makes it much easier to read/follow what the the code is doing. Your indentation is inconsistent and in several areas it's missing.

5) Your regex's are using 2 or 3 sets of capturing parens, but you're only using 1 of them in each case. You should only capture what you need.

6) There are modules available for parsing url's. You might consider using one of them.


unreal
Novice

Nov 2, 2013, 7:52 AM

Post #4 of 12 (950 views)
Re: [FishMonger] optimize script to sorting [In reply to] Can't Post

 
1.reads files into array
2.compares the elements of the array where $scalar2(file rap.txt) is a pattern

files sample

ego1.txt

Code
http://xxxx3.com/SJm9IdKHLSegffXl/xxxx-xxxx-e0002-xxxx-xxxx.xxxx 
http://xxxx3.com/ujT68cPs8NgWf0CG/xxxx-xxxx-e0004-xxxx-xxxx.xxxx
http://xxxx3.com/k1wwoXk75jJWfcG6/xxxx-xxxx-e0001-xxxx-xxxx.xxxx
http://xxxx3.com/eOpa1Egf0oG5NzIG/xxxx-xxxx-e0003-xxxx-xxxx.xxx


cat1.txt

Code
http://xxx1.net/0W9WQ5lHm3Nvw8BM/xxxx-xxxx-e0004-xxxx-xxxx.xxxx 
http://xxx1.net/WapPp3u45s9NsHHZ/xxxx-xxxx-e0001-xxxx.xxxx
http://xxx1.net/U1yfKVLfbZRNTqBN/xxxx-xxxx-e0003-xxxx-xxxx.xxxx
http://xxx1.net/KeRW0kD9rbKHZuCj/xxxx-xxxx-e0002-xxxx-xxxx.xxxx


rap.txt

Code
http://xxxx2.net/file/cc23bfcf9872da7ee86066632425ee0f/xxxx-xxxx-e0001-xxxx-xxxx.xxxx.html 
http://xxxx2.net/file/f73104c93386efc357e9e5e60bda9d6b/xxxx-xxxx-e0002-xxxx-xxxx.xxxx.html
http://xxxx2.net/file/94c8083f0e05ecc0a0243ac8fc78cd2b/xxxx-xxxx-e0003-xxxx-xxxx.xxxx.html
http://xxxx2.net/file/cd5a7d930094efe00a03c4ec3a117239/xxxx-xxxx-e0004-xxxx-xxxx.xxxx.html


next
http://xxxx3.com/eOpa1Egf0oG5NzIG/xxxx-xxxx-e0003-xxxx-xxxx.xxx
http://xxx1.net/U1yfKVLfbZRNTqBN/xxxx-xxxx-e0003-xxxx-xxxx.xxxx
http://xxxx2.net/file/94c8083f0e05ecc0a0243ac8fc78cd2b/xxxx-xxxx-e0003-xxxx-xxxx.xxxx.html

sample -> xxxx-xxxx-e0003-xxxx-xxxx.xxxx

./script <- run

final file

cat.txt

Code
http://xxx1.net/WapPp3u45s9NsHHZ/xxxx-xxxx-e0001-xxxx.xxxx 
http://xxx1.net/KeRW0kD9rbKHZuCj/xxxx-xxxx-e0002-xxxx-xxxx.xxxx
http://xxx1.net/0W9WQ5lHm3Nvw8BM/xxxx-xxxx-e0004-xxxx-xxxx.xxxx
http://xxx1.net/U1yfKVLfbZRNTqBN/xxxx-xxxx-e0003-xxxx-xxxx.xxxx

ego.txt

Code
http://xxxx3.com/k1wwoXk75jJWfcG6/xxxx-xxxx-e0001-xxxx-xxxx.xxxx 
http://xxxx3.com/SJm9IdKHLSegffXl/xxxx-xxxx-e0002-xxxx-xxxx.xxxx
http://xxxx3.com/eOpa1Egf0oG5NzIG/xxxx-xxxx-e0003-xxxx-xxxx.xxx
http://xxxx3.com/ujT68cPs8NgWf0CG/xxxx-xxxx-e0004-xxxx-xxxx.xxxx



Now you know what I mean?



Code
#!/usr/bin/perl 

use strict;
use warnings;



open(PLIK_1, "<", "cat1.txt") or die "Nie mona otworzy pliku: $!";
my @scalar = <PLIK_1>;
print "- plik cat1.txt wczytany\n";
close PLIK_1;
open(PLIK_2, "<", "rap.txt") or die "Nie mona otworzy pliku: $!";
my @scalar2 = <PLIK_2>;
print "- plik rap.txt wczytany\n";
close PLIK_2;
open(PLIK_3, "<", "ego1.txt") or die "Nie mona otworzy pliku: $!";
my @scalar3 = <PLIK_3>;
print "- plik ego1.txt wczytany\n";
close PLIK_3;
my $a;
my $b;
my $c;


my $i = 0;
my $k =0;
my $e = 0;

open(PLIK_cat, ">>", "cat.txt") or die "Nie mona otworzy pliku: $!";
while($i <= $#scalar)
{
if($scalar[$i] =~ m/xxxx1.net\/(.*)\/(.*)$/){
$b = $2;
}
if($scalar2[$i] =~ m/xxxx2.net\/(.*)(.*)\/(.*)\.html/){
$a = $3;
}

if($a eq $b)
{

print PLIK_cat $scalar[$k++];

print "Dodano ", $i," link catshare";
$i++;

}
else
{
if($k == $#scalar2)
{
$k = 0;
}
else
{
$k++;
}
}
}
close PLIK_cat;


my $s = 0;
my $a1;
my $c2;

open(PLIK_ego, ">>", "ego.txt") or die "Nie mona otworzy pliku: $!";

while($s <= $#scalar)
{
if($scalar2[$s] =~ m/xxxx2.net\/(.*)(.*)\/(.*)\.html/){
$a1 = $3;
}
if($scalar3[$s] =~ m/xxxx3.com\/(.*)\/(.*)$/){
$c2 = $2;
}
if($a1 eq $c2)
{

print PLIK_ego $scalar3[$e++];
print "Dodano ", $a," link egofiles";
$s++;
}
else
{
if($e == $#scalar3)
{
$e = 0;
}
else
{
$e++;
}
}
}
close PLIK_ego;



FishMonger
Veteran / Moderator

Nov 2, 2013, 8:34 AM

Post #5 of 12 (944 views)
Re: [unreal] optimize script to sorting [In reply to] Can't Post

Why are these 2 lines in your final output when there is no matching pattern in rap.txt?

http://xxx1.net/WapPp3u45s9NsHHZ/xxxx-xxxx-e0001-xxxx.xxxx
http://xxxx3.com/eOpa1Egf0oG5NzIG/xxxx-xxxx-e0003-xxxx-xxxx.xxx


unreal
Novice

Nov 2, 2013, 8:53 AM

Post #6 of 12 (941 views)
Re: [FishMonger] optimize script to sorting [In reply to] Can't Post

xxx replaced the real names so you don't looking their number

example
http://perlguru.com/gforum.cgi?forum=2;do=forum_view_collapsed;

http://xxxx1.com/gforum.cgi?forum=2;do=xxxxx;

1,2,3 are the only other domain


FishMonger
Veteran / Moderator

Nov 2, 2013, 8:58 AM

Post #7 of 12 (937 views)
Re: [unreal] optimize script to sorting [In reply to] Can't Post

You're not being very clear on your description of what you need.

Are you saying that given this entry in rap.txt

Quote
http://xxxx2.net/file/cc23bfcf9872da7ee86066632425ee0f/xxxx-xxxx-e0001-xxxx-xxxx.xxxx.html

The pattern you need to match is e0001?


unreal
Novice

Nov 2, 2013, 9:07 AM

Post #8 of 12 (930 views)
Re: [FishMonger] optimize script to sorting [In reply to] Can't Post

sorted in the order of links in first file
this
The pattern you need to match is xxxx-xxxx-e0003-xxxx-xxxx.xxx

example

Code
if($scalar3[$s] =~ m/xxxx3.com\/(.*)\/(.*)$/){  
$c2 = $2;

$2 = xxxx-xxxx-e0003-xxxx-xxxx.xxx


########################
where to save the link?
this
The pattern you need to match is file rap.txt = xxxx2.net
The pattern you need to match is file ego1.txt = xxxx3.net
The pattern you need to match is file cat1.txt = xxxx1.net


FishMonger
Veteran / Moderator

Nov 2, 2013, 9:14 AM

Post #9 of 12 (921 views)
Re: [unreal] optimize script to sorting [In reply to] Can't Post

Sorry, but you're not making too much sense.

Try this rewrite to see if it's close to what you want.


Code
#!/usr/bin/perl 

use strict;
use warnings;

my %pattern;

open( my $rap_fh, '<', 'rap.txt' ) or die "Nie można otworzyć 'rap.txt': $!";

while (my $line = <$rap_fh>) {
$pattern{$1}++ if $line =~ m{^.+/(.+)\.html$};
}
close $rap_fh;


open( my $cat_fh, '>', 'cat.txt' ) or die "Nie można otworzyć 'cat.txt': $!";
open( my $cat1_fh, '<', 'cat1.txt' ) or die "Nie można otworzyć 'cat1.txt': $!";

while (my $line = <$cat1_fh>) {
if ($line =~ m{^.+/(.+)$}) {
if (exists $pattern{$1}) {
print {$cat_fh} $line;
}
}
}
close $cat_fh;
close $cat1_fh;


open( my $ego_fh, '>', 'ego.txt' ) or die "Nie można otworzyć 'ego.txt': $!";
open( my $ego1_fh, '<', 'ego1.txt' ) or die "Nie można otworzyć 'ego1.txt': $!";

while (my $line = <$ego1_fh>) {
if ($line =~ m{^.+/(.+)$}) {
if (exists $pattern{$1}) {
print {$ego_fh} $line;
}
}
}
close $ego_fh;
close $ego1_fh;



unreal
Novice

Nov 2, 2013, 9:27 AM

Post #10 of 12 (917 views)
Re: [FishMonger] optimize script to sorting [In reply to] Can't Post

saves in the wrong order.

Example.
rap.txt

site.com/file1
site.com/file2
site.com/file3

cat1
site.ru/file1
site.ru/file3
site.ru/file2

run script

cat.txt

site.ru/file1
site.ru/file2
site.ru/file3

Example 2.
rap.txt

site.com/filewsx
site.com/filewss
site.com/filewre

cat1
site.ru/filewsx
site.ru/filewre
site.ru/filewss

run script

cat.txt

site.ru/filewsx
site.ru/filewss
site.ru/filewre


and your script

Example.
rap.txt

site.com/file1
site.com/file2
site.com/file3

cat1
site.ru/file1
site.ru/file3
site.ru/file2

run script

cat.txt

site.ru/file1
site.ru/file3
site.ru/file2


(This post was edited by unreal on Nov 2, 2013, 9:40 AM)


FishMonger
Veteran / Moderator

Nov 2, 2013, 9:39 AM

Post #11 of 12 (907 views)
Re: [unreal] optimize script to sorting [In reply to] Can't Post

Try this update.

Code
#!/usr/bin/perl 

use strict;
use warnings;

my %pattern;

open( my $rap_fh, '<', 'rap.txt' ) or die "Nie można otworzyć 'rap.txt': $!";

while (my $line = <$rap_fh>) {
$pattern{$1}++ if $line =~ m{^.+/(.+)\.html$};
}
close $rap_fh;


open( my $cat1_fh, '<', 'cat1.txt' ) or die "Nie można otworzyć 'cat1.txt': $!";
my %cat;
while (my $line = <$cat1_fh>) {
chomp $line;
if ($line =~ m{^.+/(.+)$}) {
if (exists $pattern{$1}) {
$cat{$1} = $line;
}
}
}
close $cat1_fh;

open( my $cat_fh, '>', 'cat.txt' ) or die "Nie można otworzyć 'cat.txt': $!";
for my $key (sort keys %cat) {
print {$cat_fh} "$cat{$key}\n";
}
close $cat_fh;


open( my $ego1_fh, '<', 'ego1.txt' ) or die "Nie można otworzyć 'ego1.txt': $!";
my %ego;
while (my $line = <$ego1_fh>) {
chomp $line;
if ($line =~ m{^.+/(.+)$}) {
if (exists $pattern{$1}) {
$ego{$1} = $line;
}
}
}
close $ego1_fh;

open( my $ego_fh, '>', 'ego.txt' ) or die "Nie można otworzyć 'ego.txt': $!";
for my $key (sort keys %ego) {
print {$ego_fh} "$ego{$key}\n";
}
close $ego_fh;


If it doesn't work, then you'll need to do a better job of explaining what you need it to do.


(This post was edited by FishMonger on Nov 2, 2013, 9:40 AM)


unreal
Novice

Nov 2, 2013, 9:49 AM

Post #12 of 12 (904 views)
Re: [FishMonger] optimize script to sorting [In reply to] Can't Post

working good
thank you very much

now i must understand this write. I'm going to manual.
Thanks for your forbearance

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives