CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
perl: restrict perl from automaticaly creating a hash branches on check

 

First page Previous page 1 2 Next page Last page  View All


alex5161
Novice

Nov 6, 2014, 8:53 AM

Post #1 of 30 (11254 views)
perl: restrict perl from automaticaly creating a hash branches on check Can't Post

My issue is that the perl script (as I have done it so far) created empty branches when I try to check some branches on existence.

I am using multydimentional hashes: found it as the best way for information that I need to handle. Saying multidimentional I means hash of hashes ... So, I have

Code
$hsh{k1}{k2}{k3}{k4}=v1; 
$hsh{k1}{k9}{k3}{k33}=v2;
....

In some point I need to check some 'set of keys' on existence in the hash. So, I check:

Code
if ( $hsh{ch1}{ch2}{ch3} ) {  ...something... }

And, suddenly I've realized, that such check creates the intermediate branches to access the final one for checking!

So, if there is no $hsh{ch1} before the check, it will be created to access the key 'ch2', which will be created to access the 'ch3'

QUESTION: Is there a way to avoid such aromatic creation ????

Logically, I would expect the Perl to get FALSE as soon as any key is not exist in the hash, but it is not happening!

Sure I could do:

Code
if ( $hsh{ch1} && $hsh{ch1}{ch2} && $hsh{ch1}{ch2}{ch3} )  
{...}

but it is so annoying, especially when I have many hashes with 7- 10 keys!

I have creates some function to check a hash 'reasonably', sending hash reference and array of keys to be checked, and it works fine, but it is slows down the processing visibly.

Here is an example of what I am talking about:

Code
  DB<4> $h{a}{b}=[1,2]; $h{a}{c}=[3,4]  #hash with two elements (arrays) and in 2 levels 

DB<5> use Data::Dumper #good way to display

DB<6> p Dumper\($a,%h)
$VAR1 = \undef;
$VAR2 = {
'a' => {
'c' => [
3,
4
],
'b' => [
1,
2
]
}
};

DB<7> p 'da' if !$h{p}{m} #checking for an existent keys and element
da
DB<8> p Dumper\($a,%h)
$VAR1 = \undef;
$VAR2 = {
'p' => {}, # <<== created one !!!
'a' => {
'c' => [
3,
4
],
'b' => [
1,
2
]
}
};

DB<9> delete $h{p}

Once again: Is there a way to avoid such behavior without creating additional function?

Thanks!


FishMonger
Veteran / Moderator

Nov 6, 2014, 9:46 AM

Post #2 of 30 (11252 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

Did you try the autovivification pragma as was suggested on your SO cross post?


Laurent_R
Veteran / Moderator

Nov 6, 2014, 10:17 AM

Post #3 of 30 (11250 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

The behaviour you are describing is autovivification, it is very helpful in most cases, and annoying in some other.

You can use the Autovification pragma already mentioned or will have to test level by level. Something like this (untested):


Code
if (exists $hsh{k1} and exists $hsh{k1} and ...)


It is probably possible to do without the exists, but I can't test right now.


Laurent_R
Veteran / Moderator

Nov 6, 2014, 10:47 AM

Post #4 of 30 (11249 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

Just tried it now, this is a syntax somewhat simpler than what I wrote above to prevent autovivification, demonstrated here under the Perl debugger:

Code
  DB<1> $h{a}{b}{c} = 2; 

DB<2> print "foo" if defined $h{d}{e}{f} ;

DB<3> print "foo" if $h{g} && $h{g}{h} && $h{g}{h}{i};

DB<4> x \%h;
0 HASH(0x600500b60)
'a' => HASH(0x600509d30)
'b' => HASH(0x600500a88)
'c' => 2
'd' => HASH(0x600500c38)
'e' => HASH(0x600500980)
empty hash
DB<5>


Line 1 (DB<1>) is just creating the hash, to make sure it exists.

Line 2 is checking for the definedness of a deeply nested element of the hash.

Line 3 is doing the same thing, but level by level. If the first conditional expression is false, the others will not be evaluated because the are short-circuited (the whole expression cannot be right is the first one is wrong (undefined).

Line 4 prints out the structure of the hash. You can see that line 2 created all the level below the last one (autovivification of levels 1 and 2), where as line 3 did not produce any autovivification (did not create subentries for the lower levels).

Update: corrected a small error in line 3 above, but the result is the same.


(This post was edited by Laurent_R on Nov 6, 2014, 3:07 PM)


alex5161
Novice

Nov 6, 2014, 11:59 AM

Post #5 of 30 (11242 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
Did you try the autovivification pragma as was suggested on your SO cross post?


I have meat that module on my search: seems it is exactly what I need!

The problem is that any CPAN modules are required installation in existing Perl that I am not allowed to do.
To complete it I would need to request that could have an approval complication and will required to have it installed on many servers : developer servers, model-office servers, and user and production servers.

I do not know if the 'autivivification.pm' could be just placed in any accessible for me directory and work from such place without installation...

(Just get response from the author of that module, Vincent, and have realized, that it requires some shared libraries, which should be used by *.pm module... So, no way to have it without full installation.)


(This post was edited by alex5161 on Nov 7, 2014, 5:32 AM)


alex5161
Novice

Nov 6, 2014, 12:12 PM

Post #6 of 30 (11238 views)
Re: [Laurent_R] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

Thanks, Laurent_R, for your reply!

I have realized the possibility to check a hash level-by-level.
It just annoying and much wordy - not a Perl way of programming.

Also, as I have mentioned in initial post, I have created the function to check defined/exist/has-value function that do it's work very well and pretty small and clean, but... BUT instead of initial 10 min before that now it runs 40 minutes!

That is the issue why I was looking for some native solution.

Seems, there is no, besides adding the CPAN functionality.

Anyway, thanks everybody for communicating with me on that issue. At least, now I know that it is not something I just do not know Smile


Laurent_R
Veteran / Moderator

Nov 6, 2014, 3:36 PM

Post #7 of 30 (11232 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
I have realized the possibility to check a hash level-by-level.
It just annoying and much wordy - not a Perl way of programming.

I agree. The Camel Book ("Perl programming") has a note somewhere saying that autovivification might be considered to be a bug when you are only checking for the existence of something and that it might be corrected in a future version, but this has not happened so far, and it is a bit of a pity in my view.

In Reply To
Also, as I have mentioned in initial post, I have created the function to check defined/exist/has-value function that do it's work very well and pretty small and clean, but... BUT instead of initial 10 min before that now it runs 40 minutes!

With the syntax in my last post above, the coding overhead is limited and the performance overhead should also be relatively minimal, but this last point really depends on your data shape (does it match often or only rarely, etc.), something that I have no idea about.

Please also take a look at the update I made to my answer to your cross post on Perl Monks (http://www.perlmonks.org/?node_id=1106408), I have added some caveats as to why the syntax used in my example above might be problematic in some special cases and might need some improvements.



alex5161
Novice

Nov 7, 2014, 6:54 AM

Post #8 of 30 (11217 views)
Re: [Laurent_R] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

Thanks, Lauren, for this site reply and I have responsed and link-here on PerlMonks-forum.
Regarding the performance overhead I guess to check every level of a hash should not be significant, but I did not try it yet and going to do it today.
The coding is (seems) biggest issue of that way: with number of 7-10 level hashes checked in multiple places it affecting the code acctualy.
For example, instead of :

Code
... if ( $hsh{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8} ) ...

it going to be:

Code
... if ( $hsh{k1} && 
$hsh{k1}{k2} &&
$hsh{k1}{k2}{k3} &&
$hsh{k1}{k2}{k3}{k4} &&
$hsh{k1}{k2}{k3}{k4}{k6} &&
$hsh{k1}{k2}{k3}{k4}{k6}{k7} &&
$hsh{k1}{k2}{k3}{k4}{k6}{k7}{k8} ) ...

- that is not reasonable text, for my opinion.

As I sayd, I prepare the function to check it level-by-level, but here is the performance:

Code
  DB<8> $tm=time(); for $i (1..7000000) { if ($h->{key1}{in1}{deep1}) {1;} }; p "time: ".(time()-$tm)."\n"; \ 
cont: $tm=time(); for $i (1..7000000) { if (h_dfnd($h, qw(key1 in1 deep1))) {1;} }; p "time: ".(time()-$tm)."\n";\
cont: $tm=time(); for $i (1..7000000) { if (h_df_rcs($h, qw(key1 in1 deep1))) {1;} }; p "time: ".(time()-$tm)."\n"
time: 6
time: 145
time: 253

- the number is selected to have some visible time in native check
Two function used here are different in way of accessing next level: first one (the h_dfnd()) do it in for-loop, another one (h_df_rcs()) use recursion.
I did some correction lately, but still could not beat the 25-30 time slowness by function call.
Here is the code:

Code
#########  h_dfnd()  ##################################### 
## check a multidimentional hash to be defined ('no value' is fine!)
## !!! - WITHOUT creating the intermediate branches !!!
## (otherwise, the check on $hsh{k1}{k2}{k3} creates
## the $hsh{k1}{k2} = '' !!! )
sub h_dfnd { my( $h, @ks ) = @_;
$value='';# expecting: local $value; - defined in callers to be set here with value if last key received
return 0 if !@ks; # if a hash - must have a key
for my $k ( @ks ) {
return (0 || (defined $h->{$k})) if !$h->{$k}; # so, do not process eval() and exist(), but if has empty value, returns 1 as defined(..)
# return 0 if ! eval { exists ($h->{$k}) }; #-ap- should it be this way? (as it on-line)
$h = $h->{$k};
}
$value = $h;
return 1;
}

######### h_exst() #####################################
## check if a multidimentional hash is defined and has (returns it) a value
sub h_exst{ local $value='';
return ( h_dfnd(@_) && $value); #the $value is set in the h_dfnd()
}

######### h_vl() ########################################
## returns value of a hash (whithout intermediate creations) or
## the received SCALAR if not a hash reference send by first parm.
sub h_vl{ local $value=$_[0];
return ((!ref $_[0] || h_dfnd(@_))? $value :'') ; # the $value is set in the h_dfnd()
}

sub h_df_rcs{ # recursive version of the h_dfnd()
$value='';# expecting: local $value; - defined in callers to be set here with value if last key received
return 0 if ! ref $_[0] || #must be hash-ref;
! defined $_[0]->{$_[1]}; # FALSE if NO 'hash'{first-key}
if (! $_[2] ){
$value = $_[0]->{$_[1]};
return 1 ; } # TRUE if no more key passed
#if here: first-key hash exist and more key(s) received; recursive call, one less key
return h_dfnd($_[0]->{$_[1]}, @_[2..$#_]);
}

If any idea/suggestion/bug fund I will appreciate it.


FishMonger
Veteran / Moderator

Nov 7, 2014, 7:57 AM

Post #9 of 30 (11213 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

If you're trying to find ways to speed up the script, the the very first thing you should do is profile it so that you can find out where the bottlenecks are located.

Devel::NYTProf - Powerful fast feature-rich Perl source code profiler

http://search.cpan.org/~timb/Devel-NYTProf-5.06/lib/Devel/NYTProf.pm


alex5161
Novice

Nov 7, 2014, 8:03 AM

Post #10 of 30 (11212 views)
Re: [Laurent_R] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

 I did testing performance comparising in direct-access with by-level access: conclusion:
- for 10-level hash by-level is slower near in 5 times.

Here is the testing (third cal is the function processing of that check) :

Code
  DB<18> $nmb=1000000;\ 
cont: $tm=time();for $i (1..$nmb){if($h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}{k9}{k10}){1;}}; p "direct: ".(time()-$tm)."\n";\
cont: $tm=time();for $i (1..$nmb){if($h->{k1} && $h->{k1}{k2} && $h->{k1}{k2}{k3} && $h->{k1}{k2}{k3}{k4}\
cont: && $h->{k1}{k2}{k3}{k4}{k5} && $h->{k1}{k2}{k3}{k4}{k5}{k6} && $h->{k1}{k2}{k3}{k4}{k5}{k6}\
cont: && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7} && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}\
cont: && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}{k9} && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}{k9}{k10} ) {1;} \
cont: }; p "by-level: ".(time()-$tm)."\n";\
cont: $tm=time();for $i (1..$nmb){if (h_dfnd($h, qw(k1 k2 k3 k4 k5 k6 k7 k8 k9 k10))){1;}}; p "h_dfnd(): ".(time()-$tm)."\n";
direct: 2
by-level: 10
h_dfnd(): 25

DB<19> $nmb=5000*1000; p $nmb
5000000
DB<20> $nmb=5000*1000;

DB<21> $tm=time();for $i (1..$nmb){if($h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}{k9}{k10}){1;}}; p "direct: ".(time()-$tm)."\n";\
cont: $tm=time();for $i (1..$nmb){if($h->{k1} && $h->{k1}{k2} && $h->{k1}{k2}{k3} && $h->{k1}{k2}{k3}{k4}\
cont: && $h->{k1}{k2}{k3}{k4}{k5} && $h->{k1}{k2}{k3}{k4}{k5}{k6} && $h->{k1}{k2}{k3}{k4}{k5}{k6}\
cont: && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7} && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}\
cont: && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}{k9} && $h->{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8}{k9}{k10} ) {1;} \
cont: }; p "by-level: ".(time()-$tm)."\n";\
cont: $tm=time();for $i (1..$nmb){if (h_dfnd($h, qw(k1 k2 k3 k4 k5 k6 k7 k8 k9 k10))){1;}}; p "h_dfnd(): ".(time()-$tm)."\n";
direct: 9
by-level: 50
h_dfnd(): 127



FishMonger
Veteran / Moderator

Nov 7, 2014, 8:57 AM

Post #11 of 30 (11206 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

Using hashes with that many levels of sub keys indicates to me that you have a design flaw.

Also, your performance tests would be better handled by using the Benchmark module which is a core module.

http://search.cpan.org/~rjbs/perl-5.18.4/lib/Benchmark.pm


(This post was edited by FishMonger on Nov 7, 2014, 9:04 AM)


FishMonger
Veteran / Moderator

Nov 7, 2014, 9:52 AM

Post #12 of 30 (11185 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

I have not done any testing to find more a efficient (manual) solution, but 1 problem you have with your testing is that instead of checking for the existence of a hash key to determine if that level exists, you're testing the truth of the value under that key. If the key exists but its value is 0 (possibly due to a bug elsewhere), then it fail even though the key exists.

That was my verbose way of saying you need to use the exists keyword.


Code
if(exists $h->{k1} && exists $h->{k1}{k2} && ...



Laurent_R
Veteran / Moderator

Nov 7, 2014, 10:07 AM

Post #13 of 30 (11182 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

Your performance test, if I understand it correctly, is not really significant. If you are testing a million times only the cost of checking definedness for each level down to the most nested (10th) one and compare that with trying to look directly at the lowest level, and find that the first one is significantly slower than the second, you haven't really proven anything useful. I could have told you before that it was going to be slower.

But that test is not realistic for basically two reasons:

- in the real program it is likely that, most of the time, you will not have to test all ten levels and that it will short-circuit out of the nested tests well before level 10.

- your real program is presumably doing many other things that just nested hash look up. If the time spent in hash look up is only 1% of your execution time, and if hash look up now takes ten times longer, that's still a slow-down of only 10% for your overall program.

So in such a situation you should really test with the real program (or at least something sufficiently similar to your real program).

As pointed by Fishmonger, you should first profile your program (with the profiling module mentioned by Fishmonger(as it is now, without the chained tests) and see how much time you are really passing in hash lookup. It might be a fairly small part of the overall program.

And I also agree with Fishmonger that you should probably also use a benchmarking module.


(This post was edited by Laurent_R on Nov 7, 2014, 11:00 AM)


Laurent_R
Veteran / Moderator

Nov 7, 2014, 10:51 AM

Post #14 of 30 (11172 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
... 1 problem you have with your testing is that instead of checking for the existence of a hash key to determine if that level exists, you're testing the truth of the value under that key. If the key exists but its value is 0 (possibly due to a bug elsewhere), then it fail even though the key exists.

That was my verbose way of saying you need to use the exists keyword.


Code
if(exists $h->{k1} && exists $h->{k1}{k2} && ...



I do not think this is really true (although there is a small part of truth). I thought about that problem and mentionned it already yesterday evening in an update to my post on Perl Monks (which is the reason I advised Alex, in my post here 20 hours ago, to go and read the caveats I had in this post on PM: http://www.perlmonks.org/?node_id=1106420).

The reason for which I claim that what you are saying is not really true is the following.

Suppose I want to check this:

Code
if ( $hsh{k1}{k2}{k3}{k4}{k5}{k6}{k7}{k8} ) ...

If it exists, then the values of $hsh{k1}, $hsh{k1}{k2}, ..., $hsh{k1}{k2}{k3}{k4}{k5}{k6}{k7} will be hash references , and a hash reference cannot be false (unless it is not defined, of course, but that by itself solves the issue and short circuits the test returning the correct FALSE Boolean value). If it doesn't exist, then the test for truth will fail one way or another before reaching the end. So that, as I said in my PM post yesterday evening (although my example was not so deeply nested), all you need to do is something like this:


Code
... if ( $hsh{k1} &&  
$hsh{k1}{k2} &&
$hsh{k1}{k2}{k3} &&
$hsh{k1}{k2}{k3}{k4} &&
$hsh{k1}{k2}{k3}{k4}{k6} &&
$hsh{k1}{k2}{k3}{k4}{k6}{k7} &&
exists $hsh{k1}{k2}{k3}{k4}{k6}{k7}{k8} ) ...


or possibly:


Code
... if ( $hsh{k1} &&  
$hsh{k1}{k2} &&
$hsh{k1}{k2}{k3} &&
$hsh{k1}{k2}{k3}{k4} &&
$hsh{k1}{k2}{k3}{k4}{k6} &&
$hsh{k1}{k2}{k3}{k4}{k6}{k7} &&
defined $hsh{k1}{k2}{k3}{k4}{k6}{k7}{k8} ) ...

depending on the exact purpose of the test.


FishMonger
Veteran / Moderator

Nov 7, 2014, 11:11 AM

Post #15 of 30 (11164 views)
Re: [Laurent_R] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

My point is that a conditional test should actually test what you meant and not rely on the side effect of a closely related test returning a nearly equivalent result.

In this case testing the truthfulness of the value as opposed to the existence of the hash key works up until the last key where its value could legitimately evaluate to false which ends up giving a false negative for the test.


(This post was edited by FishMonger on Nov 7, 2014, 11:15 AM)


FishMonger
Veteran / Moderator

Nov 7, 2014, 11:34 AM

Post #16 of 30 (11156 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


Quote

Code
return 0 if ! ref $_[0] ||             #must be hash-ref;


That is not testing for a hash ref as the comment states. It could be a HASH ref or ARRAY ref or CODE or a number of other types of reference.

http://perldoc.perl.org/functions/ref.html


FishMonger
Veteran / Moderator

Nov 7, 2014, 11:54 AM

Post #17 of 30 (11147 views)
Re: [alex5161] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

The perlfaq4 page that you were pointed to in one of the perlmonks responses says that besides the autovivification pragma you can use the Data::Diver module. It's a pure perl module so there are no dependency or installation issues like there is with the autovivification pragma. You can simply include the module file as part of your app and add a use lib statement to point to its location.

http://search.cpan.org/~tyemq/Data-Diver-1.0101/Diver.pm


(This post was edited by FishMonger on Nov 7, 2014, 11:56 AM)


Laurent_R
Veteran / Moderator

Nov 7, 2014, 2:41 PM

Post #18 of 30 (11136 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
My point is that a conditional test should actually test what you meant and not rely on the side effect of a closely related test returning a nearly equivalent result.

In this case testing the truthfulness of the value as opposed to the existence of the hash key works up until the last key where its value could legitimately evaluate to false which ends up giving a false negative for the test.


Uh? Are you maintaining that what you said about zero hashref values in the nested hash was right? Are you insinuating that what I said about it was wrong? It seems to me that I proved my point and that you did not. Are you denying that?

If not, then you might as well be fair and admit it. Usually, when someone shows that I was wrong in some of my comments, I might be a bit angry against myself for having made a mistake, but I am grateful to the person who pointed out the error, because I most probably learnt something useful from that error and may have a chanto to avoid it next time. And, in particular, I am very grateful to you, Fishmonger, because I learnt quite a bit from you over the last couple of years, and I think that you should probably remember that I usually thank you for that. Last time I did it was not older than yesterday: http://perlguru.com/gforum.cgi?post=80135;sb=post_latest_reply;so=ASC;forum_view=forum_view_collapsed;;page=unread#unread.


As for your comment about truthfullness versus existence (versus definedness), I appreciate what you are saying, but please explain the following best-practice syntax:

Code
while (my $line = <$file_in>) { # ...


Or the very idiomatic use of an array in Boolean context:

Code
print "array is empty\n" unless @array;


What? Can @array be true or false? Obviously not. But as you well know, an array returns the number of its elements when evaluated in a scalar context, and, if that scalar context is really a Boolean context, the number of elements is converted to true or false. Anything markedly different in the construction I suggested?

Or consider this test under the debugger:


Code
$ perl -de 42 

Loading DB routines from perl5db.pl version 1.33
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

main::(-e:1): 42
DB<1> use diagnostics; # just to get more complete information about the problem

DB<2> print "true" if defined %hash;
defined(%hash) is deprecated at (eval 7)[/usr/lib/perl5/5.14/perl5db.pl:640]
line 2 (#1)
(D deprecated) defined() is not usually useful on hashes because it
checks for an undefined scalar value. If you want to see if the hash
is empty, just use if (%hash) { # not empty } for example.

defined(%hash) is deprecated at (eval 7)[/usr/lib/perl5/5.14/perl5db.pl:640] line 2.
at (eval 7)[/usr/lib/perl5/5.14/perl5db.pl:640] line 2
eval '($@, $!, $^E, $,, $/, $\\, $^W) = @saved;package main; $^D = $^D | $DB::db_stop;
print "true" if defined %hash;;

;' called at /usr/lib/perl5/5.14/perl5db.pl line 640
DB::eval called at /usr/lib/perl5/5.14/perl5db.pl line 3436
DB::DB called at -e line 1
(Maybe you should just omit the defined()?)
(Maybe you should just omit the defined()?)
at (eval 7)[/usr/lib/perl5/5.14/perl5db.pl:640] line 2
eval '($@, $!, $^E, $,, $/, $\\, $^W) = @saved;package main; $^D = $^D | $DB::db_stop;
print "true" if defined %hash;;

;' called at /usr/lib/perl5/5.14/perl5db.pl line 640
DB::eval called at /usr/lib/perl5/5.14/perl5db.pl line 3436
DB::DB called at -e line 1

DB<3> print "true" if %hash;

DB<4> $hash{a}=1;

DB<5> print "true" if %hash;
true
DB<6>

As you can see, this syntax to figure out if a hash is defined and is populated:

Code
print "true" if defined %hash;

is deprecated, and the recommended syntax is:

Code
print "true" if %hash

No, there is no a side-effect in my example, this is typical Perl.

Please note that my very first answer in this thread suggested to use the exists operator at each level, but mentionned that maybe the syntax could be simplified and the exists operator removed. A bit later, when I was in a position to test, I showed that, indeed, it could be removed for every intermediate level of nesting. That was just a quick example under the debugger. This is why I then issued a warning that the test for truthfulness should not be used for the most deeply nested level, which could be defined but false.

You may not like this syntax simplification, and I can understand and admit your reasons, but there is nothing grammatically, syntactically or semantically wrong with it, as far as I can say.

Again, I am very thankful to you for what I learnt from you, and I do want to keep a friendly relationship with you, so I probably don't intend to further argue on the subject.


FishMonger
Veteran / Moderator

Nov 7, 2014, 3:05 PM

Post #19 of 30 (11133 views)
Re: [Laurent_R] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post

It was not my intention to imply that you were wrong. If that's how it came across, I apologize.

I was not referring to your latest post where you have example using exists and defined. I was referring to the OP's tests which don't include that test.

My point on testing the hash value instead of existence is demonstrated here:

Code
#!/usr/bin/perl 

use 5.010;
use strict;
use warnings;

my %hash;
$hash{level_1} = 0;

say "testing for truthfullness";
if ( $hash{level_1} ) {
say "hash key level_1 exists";
}
else {
say "hash key level_1 does not exist";
}


say "\ntesting for existence";
if ( exists $hash{level_1} ) {
say "hash key level_1 exists";
}
else {
say "hash key level_1 does not exist";
}



Quote
c:\test>hash_test.pl
testing for truthfullness
hash key level_1 does not exist

testing for existence
hash key level_1 exists



(This post was edited by FishMonger on Nov 7, 2014, 3:06 PM)


alex5161
Novice

Nov 9, 2014, 7:52 PM

Post #20 of 30 (10895 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
... profile it
Devel::NYTProf - Powerful fast feature-rich Perl source code profiler

http://search.cpan.org/~timb/Devel-NYTProf-5.06/lib/Devel/NYTProf.pm

- Thanks, will check it! Sound interesting!


alex5161
Novice

Nov 9, 2014, 8:31 PM

Post #21 of 30 (10893 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
Using hashes with that many levels of sub keys indicates to me that you have a design flaw.

While I understand intemtion to such conclusion, I am pretty sure you are not right in current situation.
I do not want to load all readers with details of my task, but, in few words:
deepness of hashes is reflection of database tables organisation. Amount of 'keys' reflects really complicated way of organized dependencies: k1 holds amount of k2, each of one get big number of k3, followed with number of k4 on every one; after that each mentioned set get set of some k8, branched, again, by k9 in some cases; k5,k6,k7, and others when exist are the reflection of some combination of first 4.
I need to read a file with 1-2 million records of such structure and collect some amounts of unsorted relative records.
After that I need update/insert database table, performing before that some key replacement, if needed and defining some key as a condition of other keys.
Because of amount of records it is much insufficient to process that by amount of iterations with database for each record - it takes hours!
I getting some extractions from database tables, that creates additional such multilevel hashes that used to define requered values and conditions for the record-based decisions.
That why I have many much-leveled hashes and have so significant affection of the hash values checking.

If you fill you could guess better way to perform my task (in Perl) I would be interested.


In Reply To
Also, your performance tests would be better handled by using the Benchmark module which is a core module.

http://search.cpan.org/~rjbs/perl-5.18.4/lib/Benchmark.pm

Probably, you right, but it seems the learn curve higher that need, effort and usefulness: my simple and quick check I guess enough correct and representative. I have used the benchmark once, so, get some limited experience...


alex5161
Novice

Nov 9, 2014, 8:38 PM

Post #22 of 30 (10891 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
...
That was my verbose way of saying you need to use the exists keyword.


Code
if(exists $h->{k1} && exists $h->{k1}{k2} && ...


I guess, you talking about my performance checking. If so, my understanding it is not so important and using 'exist' would only slow down.
I do not need to be 99.99(9)% correct: I need some general representation of such change.


alex5161
Novice

Nov 9, 2014, 9:04 PM

Post #23 of 30 (10889 views)
Re: [Laurent_R] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
Your performance test, if I understand it correctly, is not really significant. ....
But that test is not realistic for basically two reasons:

- in the real program it is likely that, most of the time, you will not have to test all ten levels and that it will short-circuit out of the nested tests well before level 10.

That if any useful information is expected to be found in earlier than the last level: in my task in 90% I have to reach the last level.
Well, from 1500K records 100K are not found: so more 90% reach the last level in checking and other.. probably in first 4 keys comes back: still near 30-40% of actions (if linear time spredding.)
So, finaly, roughly 95% of all-key check time going to be applied to testing hash existence.

In Reply To
- ... If the time spent in hash look up is only 1% of your execution time, ....

It is over-optimistical! :)
Because of amount of records and every record decisions.... everything else so insufficient (even it takes significant amount of codding...)

In Reply To
So in such a situation you should really test with the real program (or at least something sufficiently similar to your real program).

Didn't I mention yest?
Before I have realized that and add my function to check level-by-level, program completed in near 10 minutes.
Now it works more than 40 minutes!


alex5161
Novice

Nov 9, 2014, 9:13 PM

Post #24 of 30 (10887 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To

Quote

Code
return 0 if ! ref $_[0] ||             #must be hash-ref;


That is not testing for a hash ref as the comment states. It could be a HASH ref or ARRAY ref or CODE or a number of other types of reference.

http://perldoc.perl.org/functions/ref.html

OOh, well!
It is in the hash related function: I am not in the cort room and not goint to define any comments in absolutely-legally-true condition: just a hint to the reason of code line!
It also could be a scalar reference, btw...
Point of that check if it is already reach a 'bottom'.


alex5161
Novice

Nov 9, 2014, 9:17 PM

Post #25 of 30 (10885 views)
Re: [FishMonger] perl: restrict perl from automaticaly creating a hash branches on check [In reply to] Can't Post


In Reply To
... Data::Diver module. It's a pure perl module ...
http://search.cpan.org/~tyemq/Data-Diver-1.0101/Diver.pm

I did not check that and did not know:
Thanks, will try it (Could it be better than the 'manual' check, if it is done just by Perl?! ...Not sure, but..)

First page Previous page 1 2 Next page Last page  View All
 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives