CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
Calculating Avg For different combinations in hash.

 



Tejas
User

Jun 10, 2014, 4:33 AM

Post #1 of 12 (13312 views)
Calculating Avg For different combinations in hash. Can't Post

Below is the data

Quote
744418169142,831,5,12,-.69,AURO,5MAU,20-DEC-13
744416737392,831,5,12,-15.22,AURO,5MAU,20-DEC-13
744417811392,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744418279342,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744415298192,831,5,12,-12.99,AURO,5MAU,20-DEC-13
744410571242,831,5,12,-5.89,AURO,5MAU,20-DEC-13
744411599242,831,5,12,-4.49,AURO,5MAU,20-DEC-13
744415584292,831,5,12,-16.48,AURO,5MAU,20-DEC-13
744410707242,831,5,12,-5.49,AURO,5MAU,20-DEC-13
744418511242,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744415075392,831,5,12,-10.99,AURO,5MAU,20-DEC-13
744416321142,831,5,12,-6.29,AURO,5MAU,20-DEC-13
744412254242,831,5,12,-9.99,AURO,5MAU,20-DEC-13
744415959342,831,5,12,-9.48,AURO,5MAU,20-DEC-13
744418642192,831,5,12,-8.99,AURO,5MAU,20-DEC-13
744419740342,831,5,12,-7.49,AURO,5MAU,20-DEC-13
744417666242,831,5,12,-10.99,AURO,5MAU,20-DEC-13
744417680242,831,5,12,-1.29,AURO,5MAU,20-DEC-13
744410020192,831,5,12,-1.99,AURO,5MAU,20-DEC-13
744417559292,831,5,12,-5.49,AURO,5MAU,20-DEC-13
744419275242,831,5,12,-1.99,AURO,5MAU,20-DEC-13
744419484242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744411455242,331,5,12,-.730,AURO,5MAU,20-DEC-13
744416321142,331,5,12,-6.29,AURO,5MAU,20-DEC-13
744412254242,331,5,12,-9.99,AURO,5MAU,20-DEC-13
744415959342,331,5,12,-9.48,AURO,5MAU,20-DEC-13
744418642192,331,5,12,-8.99,AURO,5MAU,20-DEC-13
744419740342,331,5,12,-7.49,AURO,5MAU,20-DEC-13
744417666242,331,5,12,-10.99,AURO,5MAU,20-DEC-13
744417680242,331,5,12,-1.29,AURO,5MAU,20-DEC-13
744410020192,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744417559292,331,5,12,-5.49,AURO,5MAU,20-DEC-13
744419275242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744419484242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744411455242,331,5,12,-.730,AURO,5MAU,20-DEC-13




Quote

Here i need STD DEVIATION AND MEAN diffrent ID's from above data (Can be many ID's) 331 and 831.


Code
while (my $line = <INPUT>) 
{
chomp $line;
my ($date,$amount,$etid) = (split /,/ , $line)[7,4,1];
$Daily_Hash{$id}{$date} += $amount; #Date is necessary as we have to calculate for whole month sometimes . and Avg needs the num of days.
#Though i have pasted the dataa for just one day
}

walk_hash(\%Daily_Hash);

my @Total_Items;

sub walk_hash {
my $h = shift;
foreach my $key (keys %$h) {
print "Walking In: $key \n";
if( ref $h->{$key}) {
walk_hash( $h->{$key} );
}
else {
push @Total_Items,$h->{$key} ; # This has to be seperated for 331 and 831 . Right nw , every thing is getting into the array (Bith 831 and 331).
#Iam Missing out here , STDDEV and Avg has to be calculated Per ID.
print "$h->{$key} \n";
}
}
}

my $Mean = average(@Total_Items);
my $stddev = stdev(@Total_Items);
print "Mean is $Mean , Standard Deviation is $stddev \n";


sub average{
my @array = @_;
my $size = @_;
# print "Size is $size \n" ;
my $total = 0;
foreach my $array(@array) {
$total += $array;
}
# print "$total is total \n" ;
my $average = $total/$size;
return $average;

}
sub stdev{

my $std ;
my @array = @_;
my $size = @_;
my $average = &average(@array);
my $sqtotal = 0;
foreach my $array(@array) {
$sqtotal += ($average-$array) ** 2; #Square
}
if($size>1)
{
$std = ($sqtotal / ($size-1)) ** 0.5; # ROOT
}
else{
$std = ($sqtotal / ($size)) ** 0.5; # ROOT{
}
return $std;
}


Quote
WE CALCULATE THE MEAN AND STD DEVIATION FOR NUMBER OF DAYS FOR EACH ID
Ex OF O/P:
ID 831 : Mean is X ,STDDEV IS Y
ID 388 : Mean is X ,STDDEV IS Y



(This post was edited by Tejas on Jun 10, 2014, 5:37 AM)


Laurent_R
Veteran / Moderator

Jun 10, 2014, 11:55 AM

Post #2 of 12 (13070 views)
Re: [Tejas] Calculating Avg For different combinations in hash. [In reply to] Can't Post


Code
my ($date,$amount,$etid) = (split /,/ , $line)[7,4,1];  
$Daily_Hash{$id}{$date} += $amount;

This is not going to work. If you had used the :

Code
use strict; 
use warnings;

(and you should always do it for any program having more than 1 line), the compiler would have told you about the mistake: you declare and initialize $etid, and you use $id next line.

Another slightly clumsy thing is that you call the average function twice: once in the main program and once within the stdev function. No big deal, it does not prevent things from working, but this is not very efficient and it also shows that your program is not carefully thought about. If you compute the mean in the main program, then pass its value to the stdev function where you need the mean.

The main problem with your post, however, is that you don't say what problem you are facing. I picked up a couple of issues with your code, but you should say what it does or what it does not do, or what errors you encounter, or how the result is different from what you expect, or what error message you get, well whatever is wrong.


Chris Charley
User

Jun 10, 2014, 3:49 PM

Post #3 of 12 (13019 views)
Re: [Tejas] Calculating Avg For different combinations in hash. [In reply to] Can't Post

If you can install modules, you could use Statistics::Descriptive;


Code
#!/usr/bin/perl 
use strict;
use warnings;
use Statistics::Descriptive;

my %by_date;
while (<DATA>) {
chomp;
my ($date,$amount,$id) = (split /,/)[7,4,1];
$by_date{$id}{$date} += $amount;
}

for my $id (keys %by_date) {
my $stat = Statistics::Descriptive::Full->new();
$stat->add_data(values $by_date{$id});
printf "ID %s: mean is %s stddev is %s\n",
$id, $stat->mean, $stat->standard_deviation;
}

__DATA__
744418169142,831,5,12,-.69,AURO,5MAU,20-DEC-13
744416737392,831,5,12,-15.22,AURO,5MAU,20-DEC-13
744417811392,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744418279342,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744415298192,831,5,12,-12.99,AURO,5MAU,20-DEC-13
744410571242,831,5,12,-5.89,AURO,5MAU,20-DEC-13
744411599242,831,5,12,-4.49,AURO,5MAU,20-DEC-13
744415584292,831,5,12,-16.48,AURO,5MAU,20-DEC-13
744410707242,831,5,12,-5.49,AURO,5MAU,20-DEC-13
744418511242,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744415075392,831,5,12,-10.99,AURO,5MAU,20-DEC-13
744416321142,831,5,12,-6.29,AURO,5MAU,20-DEC-13
744412254242,831,5,12,-9.99,AURO,5MAU,20-DEC-13
744415959342,831,5,12,-9.48,AURO,5MAU,20-DEC-13
744418642192,831,5,12,-8.99,AURO,5MAU,20-DEC-13
744419740342,831,5,12,-7.49,AURO,5MAU,20-DEC-13
744417666242,831,5,12,-10.99,AURO,5MAU,20-DEC-13
744417680242,831,5,12,-1.29,AURO,5MAU,20-DEC-13
744410020192,831,5,12,-1.99,AURO,5MAU,20-DEC-13
744417559292,831,5,12,-5.49,AURO,5MAU,20-DEC-13
744419275242,831,5,12,-1.99,AURO,5MAU,20-DEC-13
744419484242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744411455242,331,5,12,-.730,AURO,5MAU,20-DEC-13
744416321142,331,5,12,-6.29,AURO,5MAU,20-DEC-13
744412254242,331,5,12,-9.99,AURO,5MAU,20-DEC-13
744415959342,331,5,12,-9.48,AURO,5MAU,20-DEC-13
744418642192,331,5,12,-8.99,AURO,5MAU,20-DEC-13
744419740342,331,5,12,-7.49,AURO,5MAU,20-DEC-13
744417666242,331,5,12,-10.99,AURO,5MAU,20-DEC-13
744417680242,331,5,12,-1.29,AURO,5MAU,20-DEC-13
744410020192,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744417559292,331,5,12,-5.49,AURO,5MAU,20-DEC-13
744419275242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744419484242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744411455242,331,5,12,-.730,AURO,5MAU,20-DEC-13


This prints

Quote
ID 831: mean is -145.2 stddev is 0
ID 331: mean is -69.43 stddev is 0



Tejas
User

Jun 10, 2014, 7:23 PM

Post #4 of 12 (12871 views)
Re: [Laurent_R] Calculating Avg For different combinations in hash. [In reply to] Can't Post

Hi
Laurent
1. I have file which contains comma seperated values
And important values are split to etid,date and amount .

For each etid i need stddev and mean
-
And to print the total amount for each date i have date as a key in it.
The result should be just like charles's result .
Mean and stdhas to be calculated wrt number of days
Hope im clearnow

And the corect line of code is


Code
my ($date,$amount,$etid) = (split /,/ , $line)[7,4,1];   
$Daily_Hash{$etid}{$date} += $amount;

I would use my code as i dont have perms to install the module as charley suggested.


(This post was edited by Tejas on Jun 10, 2014, 7:25 PM)


Laurent_R
Veteran / Moderator

Jun 10, 2014, 11:31 PM

Post #5 of 12 (12709 views)
Re: [Tejas] Calculating Avg For different combinations in hash. [In reply to] Can't Post

Hi Tejas,

I perfectly understood what you are trying to do. What I asked you was rather this: in what respect does your program not do what you want? What is the error or the problem? Do you get an error message? Or is the result not what you expect?


Tejas
User

Jun 11, 2014, 12:35 AM

Post #6 of 12 (12666 views)
Re: [Laurent_R] Calculating Avg For different combinations in hash. [In reply to] Can't Post

Aaah, i got it now.

No Error at all.
Iam unable to deal with the multilevel hash to get the final result.
Iam able to get it when i have a single leve; hash

Code wrks well..As Such, No Problem with the code.

Just cant able to p-roduce the output the way i ant it to be with the multileve; hash

Thanks
Tejas


Tejas
User

Jun 11, 2014, 3:04 AM

Post #7 of 12 (12567 views)
Re: [Chris Charley] Calculating Avg For different combinations in hash. [In reply to] Can't Post

This doesnt really work for me and below is the error

Type of arg 1 to values must be hash (not hash element) at Test_Witm_Modules.pl line 24, near "})"

Thnaks
Tejas


Tejas
User

Jun 11, 2014, 3:10 AM

Post #8 of 12 (12561 views)
Re: [Chris Charley] Calculating Avg For different combinations in hash. [In reply to] Can't Post

This same code doesnt worlk for me , with same input and it says

Quote
Type of arg 1 to values must be hash (not hash element) at Test_Witm_Modules.pl line 23, near "})"
Execution of Test_Witm_Modules.pl aborted due to compilation errors.


So Changed to


Code
   $stat->add_data(values (%{$by_date{$id}}) ) ;

and it works :)


(This post was edited by Tejas on Jun 11, 2014, 3:16 AM)


Tejas
User

Jun 14, 2014, 10:57 AM

Post #9 of 12 (12318 views)
Re: [Laurent_R] Calculating Avg For different combinations in hash. [In reply to] Can't Post

Any Ideas Laurent.
Actually Thre is potential chance that my hash might have some more levels.

Thanks
Tejas


Laurent_R
Veteran / Moderator

Jun 15, 2014, 2:43 PM

Post #10 of 12 (11135 views)
Re: [Tejas] Calculating Avg For different combinations in hash. [In reply to] Can't Post

Hi,

sorry, I was away participating to the yearly French Perl Workshop organized by the French Perl Mongers, with Internet access only on mobile device, but no Perl compiler; I needed a compiler to try your code and also test my refactoring of it (predictably, I did not get everything right the first time I tried).

Also your message before the last said that it worked, I assumed that you had succeeded to make the required changes.

I first tried to stay relatively close to your code, but as I was going, I moved further and further away from it. I first removed the recursive call to the walk_hash sub, as I think it is not a very good idea in such a case: the code is so different depending on whether you are at the first level or at the second level that two nested loops make things much clearer in my humble opinion (I usually love recursion, it is just that I don't think it is right in this specific case). And, gradually moving away from your code, the last part I did (the calc subroutine) is ending to be entirely different from what you have. Even my formula for standard deviation calculation is different from yours. But you can easily change that if you think yours is better, I think that at least you will see how to use nested data structures.

This is what I have, which seems to be working:


Code
use strict;  
use warnings;

my %Daily_Hash;

while (my $line = <DATA>)
{
chomp $line;
my ($date,$amount,$etid) = (split /,/ , $line)[7,4,1];
$Daily_Hash{$etid}{$date} += $amount;
}

walk_hash(\%Daily_Hash);

my %Total_Items;

foreach my $key ( keys %Total_Items) {
my ($mean, $std_dev) = calc (@{$Total_Items{$key}});
print "Average and std-dev for product $key = $mean, $std_dev \n";
}


sub walk_hash {
my $h_ref = shift;
foreach my $key (keys %{$h_ref}) {
foreach my $key2 (keys %{$h_ref->{$key}}) {
push @{$Total_Items{$key}}, $h_ref->{$key}{$key2} ;
}
}
}

sub calc {
my @array = @_;
my $size = @_;
my $total = 0;
my $sq_total;
foreach my $value (@array) {
$total += $value;
$sq_total += $value ** 2;
}
my $average = $total/$size;
my $std_dev = sqrt ( $sq_total / $size - $average **2);
return $average, $std_dev;
}


__DATA__
744418169142,831,5,12,-.69,AURO,5MAU,20-DEC-13
744416737392,831,5,12,-15.22,AURO,5MAU,21-DEC-13
744417811392,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744418279342,831,5,12,-2.99,AURO,5MAU,22-DEC-13
744415298192,831,5,12,-12.99,AURO,5MAU,20-DEC-13
744410571242,831,5,12,-5.89,AURO,5MAU,20-DEC-13
744411599242,831,5,12,-4.49,AURO,5MAU,20-DEC-13
744415584292,831,5,12,-16.48,AURO,5MAU,20-DEC-13
744410707242,831,5,12,-5.49,AURO,5MAU,20-DEC-13
744418511242,831,5,12,-2.99,AURO,5MAU,20-DEC-13
744415075392,831,5,12,-10.99,AURO,5MAU,20-DEC-13
744416321142,831,5,12,-6.29,AURO,5MAU,20-DEC-13
744412254242,831,5,12,-9.99,AURO,5MAU,20-DEC-13
744415959342,831,5,12,-9.48,AURO,5MAU,20-DEC-13
744418642192,831,5,12,-8.99,AURO,5MAU,20-DEC-13
744419740342,831,5,12,-7.49,AURO,5MAU,20-DEC-13
744417666242,831,5,12,-10.99,AURO,5MAU,20-DEC-13
744417680242,831,5,12,-1.29,AURO,5MAU,20-DEC-13
744410020192,831,5,12,-1.99,AURO,5MAU,20-DEC-13
744417559292,831,5,12,-5.49,AURO,5MAU,20-DEC-13
744419275242,831,5,12,-1.99,AURO,5MAU,20-DEC-13
744419484242,331,5,12,-1.99,AURO,5MAU,22-DEC-13
744411455242,331,5,12,-.730,AURO,5MAU,20-DEC-13
744416321142,331,5,12,-6.29,AURO,5MAU,21-DEC-13
744412254242,331,5,12,-9.99,AURO,5MAU,21-DEC-13
744415959342,331,5,12,-9.48,AURO,5MAU,20-DEC-13
744418642192,331,5,12,-8.99,AURO,5MAU,20-DEC-13
744419740342,331,5,12,-7.49,AURO,5MAU,20-DEC-13
744417666242,331,5,12,-10.99,AURO,5MAU,20-DEC-13
744417680242,331,5,12,-1.29,AURO,5MAU,20-DEC-13
744410020192,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744417559292,331,5,12,-5.49,AURO,5MAU,20-DEC-13
744419275242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744419484242,331,5,12,-1.99,AURO,5MAU,20-DEC-13
744411455242,331,5,12,-.730,AURO,5MAU,20-DEC-13


First, please notice that I changed some of the dates in your input data in order to have something a bit more interesting (3 different dates for each product).

This is the output:

Code
$ perl  tejas.pl 
Average and std-dev for product 831 = -48.4, 55.7953659963501
Average and std-dev for product 331 = -23.1433333333333, 20.6518963347733


To help you understanding the data structures, this is what the %Total_Items hash looks like after it has been fully populated:

Code
0  HASH(0x60025ca40) 
331 => ARRAY(0x6005140e8)
0 '-16.28'
1 '-1.99'
2 '-51.16'
831 => ARRAY(0x6005141c0)
0 '-15.22'
1 '-2.99'
2 '-126.99'

It is a hash of arrays, with one array for each product (in this case, two) and an array element for each date.

If you want better formatting ot the output, you can change the relevant loop as follows:


Code
foreach my $key ( keys %Total_Items) { 
my ($mean, $std_dev) = calc (@{$Total_Items{$key}});
printf "Average and std-dev for product %i = %.2f and %.2f \n", $key, $mean, $std_dev;
}


Which woud print the following cleaner result:


Code
$ perl  tejas.pl 
Average and std-dev for product 831 = -48.40 and 55.80
Average and std-dev for product 331 = -23.14 and 20.65


I hope this helps.


(This post was edited by Laurent_R on Jun 15, 2014, 2:48 PM)


Tejas
User

Jun 16, 2014, 10:07 PM

Post #11 of 12 (9693 views)
Re: [Laurent_R] Calculating Avg For different combinations in hash. [In reply to] Can't Post

Is this Array of hashes.
By looking at it , it feels like an
ARRAY WITH HASH IN IT.

Can you please explain , how it is array of hasheds


Code
  push @{$Total_Items{$key}}, $h_ref->{$key}{$key2} ;


@{$Total_Items{$key}} # An Array with Hash value of $Total_Items{$key} in it ,Am i Correct?

How is it linked to the main hash %total_items ?

Thanks
Tejas


Laurent_R
Veteran / Moderator

Jun 16, 2014, 11:21 PM

Post #12 of 12 (9639 views)
Re: [Tejas] Calculating Avg For different combinations in hash. [In reply to] Can't Post

%Total_Items is a hash of arrays. The hash has two keys (331 and 831),and the values for each of these hash keys are references towards inner arrays of values.

Your original data structure, %h_ref (which I kept unchanged), by contrast is a hash of hashes: the values of the outside hash are references to inner hashes.


Code
push @{$Total_Items{$key}}, $h_ref->{$key}{$key2} ;


The right-hand side is just one element of your hash of hashes.

The left hand side just says to add the value to the inner array refered to by the value of the outer hash for the $Total_Items{$key} hash key. In other words, for each key (331 and 831) of the outer hash, the value is a reference to an array to which we add indivdual values. The @ before $Total_Items{$key} simply says that you want to to use the array referred to by the value of the hash.

I hope this is clear.


(This post was edited by Laurent_R on Jun 16, 2014, 11:23 PM)

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives