CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
Breaking up a csv file based on column

 



joseph4325
Novice

Jul 23, 2013, 6:24 PM

Post #1 of 7 (558 views)
Breaking up a csv file based on column Can't Post

I'm trying to take a csv file and break it up so that i can get an average of one of the fields.

Example Input file:
red,5
red,10
red,0
blue,6
blue,1
blue,2

So i would want the output to be:
red, 5
blue,3

(assume first column can vary)

My thought was to somehow make an array out of each "type" of the first column so that i can do a sum of the array elements divided by the number of elements in each array, but i'm not sure how to do this. Any help is appreciated.


BillKSmith
Veteran

Jul 24, 2013, 5:43 AM

Post #2 of 7 (545 views)
Re: [joseph4325] Breaking up a csv file based on column [In reply to] Can't Post

You need a complex data structure. You could use either a hash-of-hashes or a hash-of-arrays (refer: perldoc perldsc) I prefer the hash-of-hashes.


Code
use strict; 
use warnings;
my %in_hash;
while (<DATA>) {
chomp;
my ($type, $value) = split /,/;
$in_hash{$type}{count}++;
$in_hash{$type}{value} += $value;
}
while (my($type, $data) = each %in_hash) {
print $type, " ", $data->{value}/$data->{count}, "\n";
}
__DATA__
red,5
red,10
red,0
blue,6
blue,1
blue,2


OUTPUT:

Code
blue 3 
red 5

Good Luck,
Bill


joseph4325
Novice

Jul 24, 2013, 9:45 AM

Post #3 of 7 (541 views)
Re: [BillKSmith] Breaking up a csv file based on column [In reply to] Can't Post

This works great.

If i wanted to read the data as an array, like:
my @fhdata = <FH>;
(so that i can open file, store in array, close)
how would i make the while loop read from that?


BillKSmith
Veteran

Jul 24, 2013, 10:12 AM

Post #4 of 7 (538 views)
Re: [joseph4325] Breaking up a csv file based on column [In reply to] Can't Post

That is a very bad idea. I will not tell you how to do it. A much better way to read your data from a file is to replace the built-in filehandle DATA with your own filehandle. (Of course, you have to open the file to get the filehandle.)
Good Luck,
Bill


joseph4325
Novice

Jul 24, 2013, 10:27 AM

Post #5 of 7 (536 views)
Re: [BillKSmith] Breaking up a csv file based on column [In reply to] Can't Post

Ok, well i guess that shoulda been my question is...as it is...i'm opening the file handle, reading it on the while loop, then doing all the operations it needs, then closing.

Why is that preferred to opening the file, storing all the data in an array, then closing the file, and working with the data that way?


Laurent_R
Veteran / Moderator

Jul 24, 2013, 10:45 AM

Post #6 of 7 (535 views)
Re: [joseph4325] Breaking up a csv file based on column [In reply to] Can't Post

Because if you do that, you are uselessly using a large chunk of memory to store your file, while the whole point of a file handle is to iterate over the file line by line, to avoid using memory. It is also less efficient, because you are reading each piece of data twice, although the difference may be quite small. Besides, it does not scale well, if your file grows big, at one point or another your program will die for lack of memory.


joseph4325
Novice

Jul 24, 2013, 11:18 AM

Post #7 of 7 (534 views)
Re: [Laurent_R] Breaking up a csv file based on column [In reply to] Can't Post

Ok i gotcha. Thanks for the learnin'

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives