CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
Search Posts SEARCH
Who's Online WHO'S
Log in LOG

Home: Perl Programming Help: Intermediate:
Handlng Large hash



Dec 12, 2007, 8:31 AM

Post #1 of 5 (783 views)
Handlng Large hash Can't Post

I'm using a large hash which has about 300,000 keys with their corresponding values.

This program, I'm planning to run as a lookup server in IO:Socket. and other perl and java programs will use this socket for lookup.

Now, Can I force the first mention perl prgm with large hash to load all of its hash keys and values to memory so that lookup will be faster?

How to do that?

any help is greatly appreciable.


Dec 12, 2007, 9:06 AM

Post #2 of 5 (777 views)
Re: [vk.gamer] Handlng Large hash [In reply to] Can't Post

Usually perl data is not persistent. A perl program gets compiled and runs and ends and all the data the program used while executing is lost unless it was written to a file or database.

There is a module called Storable, that allows for persistent perl data. Might be the ticket for you.

Or you can possibly keep your perl program running continuously so the hash is not lost when the program ends. The simple way is:

while(1) { 
here your program runs untill you tell it not to

There are also modules for running perl programs as deamons (I have never used them). And mod_perl is another option and there are probably other options I am not aware of.


Dec 12, 2007, 9:13 PM

Post #3 of 5 (773 views)
Re: [KevinR] Handlng Large hash [In reply to] Can't Post

yes. you're right. But I'm storing data to a dbm file. and then reading the file when needed. Here, I want to speed up the lookup operation by loading entire hash data into memory. how do i do that?


Dec 12, 2007, 10:31 PM

Post #4 of 5 (772 views)
Re: [vk.gamer] Handlng Large hash [In reply to] Can't Post

This may work...

my %in_memory; 

tie my %in_dbm, "DB_File", "your.dbm";
%in_memory = %in_dbm;

Depending on the type of DBM you're using, you may need to replace DB_File with a more appropriate module (GDBM_File, NDBM_File, ODBM_File and SDBM_File), and you probably also want to add some error checking.

The downside to this method, however, is that if you update your DBM, your script will not see those changes unless it re-loads the hash, which means more coding to detect when the DBM has been updated or restarting your program.


Dec 13, 2007, 2:50 AM

Post #5 of 5 (767 views)
Re: [Kanji] Handlng Large hash [In reply to] Can't Post

Yes. This fastens the lookup to some extent but it faster than this.

Any more tips on how to do faster lookup on large hashes??

Any better alternative to IO:Socket (to put the lookup server) ?

(This post was edited by vk.gamer on Dec 13, 2007, 2:52 AM)


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives