CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Handlng Large hash

 



vk.gamer
Novice

Dec 12, 2007, 8:31 AM

Post #1 of 5 (763 views)
Handlng Large hash Can't Post

Hi,
I'm using a large hash which has about 300,000 keys with their corresponding values.

This program, I'm planning to run as a lookup server in IO:Socket. and other perl and java programs will use this socket for lookup.

Now, Can I force the first mention perl prgm with large hash to load all of its hash keys and values to memory so that lookup will be faster?

How to do that?

any help is greatly appreciable.


KevinR
Veteran


Dec 12, 2007, 9:06 AM

Post #2 of 5 (757 views)
Re: [vk.gamer] Handlng Large hash [In reply to] Can't Post

Usually perl data is not persistent. A perl program gets compiled and runs and ends and all the data the program used while executing is lost unless it was written to a file or database.

There is a module called Storable, that allows for persistent perl data. Might be the ticket for you.

Or you can possibly keep your perl program running continuously so the hash is not lost when the program ends. The simple way is:


Code
while(1) { 
here your program runs untill you tell it not to
}


There are also modules for running perl programs as deamons (I have never used them). And mod_perl is another option and there are probably other options I am not aware of.
-------------------------------------------------


vk.gamer
Novice

Dec 12, 2007, 9:13 PM

Post #3 of 5 (753 views)
Re: [KevinR] Handlng Large hash [In reply to] Can't Post

yes. you're right. But I'm storing data to a dbm file. and then reading the file when needed. Here, I want to speed up the lookup operation by loading entire hash data into memory. how do i do that?


Kanji
User

Dec 12, 2007, 10:31 PM

Post #4 of 5 (752 views)
Re: [vk.gamer] Handlng Large hash [In reply to] Can't Post

This may work...


Code
my %in_memory; 

{
tie my %in_dbm, "DB_File", "your.dbm";
%in_memory = %in_dbm;
}


Depending on the type of DBM you're using, you may need to replace DB_File with a more appropriate module (GDBM_File, NDBM_File, ODBM_File and SDBM_File), and you probably also want to add some error checking.

The downside to this method, however, is that if you update your DBM, your script will not see those changes unless it re-loads the hash, which means more coding to detect when the DBM has been updated or restarting your program.
--k.


vk.gamer
Novice

Dec 13, 2007, 2:50 AM

Post #5 of 5 (747 views)
Re: [Kanji] Handlng Large hash [In reply to] Can't Post

Yes. This fastens the lookup to some extent but it faster than this.

Any more tips on how to do faster lookup on large hashes??

Any better alternative to IO:Socket (to put the lookup server) ?


(This post was edited by vk.gamer on Dec 13, 2007, 2:52 AM)

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives