Dec 28, 2000, 7:21 AM
Post #1 of 2
I’m building a database application and seek your guidance to get up and running. I’m particularly interested in scalability and performance. The narrative below is long but is easy to understand and will give you a clear idea of what I want to achieve. As always, thank you for your help.
The file structure I propose to use is set out below. I use horse betting for demonstration purposes. The HorseID is the common thread between all files:
1) Index File (text or dat file) (only one index file in the application)
HorseID1 | Horse_name_1
HorseID2 | Horse_name_2
HorseIDn | Horse_name_n
2) HorseID1_Description File (text or dat file) (there will be a file for each horse – unlimited no of horses)
Age | Sire | Dam | Races Ran | Races Won | Cumulative Winnings
3) HorseID1_Betting File (text or dat file) (there will be a file for each horse – unlimited no of horses and unlimited no of bets on each horse) (| & ; are used to split records)
Punter Name; Amount Bet; Amount Won | Punter Name; Amount Bet; Amount Won | Punter Name; Amount Bet; Amount Won
HOW IT WORKS
Given the huge number of horses, I propose that PERL would search the index file for the name of the horse, then determine the relevant HorseID and then locate the relevant Description file & Betting File. PERL would then read the contents of these files and output the contents to the browser.
When a new horse is added, a new HorseID is created in the index file and a new Description file and Betting file are created for the horse.
New bets are added to the end of the relevant Betting file. Horse races and wins are recorded in the Description file for a horse by essentially rewriting the entire Description file of that horse.
a) Can a web server support the potentially huge number of Description and Betting files (text or dat types) or would it crash, say if there were 100,000 horses i.e. 200,000 text or dat files + 1 index file?
b) Is the idea of searching the index file first a good one, would it be fast enough if there were 100,000 horses?
c) What is the weakness in using text or dat files and how would dedicated database applications like SQL and Coldfusion be better and could they cope with the unlimited growth in the size of the Betting file? What do you recommend that I do? Is the text / dat file approach practical or eventually doomed?
d) Out of interest, what is the difference between a text and a dat file and what amount of data can they hold – would the index file ever be prevented from adding a new horse?
Thanks again for reading this far. This forum is an invaluable assistance to would be developers and I would like to extend my appreciation to the promoters and participants.