Apr 2, 2009, 12:19 PM
Post #2 of 2
Re: [young_matthewd] alternatives to File::Tail for performance
[In reply to]
You can tail the log yourself. Take a look at Recipe 8.18 in the Perl Cookbook http://www.digital-deception.net/books/O%27Reilly%20Perl%20Cookbook.pdf (PDF) for one possible implementation. I haven't benchmarked this function but it's probably faster the File::Tail.
However, before you write your own tailer, better find out if the problem is actually File::Tail or is it what you are doing in your monitor program with the tailed lines? Try breaking this down by trowing away the data right after it's read. You might get better performance using File::Tail by filtering the tail with the object of processing only interesting messages.
Finally, depending on how fast your log is growing, 5% isn't necessarily excessive. For example, if I execute:
On one of my honking fast HP servers, it takes 5-6% of one of the cpus. The bottom line is that monitoring busy processes can be expensive. There is always a trade off between how much you log/how much you monitor and system performance.
$ cat <some huge log file> | tail -f
Consider lowering the log verbosity of your servers. Do you really need all of this data?