file handles - who ate them?!

[Dave Gilbert]

Hi,
   I had a bad case of running out of file handles over the weekend and 
wondered:

    1) If there are any tools for trying to track down what ate them 
all.  Looking at the state of the system today I see 2541 
(allocated)/763 (used)/ 8192 (max).  I guess I'd like to know when it 
hit the peak and what the state of the system was around that time (to 
find out if it was a particular cron job etc). I can see what is using 
them now - but that isn't too much use.

   2) Why is there a file-nr-max? Is it the case that the kernel 
allocates file handles as it needs them (i.e. the 2541 above) - but 
never frees back?  Is it just as a stop to stop rogue programs taking 
all RAM? (i.e. why don't we just let the allocated amount grow as 
needed). Is there a reason I shouldn't just set this to a huge figure?

   3) Is it possible to change what happens on hitting the max 
condition? - in someways for me it would have been better if it had 
panic'd and rebooted rather than sitting there and sulking not being 
able to open files. (Getting it to log a dump of open files at this time 
would be the ideal).

(2.4.16, x86)

Dave