instability after increasing file-max and inode-max

[Adam McKenna]

Hello, I have a system here that was up for over 100 days, using kernel
2.2.15.  We are running Oracle 8i on it.  The middle of last week, we ran out
of file handles, so I increased /proc/sys/fs/file-max to 16384 and
/proc/sys/fs/inode-max to 16384.  This appears to have caused instability, as
the box has crashed twice since last friday (once on Saturday morning, and
once this morning), at exactly 7:03 am.

Here are the last few syslog messages before the crash:

on Saturday:

Sep 30 06:53:54 burger ctlinnd[9454]: Reading config from /etc/news/inn.conf
Sep 30 06:53:54 burger innd: s
Sep 30 06:55:01 burger /USR/SBIN/CRON[9581]: (list) CMD ([ -x /usr/bin/python -a -f /usr/lib/mailman/cron/gate_news ] && /usr/bin/python /usr/lib/mailman/cron/g ate_news)
Sep 30 07:03:04 burger syslogd 1.3-3#33: restart.

Today:

Oct  4 06:51:54 burger ctlinnd[4101]: Reading config from /etc/news/inn.conf
Oct  4 06:51:54 burger innd: s
Oct  4 06:55:01 burger /USR/SBIN/CRON[4216]: (list) CMD ([ -x /usr/bin/python -a -f /usr/lib/mailman/cron/gate_news ] && /usr/bin/python /usr/lib/mailman/cron/g ate_news)
Oct  4 07:03:03 burger syslogd 1.3-3#33: restart.

dmesg:

Linux version 2.2.15 (herbert@gondor) (gcc version 2.95.2 20000313 (Debian GNU/Linux)) #1 Thu Jun 1 10:47:16 EST 2000
Detected 596929811 Hz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 1189.48 BogoMIPS
Memory: 516308k/524224k available (1740k kernel code, 416k reserved, 5620k data, 140k init)
Dentry hash table entries: 65536 (order 7, 512k)
Buffer cache hash table entries: 524288 (order 9, 2048k)
Page cache hash table entries: 131072 (order 7, 512k)
VFS: Diskquotas version dquot_6.4.0 initialized
CPU: Intel Pentium III (Coppermine) stepping 01
Checking 386/387 coupling... OK, FPU using exception 16 error reporting.
Checking 'hlt' instruction... OK.
Checking for popad bug... OK.
POSIX conformance testing by UNIFIX
mtrr: v1.35a (19990819) Richard Gooch ([e-mail])
PCI: PCI BIOS revision 2.10 entry at 0xfdab0
PCI: Using configuration type 1
PCI: Probing PCI hardware
Linux NET4.0 for Linux 2.2
Based upon Swansea University Computer Society NET3.039
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
TCP: Hash tables configured (ehash 524288 bhash 65536)
Starting kswapd v 1.5 
Detected PS/2 Mouse Port.
pty: 256 Unix98 ptys configured
apm: BIOS not found.
Real Time Clock Driver v1.09
RAM disk driver initialized:  16 RAM disks of 4096K size
loop: registered device at major 7
PIIX4: IDE controller on PCI bus 00 dev 91
PIIX4: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0x2860-0x2867, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0x2868-0x286f, BIOS settings: hdc:DMA, hdd:pio
keyboard: Too many NACKs -- noisy kbd cable?
keyboard: Too many NACKs -- noisy kbd cable?
hda: QUANTUM FIREBALLP LM30, ATA DISK drive
hdc: CD-540E, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
hda: QUANTUM FIREBALLP LM30, 28629MB w/1900kB Cache, CHS=3649/255/63
hdc: ATAPI 40X CD-ROM drive, 128kB Cache
Uniform CDROM driver Revision: 2.56
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
md driver 0.36.6 MAX_MD_DEV=4, MAX_REAL=8
(scsi0) <Adaptec AIC-7896/7 Ultra2 SCSI host adapter> found at PCI 0/12/0
(scsi0) Wide Channel A, SCSI ID=7, 32/255 SCBs
(scsi0) Downloading sequencer code... 392 instructions downloaded
(scsi1) <Adaptec AIC-7896/7 Ultra2 SCSI host adapter> found at PCI 0/12/1
(scsi1) Wide Channel B, SCSI ID=7, 32/255 SCBs
(scsi1) Downloading sequencer code... 392 instructions downloaded
scsi: <fdomain> Detection failed (no card)
NCR53c406a: no available ports found
sym53c416.c: Version 1.0.0
Failed initialization of WD-7000 SCSI card!
IBM MCA SCSI: No Microchannel-bus support present -> Aborting.
megaraid: v107 (December 22, 1999)
aec671x_detect: 
3w-xxxx: tw_findcards(): No cards found.
scsi0 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 5.1.29/3.2.4
       <Adaptec AIC-7896/7 Ultra2 SCSI host adapter>
scsi1 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 5.1.29/3.2.4
       <Adaptec AIC-7896/7 Ultra2 SCSI host adapter>
scsi : 2 hosts.
scsi : detected total.
Partition check:
 hda: hda1 hda2 hda3 hda4 < hda5 hda6 hda7 hda8 hda9 hda10 hda11 >
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 140k freed
NET4: Unix domain sockets 1.0 for Linux NET4.0.
Adding Swap: 136544k swap-space (priority -1)
Adding Swap: 1028120k swap-space (priority -2)
eepro100.c:v1.09j-t 9/29/99 Donald Becker http://cesdis.gsfc.nasa.gov/linux/drivers/eepro100.html
eepro100.c: $Revision: 1.20.2.3 $ 2000/03/02 Modified by Andrey V. Savochkin <[e-mail]> and others
eth0: Intel PCI EtherExpress Pro100 at 0xe00aa000, 00:D0:B7:65:DF:BE, IRQ 10.
  Receiver lock-up bug exists -- enabling work-around.
  Board assembly 000000-000, Physical connectors present: RJ45
  Primary interface chip i82555 PHY #1.
  General self-test: passed.
  Serial sub-system self-test: passed.
  Internal registers self-test: passed.
  ROM checksum self-test: passed (0x04f4518b).
Serial driver version 4.27 with no serial options enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A

If anyone could give me a clue here I'd appreciate it.  I'm going to decrease
the parameters I changed back to 8192, hopefully that will help things.

--Adam

-- 
Adam McKenna <[e-mail]> | "No matter how much it changes, 
http://flounder.net/publickey.html   |  technology's just a bunch of wires 
GPG: 17A4 11F7 5E7E C2E7 08AA        |  connected to a bunch of other wires."
     38B0 05D0 8BF7 2C6D 110A        |  Joe Rogan, _NewsRadio_