I feel kind of foolish after bragging how stable our Linux systems
are under SMP to be reporting this, but here goes anyway. One of our SMP
systems running Linux 2.0.32 is reporting several oops and general
protection errors in the past couple of days (none of these has killed the
machine) and I wanted to report them here so that they can hopefully help
resolve the stability issues that others are reporting. What follows is
first, the .config file from the kernel running on this system, followed
by the syslog messages concerning the oops and faults and entries from the
System.map file that coorespond to the EIP and Call Trace numbers. If more
information is needed, please let me know and I will try to provide it.
Here is some information about the system:
$ uname -a
Linux log2 2.0.32 #1 Mon Nov 24 00:57:34 CST 1997 i686
$ ls -l /lib
total 2014
-rwxr-xr-x 2 root root 21367 May 11 1996 ld-linux.so.1*
-rwxr-xr-x 2 root root 21367 May 11 1996 ld-linux.so.1.7.14*
-rwxr-xr-x 1 root root 20722 Mar 25 1996 ld-linux.so.1.7.3*
-rwxr-xr-x 2 root root 24580 May 11 1996 ld.so*
-rwxr-xr-x 1 root root 20484 Aug 15 1995 ld.so-1.7.3*
-rwxr-xr-x 2 root root 24580 May 11 1996 ld.so.1.7.14*
lrwxrwxrwx 1 root root 13 Aug 13 20:31 libc.so.4 -> libc.so.4.7.
5*
-rwxr-xr-x 1 bin bin 634880 Oct 15 1995 libc.so.4.7.5*
lrwxrwxrwx 1 root root 14 Aug 13 20:31 libc.so.5 -> libc.so.5.3.
12*
-rwxr-xr-x 1 root root 562683 Mar 25 1996 libc.so.5.0.9*
-rwxr-xr-x 1 bin bin 583795 Apr 25 1996 libc.so.5.3.12*
lrwxrwxrwx 1 root root 18 Aug 13 20:31 libcurses.so.0 -> libcurs
es.so.0.1.2*
-rwxr-xr-x 1 bin bin 49152 Dec 13 1994 libcurses.so.0.1.2*
lrwxrwxrwx 1 root root 18 Aug 13 20:31 libcurses.so.1 -> libcurs
es.so.1.0.0*
-rwxr-xr-x 1 root root 52641 Mar 25 1996 libcurses.so.1.0.0*
lrwxrwxrwx 1 root root 10 Aug 13 20:31 libdl.so -> libdl.so.1*
lrwxrwxrwx 1 root root 15 Aug 13 20:31 libdl.so.1 -> libdl.so.1.
7.14*
-rwxr-xr-x 1 root root 6983 May 11 1996 libdl.so.1.7.14*
-rwxr-xr-x 1 root root 6843 Mar 25 1996 libdl.so.1.7.3*
lrwxrwxrwx 1 root root 14 Aug 13 20:31 libe2fs.so.1 -> libe2fs.s
o.1.0*
-rwxr-xr-x 1 root root 31880 Mar 25 1996 libe2fs.so.1.0*
lrwxrwxrwx 1 root root 13 Aug 13 20:31 libe2p.so.1 -> libe2p.so.
1.0*
-rwxr-xr-x 1 root root 6580 Mar 25 1996 libe2p.so.1.0*
lrwxrwxrwx 1 root root 12 Aug 13 20:31 libet.so.1 -> libet.so.1.
0*
-rwxr-xr-x 1 root root 4068 Mar 25 1996 libet.so.1.0*
lrwxrwxrwx 1 root root 14 Aug 13 20:31 libm.so.4 -> libm.so.4.6.
27*
-rwxr-xr-x 1 bin bin 110592 Dec 13 1994 libm.so.4.6.27*
lrwxrwxrwx 1 root root 13 Aug 13 20:31 libm.so.5 -> libm.so.5.0.
6*
-rwxr-xr-x 1 root root 35942 Mar 25 1996 libm.so.5.0.0*
-rwxr-xr-x 1 bin bin 36156 Apr 25 1996 libm.so.5.0.6*
lrwxrwxrwx 1 root root 12 Aug 13 20:31 libss.so.1 -> libss.so.1.
0*
-rwxr-xr-x 1 root root 12912 Mar 25 1996 libss.so.1.0*
lrwxrwxrwx 1 root root 19 Aug 13 20:31 libtermcap.so.2 -> libter
mcap.so.2.0.0*
-rwxr-xr-x 1 root root 11360 Mar 25 1996 libtermcap.so.2.0.0*
drwxr-xr-x 6 root root 2048 Dec 2 02:33 modules/
$ lsmod
Module: #pages: Used by:
st 6 0
raid0 1 1 (autoclean)
$ cat /proc/cpuinfo
processor : 0
cpu : 686
model : Pentium Pro
vendor_id : GenuineIntel
stepping : 9
fdiv_bug : no
hlt_bug : no
f00f_bug : no
fpu : yes
fpu_exception : yes
cpuid : yes
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmov
bogomips : 199.07
processor : 1
cpu : 686
model : Pentium Pro
vendor_id : GenuineIntel
stepping : 9
fdiv_bug : no
hlt_bug : no
f00f_bug : no
fpu : yes
fpu_exception : yes
cpuid : yes
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmov
bogomips : 199.07
$ cat /proc/smp
CPUS: 2
SUM P1 P0
0: 20037001 20037001 0 timer
1: 275 275 0 keyboard
2: 0 0 0 cascade
9: 1057221 1057221 0 + BusLogic BT-958
10: 8828056 8828056 0 3c509
11: 2065312 2065312 0 + BusLogic BT-958
13: 9544195 2543140 7001055 + IPI
LCK: 4048370902 3134554989 913815913 spins from int
LCK: 76936854 4181931907 189972243 spins from syscall
LCK: 1345338049 514131629 831206420 spins from sysidle
IDLE 34734667 17670982 17063685 idle ticks
IPI: 9544195 received
Anyhow, here is the .config file for this kernel (version 2.0.32)
and the syslog/system map info for the faults:
#
# Automatically generated by make menuconfig: don't edit
#
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODVERSIONS=y
CONFIG_KERNELD=y
#
# General setup
#
# CONFIG_MATH_EMULATION is not set
CONFIG_NET=y
# CONFIG_MAX_16M is not set
CONFIG_PCI=y
# CONFIG_PCI_OPTIMIZE is not set
CONFIG_SYSVIPC=y
CONFIG_BINFMT_AOUT=y
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_JAVA is not set
CONFIG_KERNEL_ELF=y
CONFIG_M586=y
#
# Floppy, IDE, and other block devices
#
CONFIG_BLK_DEV_FD=y
CONFIG_BLK_DEV_IDE=y
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_BLK_DEV_IDE_PCMCIA is not set
CONFIG_BLK_DEV_CMD640=y
# CONFIG_BLK_DEV_CMD640_ENHANCED is not set
CONFIG_BLK_DEV_RZ1000=y
CONFIG_BLK_DEV_TRITON=y
# CONFIG_IDE_CHIPSETS is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_MD=y
CONFIG_MD_LINEAR=m
CONFIG_MD_STRIPED=m
CONFIG_BLK_DEV_RAM=m
# CONFIG_BLK_DEV_XD is not set
# CONFIG_BLK_DEV_HD is not set
#
# Networking options
#
CONFIG_FIREWALL=y
CONFIG_NET_ALIAS=y
CONFIG_INET=y
# CONFIG_IP_FORWARD is not set
CONFIG_IP_MULTICAST=y
CONFIG_SYN_COOKIES=y
CONFIG_RST_COOKIES=y
CONFIG_IP_FIREWALL=y
CONFIG_IP_FIREWALL_VERBOSE=y
# CONFIG_IP_MASQUERADE is not set
# CONFIG_IP_TRANSPARENT_PROXY is not set
CONFIG_IP_ALWAYS_DEFRAG=y
CONFIG_IP_ACCT=y
# CONFIG_IP_ROUTER is not set
CONFIG_NET_IPIP=m
# CONFIG_IP_MROUTE is not set
CONFIG_IP_ALIAS=m
# CONFIG_INET_PCTCP is not set
CONFIG_INET_RARP=m
CONFIG_NO_PATH_MTU_DISCOVERY=y
CONFIG_IP_NOSR=y
CONFIG_SKB_LARGE=y
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_AX25 is not set
# CONFIG_BRIDGE is not set
# CONFIG_NETLINK is not set
#
# SCSI support
#
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=m
CONFIG_BLK_DEV_SR=m
CONFIG_CHR_DEV_SG=m
# CONFIG_SCSI_MULTI_LUN is not set
CONFIG_SCSI_CONSTANTS=y
#
# SCSI low-level drivers
#
# CONFIG_SCSI_7000FASST is not set
# CONFIG_SCSI_AHA152X is not set
# CONFIG_SCSI_AHA1542 is not set
# CONFIG_SCSI_AHA1740 is not set
CONFIG_SCSI_AIC7XXX=y
CONFIG_AIC7XXX_TAGGED_QUEUEING=y
# CONFIG_OVERRIDE_CMDS is not set
# CONFIG_AIC7XXX_PAGE_ENABLE is not set
CONFIG_AIC7XXX_PROC_STATS=y
CONFIG_AIC7XXX_RESET_DELAY=15
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_IN2000 is not set
# CONFIG_SCSI_AM53C974 is not set
CONFIG_SCSI_BUSLOGIC=y
CONFIG_SCSI_OMIT_FLASHPOINT=y
# CONFIG_SCSI_DTC3280 is not set
# CONFIG_SCSI_EATA_DMA is not set
# CONFIG_SCSI_EATA_PIO is not set
CONFIG_SCSI_EATA=y
CONFIG_SCSI_EATA_TAGGED_QUEUE=y
CONFIG_SCSI_EATA_LINKED_COMMANDS=y
CONFIG_SCSI_EATA_MAX_TAGS=16
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GENERIC_NCR5380 is not set
# CONFIG_SCSI_NCR53C406A is not set
# CONFIG_SCSI_NCR53C7xx is not set
# CONFIG_SCSI_NCR53C8XX is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_PAS16 is not set
# CONFIG_SCSI_QLOGIC_FAS is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_SEAGATE is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_T128 is not set
# CONFIG_SCSI_U14_34F is not set
# CONFIG_SCSI_ULTRASTOR is not set
# CONFIG_SCSI_GDTH is not set
#
# Network device support
#
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_EQUALIZER=m
# CONFIG_DLCI is not set
CONFIG_PLIP=m
CONFIG_PPP=m
CONFIG_SLIP=m
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLIP_SMART=y
# CONFIG_SLIP_MODE_SLIP6 is not set
# CONFIG_NET_RADIO is not set
CONFIG_NET_ETHERNET=y
CONFIG_NET_VENDOR_3COM=y
# CONFIG_EL1 is not set
# CONFIG_EL2 is not set
# CONFIG_ELPLUS is not set
# CONFIG_EL16 is not set
CONFIG_EL3=y
CONFIG_VORTEX=m
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_ISA is not set
CONFIG_NET_EISA=y
# CONFIG_AC3200 is not set
# CONFIG_APRICOT is not set
# CONFIG_EEXPRESS_PRO100B is not set
CONFIG_DE4X5=m
CONFIG_DEC_ELCP=m
# CONFIG_DGRS is not set
# CONFIG_ZNET is not set
# CONFIG_NET_POCKET is not set
# CONFIG_TR is not set
# CONFIG_FDDI is not set
# CONFIG_ARCNET is not set
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# CD-ROM drivers (not for SCSI or IDE/ATAPI drives)
#
# CONFIG_CD_NO_IDESCSI is not set
#
# Filesystems
#
# CONFIG_QUOTA is not set
CONFIG_MINIX_FS=y
# CONFIG_EXT_FS is not set
CONFIG_EXT2_FS=y
# CONFIG_XIA_FS is not set
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
# CONFIG_UMSDOS_FS is not set
CONFIG_PROC_FS=y
# CONFIG_NFS_FS is not set
# CONFIG_SMB_FS is not set
CONFIG_ISO9660_FS=m
# CONFIG_HPFS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_UFS_FS is not set
#
# Character devices
#
CONFIG_SERIAL=y
# CONFIG_DIGI is not set
# CONFIG_CYCLADES is not set
# CONFIG_STALDRV is not set
# CONFIG_RISCOM8 is not set
# CONFIG_PRINTER is not set
# CONFIG_SPECIALIX is not set
# CONFIG_MOUSE is not set
# CONFIG_UMISC is not set
# CONFIG_QIC02_TAPE is not set
# CONFIG_FTAPE is not set
# CONFIG_APM is not set
# CONFIG_WATCHDOG is not set
# CONFIG_RTC is not set
#
# Sound
#
# CONFIG_SOUND is not set
#
# Kernel hacking
#
# CONFIG_PROFILE is not set
And finally, here are the 4 reported faults and information I took
from the System.map file regarding the addresses reported:
syslog output:
-------------
Dec 1 18:44:31 log2 kernel: Unable to handle kernel paging request at virtual a
ddress d0000004
Dec 1 18:44:31 log2 kernel: current->tss.cr3 = 0e875000,
Dec 1 18:44:31 log2 kernel: *pde = 00000000
Dec 1 18:44:31 log2 kernel: Oops: 0002
Dec 1 18:44:31 log2 kernel: CPU: 1
Dec 1 18:44:31 log2 kernel: EIP: 0010:[<001175a6>]
Dec 1 18:44:31 log2 kernel: EFLAGS: 00010006
Dec 1 18:44:31 log2 kernel: eax: 03fe79b0 ebx: 00000000 ecx: 10000000 edx
: 00000000
Dec 1 18:44:31 log2 kernel: esi: 00000212 edi: 0000012c ebp: 0e5e2ed0 esp
: 0e5e2ec8
Dec 1 18:44:31 log2 kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 001
8
Dec 1 18:44:31 log2 kernel: Process hostcheck (pid: 281, process nr: 20, stackp
age=0e5e2000)
Dec 1 18:44:31 log2 kernel: Stack: 00000001 03fe7810 00fd1444 00154ec3 03fe79b0
00fd1490 03fe7810 9b64f15b
Dec 1 18:44:31 log2 kernel: 001542e5 03fe7810 00000001 0000012c 03fe7810
001d9cdc 00fd1490 00000000
Dec 1 18:44:31 log2 kernel: 03fe7810 03fe78d0 00000000 00000003 00007d78
03fe79dc 001d1a78 001d9cdc
Dec 1 18:44:31 log2 kernel: Call Trace: [<00154ec3>] [<001542e5>] [<0014fe60>]
[<0015a222>] [<0013f2bc>] [<0013f4f5>] [<0012a3
a2>]
Dec 1 18:44:31 log2 kernel: [<0012a423>] [<10805000>] [<0012a4a2>] [<001
0ad8b>]
Dec 1 18:44:31 log2 kernel: Code: 89 51 04 85 d2 74 07 bb 01 00 00 00 89 0a c7
40 04 00 00 00
-------------------------------------------------------------------------------
Here are portions of the System.map that coorespond to EIP and Call Trace
numbers:
EIP: 001175a6 00117590 T del_timer
001175d0 T tqueue_bh
Call Trace: 00154ec3 00154ea8 T tcp_reset_xmit_timer
00154f44 t tcp_retransmit_time
001542e5 00154040 T tcp_send_fin
001542f0 T tcp_send_synack
0014fe60 0014fd3c t tcp_close
0014ffa0 t wait_for_connect
0015a222 0015a1ac t inet_release
0015a22c t inet_bind
0013f2bc 0013f260 T sock_release
0013f2fc t sock_lseek
0013f4f5 0013f4d0 t sock_close
0013f4fc t sock_fasync
0012a3a2 0012a384 T __fput
0012a3c8 T close_fp
0012a423 0012a3c8 T close_fp
0012a430 T sys_close
10805000 ??
0012a4a2 0012a430 T sys_close
0012a4ac T sys_vhangup
0010ad8b 0010ac70 T system_call
0010add0 T ret_from_sys_call
===============================================================================
Dec 3 15:23:36 log2 kernel: Unable to handle kernel paging request at virtual a
ddress f2000028
Dec 3 15:23:36 log2 kernel: current->tss.cr3 = 0d83b000,
Dec 3 15:23:36 log2 kernel: *pde = 00000000
Dec 3 15:23:36 log2 kernel: Oops: 0002
Dec 3 15:23:36 log2 kernel: CPU: 0
Dec 3 15:23:36 log2 kernel: EIP: 0010:[<001175b2>]
Dec 3 15:23:36 log2 kernel: EFLAGS: 00010006
Dec 3 15:23:36 log2 kernel: eax: 0394cf20 ebx: 00000001 ecx: 00000001 edx
: 32000028
Dec 3 15:23:36 log2 kernel: esi: 00000206 edi: 00000001 ebp: 0394cf00 esp
: 0394cef8
Dec 3 15:23:36 log2 kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 001
8
Dec 3 15:23:36 log2 kernel: Process tar (pid: 12947, process nr: 61, stackpage=
0394c000)
Dec 3 15:23:36 log2 kernel: Stack: 07777018 061e2414 0394cf34 00116dcd 0394cf20
0bc28390 00000202 0394cf48
Dec 3 15:23:36 log2 kernel: 001ce964 20000000 00000001 32000028 00007993
001f0098 001ace3e 0394cf50
Dec 3 15:23:36 log2 kernel: 00117365 00000800 0ccdcd48 00000000 07777018
0bc2838c 0bc28300 00132874
Dec 3 15:23:36 log2 kernel: Call Trace: [<00116dcd>] [<001ace3e>] [<00117365>]
[<00132874>] [<0012aa84>] [<0010ad8b>]
Dec 3 15:23:36 log2 kernel: Code: 89 0a c7 40 04 00 00 00 00 c7 00 00 00 00 00
56 9d 89 d8 8d
-------------------------------------------------------------------------------
Here are portions of the System.map that coorespond to EIP and Call Trace
numbers:
EIP: 001175b2 00117590 T del_timer
001175d0 T tqueue_bh
Call Trace: 00116dcd 00116a80 T schedule
00116de8 T sys_pause
001ace3e 001ace50 t requeue_sd_request
001adc10 t check_scsidisk_media_change
00117365 001172e4 T interruptible_sleep_on
0011738c T sleep_on
00132874 00132748 t pipe_write
0013298c t pipe_lseek
0012aa84 0012a90c T sys_write
0012aac0 t sock_readv_writev
0010ad8b 0010ac70 T system_call
0010add0 T ret_from_sys_call
===============================================================================
Dec 3 16:10:57 log2 kernel: general protection: 0000
Dec 3 16:10:57 log2 kernel: CPU: 1
Dec 3 16:10:57 log2 kernel: EIP: 0010:[<00117370>]
Dec 3 16:10:57 log2 kernel: EFLAGS: 00010816
Dec 3 16:10:57 log2 kernel: eax: 8890f432 ebx: 0394cf48 ecx: 0c4c7f68 edx
: 8890f432
Dec 3 16:10:57 log2 kernel: esi: 00000202 edi: 0c4c7f68 ebp: 0c4c7f70 esp
: 0c4c7f5c
Dec 3 16:10:57 log2 kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 001
8
Dec 3 16:10:57 log2 kernel: Process dd (pid: 12948, process nr: 62, stackpage=0
c4c7000)
Dec 3 16:10:57 log2 kernel: Stack: 0bc28300 0ff68440 0bc28300 0e799018 0394cf48
0bc28300 001325e6 0bc28390
Dec 3 16:10:57 log2 kernel: 00000000 0ff68440 00000200 0bc28300 00000200
00000000 0012a8e4 0bc28300
Dec 3 16:10:57 log2 kernel: 0ff68440 08003d32 00000200 0e799018 08003d32
00000000 bffffb38 0010ad8b
Dec 3 16:10:57 log2 kernel: Call Trace: [<001325e6>] [<0012a8e4>] [<0010ad8b>]
Dec 3 16:10:57 log2 kernel: Code: 8b 42 04 39 c8 74 05 89 c2 eb f5 90 89 5a 04
56 9d 8d 65 ec
-------------------------------------------------------------------------------
Here are portions of the System.map that coorespond to EIP and Call Trace
numbers:
EIP: 00117370 001172e4 T interruptible_sleep_on
0011738c T sleep_on
Call Trace: 001325e6 00132524 t pipe_read
00132748 t pipe_write
0012a8e4 0012a810 T sys_read
0012a90c T sys_write
0010ad8b 0010ac70 T system_call
0010add0 T ret_from_sys_call
===============================================================================
Dec 4 03:12:38 log2 kernel: general protection: 0000
Dec 4 03:12:38 log2 kernel: CPU: 1
Dec 4 03:12:38 log2 kernel: EIP: 0010:[<0015364f>]
Dec 4 03:12:38 log2 kernel: EFLAGS: 00010216
Dec 4 03:12:38 log2 kernel: eax: 00003cb7 ebx: 02288ac0 ecx: b1f84aed edx
: 62288830
Dec 4 03:12:38 log2 kernel: esi: 00003cb7 edi: 09e41018 ebp: b1f84aed esp
: 0445bea4
Dec 4 03:12:38 log2 kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 001
8
Dec 4 03:12:38 log2 kernel: Process gunzip (pid: 20881, process nr: 55, stackpa
ge=0445b000)
Dec 4 03:12:38 log2 kernel: Stack: 04bfe6c4 09e41018 00000000 b1f84aed 00000000
04bfe2c8 00151e2c 09e41018
Dec 4 03:12:38 log2 kernel: 09e41018 00f80b20 ee342977 00f80b4c 00000001
00000000 000000ff 00000202
Dec 4 03:12:38 log2 kernel: 000003fd 1402020a 00000000 00152fd7 09e41018
00f80b20 b1f84aed 00000014
Dec 4 03:12:38 log2 kernel: Call Trace: [<00151e2c>] [<00152fd7>] [<00149fab>]
[<00142c60>] [<0011ebdb>] [<0010ac4b>]
Dec 4 03:12:38 log2 kernel: Code: 8a 02 83 e0 0f 8d 2c 82 8a 55 0d 88 d0 c0 e8
01 83 e0 01 83
Dec 4 03:12:38 log2 kernel: Aiee, killing interrupt handler
-------------------------------------------------------------------------------
Here are portions of the System.map that coorespond to EIP and Call Trace
numbers:
EIP: 0015364f 00153564 T tcp_write_xmit
0015374c T tcp_do_retransmit
Call Trace: 00151e2c 00151858 t tcp_ack
0015211c t tcp_fin
00152fd7 00152788 T tcp_rcv
00153138 T tcp_new_window
00149fab 00149b58 T ip_rcv
0014a0dc t ip_frag_create
00142c60 00142b64 T net_bh
00142c80 T dev_tint
0011ebdb 0011eba0 T do_bottom_half
0011ec00 T get_ioport_list
0010ac4b 0010ac40 t handle_bottom_half
0010ac60 t reschedule
===============================================================================
--- +-----------------------+-------------------------+------------------------+ | Tom Schenk | Use Linux! | All opinions expressed | | tschenk@dejanews.com | Friends don't let | are mine and not those | | tschenk@theoffice.net | friends do Windows! | of my employer. | +-----------------------+-------------------------+------------------------+