The first was a cascade, starting when doing a dselect installlation
from NFS. The second was in the swapper, causing a panic, and I
haven't typed it in from the paper that I copies it down to yet.
Anywat, the first:
Unable to handle kernel paging request at virtual address 48e0c0ef
current->tss.cr3 = 00b8b000, r3 = 00b8b000
*pde = 00000000
Oops: 0002
CPU: 0
EIP: 0010:[<c012f034>]
EFLAGS: 00010202
eax: c07be860 ebx: 48e0c0ef ecx: c121433e edx: 48e0c0ef
esi: 9e4c0000 edi: c01bfb10 ebp: 00000000 esp: c0dafe9c
ds: 0018 es: 0018 ss: 0018
Process dpkg (pid: 1559, process nr: 37, stackpage=c0daf000)
Stack: 00000001 00000008 00008124 c0c7b000 00000000 00000004 00000000 00000ad6
00000008 00000000 00000001 c0efdca0 c012e815 00000007 00000000 00000000
c0daff54 c012ea21 00000008 00000000 c0139835 00000000 c0daff54 00008124
Call Trace: [<c012e815>] [<c012ea21>] [<c0139835>] [<c012a4f0>] [<c012a678>]
[<c013bbc7>] [<c012a896>] [<c0122a43>] [<c0122bf7>] [<c010995e>]
Code: 89 02 89 09 89 49 04 eb ad 8d 76 00 83 7c 24 24 00 75 4d a1
My klogd is recent, but wasn't translating addresses (the kern.log shows
a complaint about "Error seeking in /dev/kmem"), so I had to do the
mapping by hand in the System.map:
c012ef90 T select_dcache
c012f034 = select_dcache+164
c012f12c T prune_dcache
c012e7e8 t try_to_free_inodes
c012e815 = try_to_free_inodes+45
c012e830 T free_inode_memory
c012ea10 T get_empty_inode
c012ea21 = get_empty_inode+17
c012eaa8 t get_new_inode
c0139800 T ext2_new_inode
c0139835 = ext2_new_inode+53
c0139e78 T ext2_count_free_inodes
c012a474 t do_follow_link
c012a4f0 = do_follow_link+124
c012a4fc T lookup_dentry
c012a4fc T lookup_dentry
c012a678 = lookup_dentry+380
c012a6b8 T __namei
c013bba0 T ext2_create
c013bbc7 = ext2_create+39
c013bce8 T ext2_mknod
c012a73c T open_namei
c012a896 = open_namei+346
c012aa50 T do_mknod
c01229f8 t do_open
c0122a43 = do_open+75
c0122b2c T get_unused_fd
c0122bbc T sys_open
c0122bf7 = sys_open+59
c0122c28 T sys_creat
c0109924 T system_call
c010995e = system_call+58
c0109964 T ret_from_sys_call
This is the code, disassembled. The assembly is from the .o file, so
relocations are not complete.
while (next != &dentry_unused && depth--) {
18e: 81 fb 00 00 00 cmpl $0x0,%ebx
193: 00
194: 0f 84 27 01 00 je 2c1 <select_dcache+18d>
199: 00
19a: ff 4c 24 1c decl 0x1c(%esp,1)
19e: 83 7c 24 1c ff cmpl $0xffffffff,0x1c(%esp,1)
1a3: 0f 84 18 01 00 je 2c1 <select_dcache+18d>
1a8: 00
struct list_head *tmp = next;
1a9: 89 d9 movl %ebx,%ecx
struct dentry *dentry = list_entry(tmp, struct dentry, d_lru);
1ab: 8d 53 e0 leal 0xffffffe0(%ebx),%edx
struct inode *inode = dentry->d_inode;
1ae: 8b 73 e8 movl 0xffffffe8(%ebx),%esi
unsigned long value = 0;
1b1: c7 44 24 10 00 movl $0x0,0x10(%esp,1)
1b6: 00 00 00
next = tmp->prev;
1b9: 8b 5b 04 movl 0x4(%ebx),%ebx
if (forward)
1bc: 83 7c 24 24 00 cmpl $0x0,0x24(%esp,1)
1c1: 74 02 je 1c5 <select_dcache+91>
next = tmp->next;
1c3: 8b 19 movl (%ecx),%ebx
if (dentry->d_count) {
1c5: 83 3a 00 cmpl $0x0,(%edx)
1c8: 74 1a je 1e4 <select_dcache+b0>
dentry_stat.nr_unused--;
1ca: ff 0d 04 00 00 decl 0x4
1cf: 00
list_del(tmp);
1d0: 8b 51 04 movl 0x4(%ecx),%edx
1d3: 8b 01 movl (%ecx),%eax
1d5: 89 50 04 movl %edx,0x4(%eax)
*1d8: 89 02 movl %eax,(%edx) * <-- Error here
INIT_LIST_HEAD(tmp);
1da: 89 09 movl %ecx,(%ecx)
1dc: 89 49 04 movl %ecx,0x4(%ecx)
continue;
1df: eb ad jmp 18e <select_dcache+5a>
Clearly, something on the dentry chain has a trashed prev pointer.
This cascaded as follows (untranslated, sorry. I can do so if desired.)
There was a chunk of null bytes in the syslog in the middle of the oopsen.
Dec 6 00:33:21 mv kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000
Dec 6 00:33:21 mv kernel: current->tss.cr3 = 01685000, r3 = 01685000
Dec 6 00:33:21 mv kernel: *pde = 00000000
Dec 6 00:33:21 mv kernel: Oops: 0000
Dec 6 00:33:21 mv kernel: CPU: 0
Dec 6 00:33:21 mv kernel: EIP: 0010:[<c012f1cc>]
Dec 6 00:33:21 mv kernel: EFLAGS: 00010207
Dec 6 00:33:21 mv kernel: eax: c0a39f40 ebx: c01e7f2c ecx: 00000000 edx: c0d969c0
Dec 6 00:33:21 mv kernel: esi: 00000000 edi: c0980002 ebp: c01e7f2c esp: c0daff64
Dec 6 00:33:21 mv kernel: ds: 0018 es: 0018 ss: 0018
Dec 6 00:33:21 mv kernel: Process umount (pid: 8840, process nr: 37, stackpage=c0daf000)
Dec 6 00:33:21 mv kernel: Stack: c01e7f2c 00000002 c0980002 c07ad4c4 c0126f5c c01e7f2c 00000002 ffffffff
Dec 6 00:33:21 mv kernel: 00000002 fffffffa c0980002 c0127044 00000002 00000000 00000000 08040002
Dec 6 00:33:21 mv kernel: c09880e0 bffffce8 c0127127 00000002 c0dae000 0804da91 0804b93f c010995e
Dec 6 00:33:21 mv kernel: Call Trace: [<c0126f5c>] [<c0127044>] [<c0127127>] [<c010995e>]
Dec 6 00:33:21 mv kernel: Code: 8b 09 39 6e 2c 75 ef 8b 46 04 8b 16 89 42 04 89 10 a1 10 fb
Dec 6 00:35:46 mv kernel: Unable to handle kernel paging request at virtual address 48e0c0d7
Dec 6 00:35:46 mv kernel: current->tss.cr3 = 0114c000, r3 = 0114c000
Dec 6 00:35:46 mv kernel: *pde = 00000000
Dec 6 00:35:46 mv kernel: Oops: 0000
Dec 6 00:35:46 mv kernel: CPU: 0
Dec 6 00:35:46 mv kernel: EIP: 0010:[<c012f00a>]
Dec 6 00:35:46 mv kernel: EFLAGS: 00010217
Dec 6 00:35:46 mv kernel: eax: 00000e17 ebx: 48e0c0ef ecx: 48e0c0ef edx: 48e0c0cf
Dec 6 00:35:46 mv kernel: esi: c08e0894 edi: 00000000 ebp: 00000000 esp: c0b11ed8
Dec 6 00:35:46 mv kernel: ds: 0018 es: 0018 ss: 0018
Dec 6 00:35:46 mv kernel: Process mount (pid: 8846, process nr: 37, stackpage=c0b11000)
Dec 6 00:35:46 mv kernel: Stack: 00000001 00000008 c0b11fb4 00000002 00000000 00000004 00000000 00000ad6
Dec 6 00:35:46 mv kernel: 00000008 00000000 00000001 c0efdca0 c012e815 00000007 00000000 00000002
Dec 6 00:35:46 mv kernel: 00000002 c012ea21 00000008 00000002 c0143663 00000002 00000002 c0143ccf
Dec 6 00:35:46 mv kernel: Call Trace: [<c012e815>] [<c012ea21>] [<c0143663>] [<c0143ccf>] [<c011a3e1>] [<c0143d61>] [<c0145480>]
Dec 6 00:35:46 mv kernel: [<c010995e>]
Dec 6 00:35:46 mv kernel: Code: 8b 73 e8 c7 44 24 10 00 00 00 00 8b 5b 04 83 7c 24 24 00 74
Dec 6 00:35:55 mv kernel: Unable to handle kernel paging request at virtual address 48e0c0d7
Dec 6 00:35:55 mv kernel: current->tss.cr3 = 00d1e000, r3 = 00d1e000
Dec 6 00:35:55 mv kernel: *pde = 00000000
Dec 6 00:35:55 mv kernel: Oops: 0000
Dec 6 00:35:55 mv kernel: CPU: 0
Dec 6 00:35:55 mv kernel: EIP: 0010:[<c012f00a>]
Dec 6 00:35:55 mv kernel: EFLAGS: 00010217
Dec 6 00:35:55 mv kernel: eax: 00000e20 ebx: 48e0c0ef ecx: 48e0c0ef edx: 48e0c0cf
Dec 6 00:35:55 mv kernel: esi: c08e0894 edi: 00000000 ebp: 00000000 esp: c0b19e9c
Dec 6 00:35:55 mv kernel: ds: 0018 es: 0018 ss: 0018
Dec 6 00:35:55 mv kernel: Process shutdown (pid: 8849, process nr: 47, stackpage=c0b19000)
Dec 6 00:35:55 mv kernel: Stack: 00000001 00000008 000081a4 c16f52dc 00000000 00000004 00000000 00000ad7
Dec 6 00:35:55 mv kernel: 00000008 00000000 00000001 c0efdca0 c012e815 00000007 00000000 00000000
Dec 6 00:35:55 mv kernel: c0b19f54 c012ea21 00000008 00000000 c0139835 00000000 c0b19f54 000081a4
Dec 6 00:35:55 mv kernel: Call Trace: [<c012e815>] [<c012ea21>] [<c0139835>] [<c012a4f0>] [<c012a678>] [<c013bbc7>] [<c012a896>]
Dec 6 00:35:55 mv kernel: [<c0122a43>] [<c0122bf7>] [<c010995e>]
Dec 6 00:35:55 mv kernel: Code: 8b 73 e8 c7 44 24 10 00 00 00 00 8b 5b 04 83 7c 24 24 00 74
16 mv kernel: 00000002 c012ea21 00000008 00000001 c0143663 00000001 00000002 c0143ccf
Dec 6 00:36:16 mv kernel: Call Trace: [<c012e815>] [<c012ea21>] [<c0143663>] [<c0143ccf>] [<c011a3e1>] [<c0143d61>] [<c0145480>]
Dec 6 00:36:16 mv kernel: [<c010995e>]
Dec 6 00:36:16 mv kernel: Code: 8b 73 e8 c7 44 24 10 00 00 00 00 8b 5b 04 83 7c 24 24 00 74