Kernel panic
BUG: unable to handle kernel NULL pointer dereference at virtual address 00000008
printing eip:
15c16000 -> *pde = 00000005:ae4de027
13721000 -> *pme = 00000000:00000000
Oops: 0000 [#1]
SMP
Modules linked in:
CPU: 1
EIP: 0061:[<c03374f6>] Not tainted VLI
EFLAGS: 00010292 (2.6.18.8-linode16 #1)
EIP is at rb_erase+0xb6/0x280
eax: 00000000 ebx: 00000000 ecx: 00000000 edx: c75b2748
esi: df7f5f48 edi: df7f53c8 ebp: c0640ae0 esp: c15a3f38
ds: 007b es: 007b ss: 0069
Process events/1 (pid: 11, ti=c15a2000 task=c159a610 task.ti=c15a2000)
Stack: 00000000 00000001 df7f53c0 c055fe84 c15737c0 00000000 c02fef44 c055fe80
c012e772 00000000 c159a610 c15737c0 c15737e0 c15737cc 00000000 c02fef00
c15737cc c15737c0 c15737d4 c15a3fa0 c012ef77 00000001 00000000 c054d100
Call Trace:
[<c02fef44>] key_cleanup+0x44/0xe0
[<c012e772>] run_workqueue+0x72/0xf0
[<c02fef00>] key_cleanup+0x0/0xe0
[<c012ef77>] worker_thread+0x127/0x150
[<c0117120>] default_wake_function+0x0/0x10
[<c012ee50>] worker_thread+0x0/0x150
[<c0131bd6>] kthread+0xf6/0x100
[<c0131ae0>] kthread+0x0/0x100
[<c0102b95>] kernel_thread_helper+0x5/0x10
Code: 02 83 e0 03 09 c8 89 02 83 7c 24 04 01 74 48 83 c4 08 5b 5e 5f 5d c3 83 c8 01 89 ea 89 01 89 f0 83 26 fe e8 3d fd ff ff 8b 4e 08 <8b> 59 08 85 db 74 09 f6 03 01 0f 84 21 01 00 00 8b 41 04 85 c0
EIP: [<c03374f6>] rb_erase+0xb6/0x280 SS:ESP 0069:c15a3f38</c03374f6></c0102b95></c0131ae0></c0131bd6></c012ee50></c0117120></c012ef77></c02fef00></c012e772></c02fef44></c03374f6>
What really sucks about this is that since the panic sometime last night, my node has shown as "Up" in the linode manager, and the CPU usage appears very high on the graph during the entire time it was down. Is there anything I can do to prevent it from eating CPU, etc, when it is hosed like this? Or get it to reboot automagically? Lassie was enabled, BTW.
caker
Linode Staff
You could set "/proc/sys/kernel/paniconoops" to 1, that way your kernel would exit if it hits an oops, and then Lassie would kick in.
-Chris
Thanks :)
1) ssh session stalled.
2) console screen session – connected but wouldn't take any keyboard I/O.
3) CPU graph was maxed out.
Tom eventually suggested forcing a power cycle of the Linode. That did the trick. I'm thinking it sounds like a potential UML bug of some kind.
Unfortunately, I didn't see any kernel oops message or I'd have had posted it.
I'm running the 2.6-latest kernel with my Linode profile. caker's tip sounds great -- thanks, caker! Will try. (Not sure what my UML kernel was previously, but it was whatever was 2.6-latest as of around May 2, 2009, when I last booted it.)
sysctl -w kernel.paniconoops=1
vi /etc/sysctl.conf
…and added:
kernel.paniconoops=1
(This being on Ubuntu.)
Feb 11 21:57:39 bender kernel: <1>BUG: unable to handle kernel NULL pointer dereference at virtual address 00000008
Feb 11 21:57:39 bender kernel: c0337f07
Feb 11 21:57:39 bender kernel: Modules linked in:
Feb 11 21:57:39 bender kernel: EIP: 0061:[<c0337f07>] Not tainted VLI
Feb 11 21:57:39 bender kernel: EFLAGS: 00010282 (2.6.18.8-linode19 #1)
Feb 11 21:57:39 bender kernel: [<c0338047>] rb_insert_color+0x77/0xe0
Feb 11 21:57:39 bender kernel: [<c02ff9a0>] key_alloc+0x260/0x370
Feb 11 21:57:39 bender kernel: [<c0300af0>] keyring_alloc+0x30/0x80
Feb 11 21:57:39 bender kernel: [<c03026a4>] alloc_uid_keyring+0x54/0xd0
Feb 11 21:57:39 bender kernel: [<c0127f84>] alloc_uid+0xe4/0x1b0
Feb 11 21:57:39 bender kernel: [<c012bd4b>] set_user+0xb/0xa0
Feb 11 21:57:39 bender kernel: [<c012d876>] sys_setuid+0x156/0x170
Feb 11 21:57:39 bender kernel: [<c010547f>] syscall_call+0x7/0xb</c010547f></c012d876></c012bd4b></c0127f84></c03026a4></c0300af0></c02ff9a0></c0338047></c0337f07>
Fortunately, I caught it within minutes (thanks to wasitup.com) and was able to force a reboot.