Linode crashing periodically

My linode (SwamiProject) is crashing periodically (once every week or so). When this happens, I can still ping the server address, but can't SSH in, access any web pages, etc. The system graphs on the Linode management page, show that the CPU spikes and then somewhat flatlines at 6% CPU usage, around when the crash is suspected to have occurred (it happened today 2012-03-29 at around 13:45 on the graph). The I/O and network throughput is at 0.

This last crash, I opened a LISH console and managed to capture the following output. I could not figure out how to scroll back in the text, but it looks like a Kernel oops to me. Unfortunately this excerpt is probably not all that useful, since it doesn't show the top of the call stack. I did not find this in any logs either after rebooting the system. The system is running Ubuntu 11.04.

Any help would be much appreciated!

[] ? docoprocessorsegment_overrun+0x80/0x80

[] ? oops_end+0x71/0xa0

[] ? doinvalidop+0x7f/0x90

[] ? swapcountcontinued+0x176/0x180

[] ? hypercall_page+0x227/0x1000

[] ? xenforceevtchn_callback+0x17/0x30

[] ? check_events+0x8/0xc

[] ? xenrestorefldirectreloc+0x4/0x4

[] ? freehotcold_page+0xd3/0x140

[] ? xenforceevtchn_callback+0x17/0x30

[] ? ptepfnto_mfn+0xb5/0xd0

[] ? error_code+0x5a/0x60

[] ? trypreservelarge_page+0x7b/0x340

[] ? docoprocessorsegment_overrun+0x80/0x80

[] ? swapcountcontinued+0x176/0x180

[] ? swapentryfree+0xf1/0x120

[] ? freeswapand_cache+0x27/0xd0

[] ? zappterange+0x1b3/0x470

[] ? _pagevecfree+0x1d/0x30

[] ? unmappagerange+0x111/0x190

[] ? unmap_vmas+0x12b/0x1e0

[] ? exit_mmap+0x91/0x140

[] ? mmput+0x2b/0xc0

[] ? exit_mm+0xef/0x120

[] ? rawspinlockirq+0x10/0x20

[] ? do_exit+0x125/0x350

[] ? xenforceevtchn_callback+0x17/0x30

[] ? dogroupexit+0x3c/0xa0

[] ? sysexitgroup+0x11/0x20

[] ? syscall_call+0x7/0xb

[] ? sctprcvootb+0x50/0xf0

5 Replies

It's a screen session, so you can ctrl+a, [ to enter copy mode, then scroll up. This kind of thing tends to happen when you run out of memory and so then the OOMKiller steps in and kills off some running processes. Assuming this is the case here, you'll want to evaluate your memory usage and optimize where appropriate, or consider upgrading.

Which kernel were you running?

> Which kernel were you running?

Its running kernel 2.6.39.1-linode34 which I see is deprecated. I forgot that the Kernel is selected from the configuration, rather than the distribution. So this may be the kernel I had from the beginning.

I'm going to go ahead and try the latest kernel and will also look into memory usage. Its probably related to apache. PHP can consume a lot of memory on one of the Drupal based websites I have, so it may be that someone is spidering that site and causing the crash.

Thanks for the tips.

@elementgreen:

Its probably related to apache.

Yes, Apache can be quite a hog. Try using nginx.

I used to use Apache until some time last year, when it started taking about 40 seconds to load a page, which was ridiculous and finally convinced me to switch. I've had no problems at all with nginx. :D

@Keiji:

@elementgreen:

Its probably related to apache.

Yes, Apache can be quite a hog.Try to change your configuration off defaults and use mpm-worker instead of mpm-prefork, (and php-fastcgi instead of mod_php), like everyone should.
Fixed that for you.

It's totally not fair to compare "out of the box" (and definitely idiotic) config of Apache, with mpm-prefork, mod_php, and huge clients settings with tweaked (and you NEED to tweak it to make it work at all) nginx with its non-forking engine, and PHP as fastcgi.

When you'll set your Apache to use mpm-worker (or the poll-based one that's even more like Nginx) engine, and set up PHP as external fastcgi process (like you HAVE to do with nginx), the performance difference is much smaller. And the supportability and flexibility is much higher. ;)

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct