512 Linode - Apache2 / Mysql / PHP - Memory?

I'm setting up my new Linode, trying to get a site that's on shared hosting over to it. Just today, while trying to import the database (14M worth of SQL DDL and DML) I got an out of memory error and the script failed. :(

I ran a ps aux and this is what I have:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.2   2728  1140 ?        Ss   Nov06   0:01 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Nov06   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    Nov06   0:00 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S    Nov06   0:23 [kworker/0:0]
root         5  0.0  0.0      0     0 ?        S    Nov06   0:00 [kworker/u:0]
root         6  0.0  0.0      0     0 ?        S    Nov06   0:00 [migration/0]
root         7  0.0  0.0      0     0 ?        S    Nov06   0:00 [migration/1]
root         8  0.0  0.0      0     0 ?        S    Nov06   0:00 [kworker/1:0]
root         9  0.0  0.0      0     0 ?        S    Nov06   0:00 [ksoftirqd/1]
root        10  0.0  0.0      0     0 ?        S    Nov06   0:00 [migration/2]
root        11  0.0  0.0      0     0 ?        S    Nov06   0:00 [kworker/2:0]
root        12  0.0  0.0      0     0 ?        S    Nov06   0:00 [ksoftirqd/2]
root        13  0.0  0.0      0     0 ?        S    Nov06   0:00 [migration/3]
root        14  0.0  0.0      0     0 ?        S    Nov06   0:00 [kworker/3:0]
root        15  0.0  0.0      0     0 ?        S    Nov06   0:00 [ksoftirqd/3]
root        16  0.0  0.0      0     0 ?        S<   Nov06   0:00 [khelper]
root        17  0.0  0.0      0     0 ?        S    Nov06   0:00 [kworker/u:1]
root        21  0.0  0.0      0     0 ?        S    Nov06   0:00 [xenwatch]
root        22  0.0  0.0      0     0 ?        S    Nov06   0:00 [xenbus]
root       148  0.0  0.0      0     0 ?        S    Nov06   0:05 [sync_supers]
root       150  0.0  0.0      0     0 ?        S    Nov06   0:00 [bdi-default]
root       152  0.0  0.0      0     0 ?        S<   Nov06   0:00 [kblockd]
root       162  0.0  0.0      0     0 ?        S<   Nov06   0:00 [md]
root       246  0.0  0.0      0     0 ?        S<   Nov06   0:00 [rpciod]
root       248  0.0  0.0      0     0 ?        S    Nov06   0:36 [kworker/0:1]
root       279  0.0  0.0      0     0 ?        S    Nov06   0:17 [kswapd0]
root       280  0.0  0.0      0     0 ?        SN   Nov06   0:00 [ksmd]
root       281  0.0  0.0      0     0 ?        S    Nov06   0:00 [fsnotify_mark]
root       285  0.0  0.0      0     0 ?        S    Nov06   0:00 [ecryptfs-kthrea]
root       287  0.0  0.0      0     0 ?        S<   Nov06   0:00 [nfsiod]
root       290  0.0  0.0      0     0 ?        S    Nov06   0:00 [jfsIO]
root       291  0.0  0.0      0     0 ?        S    Nov06   0:00 [jfsCommit]
root       292  0.0  0.0      0     0 ?        S    Nov06   0:00 [jfsCommit]
root       293  0.0  0.0      0     0 ?        S    Nov06   0:00 [jfsCommit]
root       294  0.0  0.0      0     0 ?        S    Nov06   0:00 [jfsCommit]
root       295  0.0  0.0      0     0 ?        S    Nov06   0:00 [jfsSync]
root       296  0.0  0.0      0     0 ?        S<   Nov06   0:00 [xfs_mru_cache]
root       297  0.0  0.0      0     0 ?        S<   Nov06   0:00 [xfslogd]
root       298  0.0  0.0      0     0 ?        S<   Nov06   0:00 [xfsdatad]
root       299  0.0  0.0      0     0 ?        S<   Nov06   0:00 [xfsconvertd]
root       300  0.0  0.0      0     0 ?        S<   Nov06   0:00 [glock_workqueue]
root       301  0.0  0.0      0     0 ?        S<   Nov06   0:00 [delete_workqueu]
root       302  0.0  0.0      0     0 ?        S<   Nov06   0:00 [gfs_recovery]
root       303  0.0  0.0      0     0 ?        S<   Nov06   0:00 [crypto]
root       865  0.0  0.0      0     0 ?        S    Nov06   0:00 [khvcd]
root       977  0.0  0.0      0     0 ?        S<   Nov06   0:00 [kpsmoused]
root       978  0.0  0.0      0     0 ?        S    Nov06   0:54 [kworker/1:1]
root       981  0.0  0.0      0     0 ?        S    Nov06   0:47 [kworker/2:1]
root      1008  0.0  0.0      0     0 ?        S    Nov06   0:02 [kjournald]
root      1012  0.0  0.0      0     0 ?        S    Nov06   0:50 [kworker/3:1]
root      1033  0.0  0.1   2368   544 ?        S    Nov06   0:00 upstart-udev-bridge --daemon
root      1035  0.0  0.0   2236   256 ?        S

~~Top shows this:

top - 12:22:12 up 19 days, 13:35,  2 users,  load average: 0.00, 0.01, 0.05
Tasks:  76 total,   1 running,  75 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    509084k total,   485356k used,    23728k free,    30152k buffers
Swap:   262140k total,    14152k used,   247988k free,   407212k cached

I'm confused, it doesn't look like the processes add up to the amount of memory that is being used. I'm fairly new to this, and need some suggestions.

This is an Ubuntu 10.04 Linode.~~

9 Replies

I did some reading on memory reporting in Linux. So now I understand the meaning of "cached" in Top.

This command was helpful:

# free -m
             total       used       free     shared    buffers     cached
Mem:           497        450         46          0         29        357
-/+ buffers/cache:         63        433
Swap:          255          9        246

Plenty of free memory (433M).

So I'm not sure why MySql would hiccup on such a small script. My research indicates MySql doesn't have an upper bound memory setting.

Any ideas?

You mention a "script"… are you importing a file of SQL commands (the output from mysqldump or similar) directly into MySQL (mysql [blah] < something.sql), or are you using some other thing? PHP does have its own memory limits, which an awkward import script could run into.

I do agree that MySQL shouldn't be having a problem here, nor does it look like the system itself has had a recent memory crunch.

The script is a dump of both table structures and data from a Simplemachines Forum (SMF). I've dumped and reloaded entire databases from scripts like this multiple times on both CentOS (current host) and Windows (Local dev) without issues in the past.

I am running the file via mysql command line on the Linux box itself. the exact error is this:

ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '

**Fatal error**:  Out of memory (allocated 34603008) (tried to allocat' at line 1

I can't believe this. The export script aborted and that out of memory error is the last line that was generated by the PHP extract.

The problem is with the extract.

Sorry to waste anybody's time.

Haha, interesting. So you ran out of memory while you were producing the DB dump with a PHP script, but didn't realize it until you tried to reload the dump? I hope you didn't lose any data.

@hybinet:

Haha, interesting. So you ran out of memory while you were producing the DB dump with a PHP script, but didn't realize it until you tried to reload the dump? I hope you didn't lose any data.

Yup. The SMF db dump process didn't report any errors, and even gave me the file (partial).

I didn't lose anything, since I'm still "kicking the tires" on the Linode. I was trying to set up a test instance.

Another reason why I want away from "tools" that encapsulate the job that really needs to be done. I am much happier at the command line with full control.

@JFlame:

Another reason why I want away from "tools" that encapsulate the job that really needs to be done. I am much happier at the command line with full control.

You might check if your shared host has a backup on their panel, it likely will dump out the database. I came from one where the backup utility gave me a .tar.gz file. In it, it had the filesystem plus a mysqldump of the database.

@glg:

@JFlame:

Another reason why I want away from "tools" that encapsulate the job that really needs to be done. I am much happier at the command line with full control.

You might check if your shared host has a backup on their panel, it likely will dump out the database. I came from one where the backup utility gave me a .tar.gz file. In it, it had the filesystem plus a mysqldump of the database.

Yes, I found they do have that capability and I dumped it out via that method and loaded it just fine.

The full backup utility is annoying though because it is limited to backing up the root directory, which has all my websites and exceeds the backup limit for the host. :(

Soon I'll be out of there though. The test instance of the site runs awesome on the Linode. I've never seen such good response times, even for an empty site on the shared host.

@JFlame:

The full backup utility is annoying though because it is limited to backing up the root directory, which has all my websites and exceeds the backup limit for the host. :(

wow, ingenious design they've got :? I think you'll be a lot happier here.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct