My Linode is Slow!

You've hit the IO limiter:

iocount=8002877 iorate=499 iotokens=-193 tokenrefill=512 token_max=400000

due to your high swap usage (most likely):

Filename Type Size Used Priority

/dev/ubd/1 partition 263160 130320 -1

-Chris

So what do I need to do to figure this problem out?

I need more ram ie move up ? What's the best way to benchmark and figure this thing out?

> iocount=8002877 iorate=499 iotokens=-193 tokenrefill=512 token_max=400000

due to your high swap usage (most likely):

Filename Type Size Used Priority

/dev/ubd/1 partition 263160 130320 -1

What are 'good' values for those and what do they mean?

My CATALINA_OPTS are

"-Xint -Xrs -Xmx48m -Xverify:none -XX:+UseParallelGC"

IE only 48m ram. But how does the jre (1.5) manage that. IE, will it hit 48mb of ram then start using swap or is it locked at using a total of 48mb ram + swap.

So if I were to increase it to say 96mb it could just overflow the rest of the ram I specify and use swap cause there isn't enough on the system.

What's the best way to figure out if its swapping or using ram?

I'm used to having 1-2gb of ram so this memory limited env is hard for me. I'm sure it can easily be done :-D

Cheers

Willie

Ah… been doing some reading on this forum. I kinda understand a bit more but some advice would be great.

This is what my cat /proc/iostatus is now (i've been watching iotokens come out of negative value)

iocount=12475222 iorate=107 iotokens=1823 tokenrefill=512 token_max=400000

From another post I found

io_count = total amount of tokens used since boot

io_rate = tokens being used per time unit (per second???)

io_tokens = number of tokens available to be used

tokenrefill = number of tokens that iotokens will increase by per second.

My io_tokens keeps going negative. So anyway - untill I figure this thing out I've stopped my services to make sure I aint killing anyone else (although I suppose that's what the limiter stops by itself)

hehe. Just while I was writing this I ps aux 'd and found a rouge java process running ant. I tried to compile a webapp last night and it tanked. I thought it would be killed when I logged out but obviously it wasn't.

Even on my machine the jdk couldn't compile the same webapp with 768mb ram - it kept getting out of memory exceptions. So I finally compiled it using jikes instead (jikes literally did it in 30 seconds where jdk javac took 2 minutes then crashed with an out of memory exception)

Then I uploaded the war. But all this time it seems that process has been chomping away. But anyway - since I killed it it hasn't had a positive effect on my io_tokens. So I've still shut tomcat down till I figure out how to run it without killing the machine.

Currently my swap is 256mb for a 128mb linode. Perhaps I could resize my partitions and increase swap to 512mb? Or should I increase my jdk memory allocation for the tomcat process or should I just bite the bullet and fork out some more money to caker for some more memory ;-)

Cheers

Willie

Since I shut down tomcat my io_tokens has skyrocketed:

iocount=12793985 iorate=0 iotokens=71668 tokenrefill=512 token_max=400000

The console still lags a bit though….. Hmmmm….

Maybe I could use Jetty instead of Tomcat? I understand its pretty small on its memory usage. Though I'd rather not go through the pain of learning another app server inside out. Tomcat is so easy.

I increased my CATALINA_OPTS java heap to -Xmx64m.

Now:

io_tokens is sitting at around 395,000 - 400,000 (usually stays close to 400000)

io_rate is sitting at around 15 to 5000. (usually stays close to 15-30)

How can such a minor change make so much difference? I mean, I only added a miniscule 16mb to the java heap and its now rocking. Such a minor change couldnt possibly have such a huge effect could it?

Cheers

Willie

If the minor change makes the difference between having to swap and not having to swap, then it certainly could have a major impact on I/O performance.

You might try the 1.4 series JVM instead of 1.5 too. I've tried 1.5 on linode with jboss and it will get to a point where it seems to max out CPU usage and hang. I disabled class sharing or whatever it's called and that helped slightly but it still wasn't stable and used more memory than 1.4.