Poor Game Server Performance
ZombieMod: 3-4 players against 20-30 Bots (CPU Players)
GunGame: 3-4 Players against 3-4 other players (No Bots)
Now when playing I seem to get a lot of lag. With ZombieMod it can go 5-10 seconds before I get "lagged" back about 2 seconds, sometimes it can get shot back 5-6 seconds, this happens for everyone on the server. When playing GunGame I get similair and lesser lag.
This to some extent infers that the problem is CPU usage. When running ZombieMod by CPU usage goes to around 60-70%, but when running GunGame I still get 30-40%.
I have had to replace my /proc/cpuinfo with one that the server can understand (See http://www.linode.com/forums/viewtopic.php?t=2491
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Xeon(TM) CPU 2.40GHz
stepping : 7
cpu MHz : 2393.024
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid
bogomips : 4789.92
Ok, now that i've typed a wall of text, does anyone have an suggestions as to how to fix this problem? It half-seems like my servers connection gets cut off for a second every few seconds, but I know this is not the case.
Thanks,
Smark
9 Replies
* IO Limiter: If Counter Strike is IO bound or causes your Linode to start swapping, the IO limiter will kill you. Running 30-40 bots may be the culprit, but that doesn't explain the issue when it's just humans playing.
****/dev/random:**** If Counter Strike is pulling random data from /dev/random–and I can think of several reasons it would--entropy can run out in a hurry. Linux usually generates entropy (in part) from keyboard & mouse interrupts. Being a headless system, Linodes run out quickly.</list>
I'm not sure if this helps, but it's something to investigate.
–James
Is there any monitoring tools that I should look into that could help me to isolate the issue? IE, what the server is drawing on. Maybe find what the server needs the "most of" and try to figure out a way to make that more efficient.
Thanks,
Smark
while true; do cat /proc/sys/kernel/random/entropy_avail /proc/io_status; sleep 1; done
You can stop that with ctrl-c. If you don't want to watch it in a terminal, it should be fairly easy to set up job to graph the numbers with rrdtool.
If you don't see the numbers changing, then you might need to look somewhere else.
If you can't figure it out, try asking in the IRC channel. That's usually a good place to get help.
–James
/proc/sys/kernel/random/entropy_avail
~~
/proc/iostatus - iorate
~~
/proc/iostatus - iotokens
~~
The Y-axis shows iorate, iotokens, and entropy_avail
The X-axis shows time (each number represents about half a second)
The spike in the io_status graph is going to be the loading of the map, which was roughly 40-50mb
Basically is what I did, was I had each output to a different file, here was the sequence of events:
1. Start Logging
2. 30 seconds later start the server
3. 1 min after that I connected which triggered the bots to join the server
4. Play for about 3-4 minutes
5. I then reduced the bot count to 10 from 32
6. Played for another 3-4 minutes.
7. Reduced the bot count to 5
8. Played for another 3-4 minutes
9. Disconnected from the server
10. 10-30 seconds later I shut down the server
Im not sure how to interpret the iotokens graph, but the entropy graph seems to drop, then go higher every time i reduce the CPU player count…
Thanks for all the help so far,
Smark
PS. The actual data can be found here:
~~
Yes, that's a problem. That means your I/O is running at a drastically reduced rate. Most likely, this is a result of not having enough physical RAM. Once a Linode starts swap thrashing, the I/O tokens will disappear fast.
You'll want to reduce your memory footprint (or upgrade your Linode).
Xan, thanks. I am going to look into updating my linode or maybe having two separate linodes.
Right after my post, I disabled a couple of routines that were doing a lot of IO (mainly updating fields in an sqlite database every minute) and now, it looks like this:
iocount=1877652 iorate=0 iotokens=2000000 tokenrefill=512 token_max=2000000. that's a dramatic change.
What would be a good source of information to understand iocount, iorates, why they would decrease or the stock of io_tokens would replenish?
The IO looks pretty good in my opinion. It looks like when you first start the mission, there's a heavy IO load and the tokens drop rapidly. From then on, the amount of tokens gradually climbs, so you're using less than you're regaining. At least with just the one player and some bots, anyway. Sharp spikes down towards zero would be a sign of trouble here, IMHO.
I'm wondering if the other nodes on the host machine are simply having activity spikes that are interfering with the "real time" nature of the service you're running.