Linode.com Forum

[irgeek]

OK, I have no idea what's going on here. Of course, I do have a theory. I was recently messing around on my machine and I noticed that the start times of processes--as reported by ps--were wrong. I've since run some tests to see what's wrong, but that's just confused me more.

The strange behaviour is this: when I run 'date' the date/time are reported correctly. When I run ps, the start time of processes is 37'6" in the future. Here's some output to clear up what I mean. I just wrote a script that runs 'date' then 'ps a --format lstart cmd' then 'date' again with some 'echo's to spread it out.

The Script
Code: #!/bin/bash

/bin/echo "";
/bin/date;
/bin/echo "";
/bin/echo "";
/bin/ps a --format "lstart cmd"
/bin/echo "";
/bin/echo "";
/bin/date
/bin/echo "";


The Output
Code: [root@mt-elbert root]# ./clock_test

Sun Aug 31 01:06:19 MDT 2003


                 STARTED CMD
Mon Sep  1 13:23:48 2003 -bash
Mon Sep  1 13:25:51 2003 su
Mon Sep  1 13:25:51 2003 bash
Mon Sep  1 14:12:10 2003 /bin/bash ./clock_test
Mon Sep  1 14:12:10 2003 ps a --format lstart cmd


Sun Aug 31 01:06:19 MDT 2003


You can see from the two 'date' commands that very little time elapsed between them, but then the last line of the ps command says that ps was started far in the future. Now, the only theory I can come up with is that the host server has it's time set incorrectly somehow. I believe that UML runs each process as a process on the server, so the start time reported is coming from the host server. That seemed logical until I figured it was 37 hours in the future. Why would the server be 37 hours ahead of my clock?

Has anybody else seen this behaviour? If you runs these commands do you get the same weirdness? I need some help figuring this one out. Thanks.

--James

[/b]

[caker]

I synced the clock on the host of your Linode, and it was only ~100 seconds off, so that's not the problem. Also, I would think that if the time is wrong on the host, that everything that uses the Linode's clock would be off. Remember that your Linode's kernel only uses the hosts's clock when booting -- after that I believe it keeps it's own time.

I'll dig through some source code (of ps, specifically) to see where the heck ps is getting that time from.

-Chris

[caker]

Jeff Dike recently made me aware of a patch that fixes this. It was submitted earlier this year and he'll be including it in the next UML patch. It syncs the UML's clock to the host's clock, so there won't be missed ticks like we see now with sleep/etc.

-Chris

[bjohnson]

caker wrote: Remember that your Linode's kernel only uses the hosts's clock when booting -- after that I believe it keeps it's own time.


Can you verify this? I thought the UML clock was tied to the host clock. If not, I guess I should be running ntp...

[schof]

bjohnson wrote: caker wrote: Remember that your Linode's kernel only uses the hosts's clock when booting -- after that I believe it keeps it's own time.


Can you verify this? I thought the UML clock was tied to the host clock. If not, I guess I should be running ntp...

Sounds like you should be running ntp anyway. Judging from caker's comments, he isn't running it regularly. "I synced the clock on the host of your Linode, and it was only ~100 seconds off, so that's not the problem."

[caker]

I don't believe this issue causes UML to lose time as far as the actual time-of day clock is concerned. UML depends on the host to send SIGALARMS, and I'm guessing it only checks to see if it should send a signal every so often, there-by missing some. This patch makes it use the host's Time Stamp Counter.

Things like sleep and vmstat setup an alarm(), so that's why they are off in their delays...

I dug up Sapan's original post:

Sapan Bhatia wrote:
The clock skew bug is evidently caused by the inability of the host to
generate SIGALRMs at the frequency HZ because of its low resolution
(host HZ = 100). So lags add up in milliseconds and there's no
differenciation between say HZ=52 and HZ=60. Both result in an actual
frequency of b/w 40 and 50 Hz depending on the load.

I have a solution that fixes this for the basic test case. It uses the
Time Stamp Counter to disperse fluctuations in the delay.

Here are some results:
Code:          Sleep 30      Sleep 60
[original UML]
HZ=52         31.261         63.412      
HZ=80         48.070             137.990
HZ=50         30.068         60.124

[patched]
HZ=52         30.021         60.021
HZ=80         30.027         60.027
HZ=50         30.006         60.040


http://marc.theaimsgroup.com/?l=user-mode-linux-devel&m=103255224715701&w=2

-Chris

[caker]

NTP is running on all the hosts, and the way the UML works (by host offsets) it would be counterproductive to run it inside your Linode, too.

-Chris