| Author |
Message |
kbrantley
Joined: 21 Sep 2007
Posts: 77
|
| Posted: Sun Mar 07, 2010 10:56 am Post subject: Clock drift! |
|
|
I noticed last night that the clock on my linode was two minutes (okay okay, 104 seconds) slow. Thinking that the clock on the linode was synched to the clock on the host, I threw in a support ticket ... and it seems that the newer paravirt kernels (if you've selected the "Latest 2.6 Paravirt" option) will now drift.
Because this is indeed a VPS provider, the host load may drastically slow your clock down. In short, your clock is going to run slow.
If you don't want your clock to run slow, install and run ntpd.
So go install ntpd! Package name might be "ntp" or "ntpd." Be sure to start the service too.
Hey, linode people! Any chance we could get an internal ntp server to sync to? :)
edit: hey linode people! How about opening up the ntp daemon on the hosts to the VMs they support? I like that idea even better. |
|
| Back to top |
|
JshWright
Joined: 27 Oct 2008
Posts: 173
|
| Posted: Sun Mar 07, 2010 12:07 pm Post subject: |
|
|
| I realize you weren't aware of it, but I thought this was fairly common knowledge? |
|
| Back to top |
|
kbrantley
Joined: 21 Sep 2007
Posts: 77
|
| Posted: Sun Mar 07, 2010 12:46 pm Post subject: |
|
|
Previously, with the UML kernerls, you couldn't adjust your clock -- it was hard synced with the host.
I had a UML based linode, that is now a paravirt-kernel linode.
This is news to me. |
|
| Back to top |
|
glg
Joined: 09 Jan 2009
Posts: 452
|
| Posted: Sun Mar 07, 2010 12:54 pm Post subject: |
|
|
| do a search in the forums, it's pretty well known. Just grab ntpdate. |
|
| Back to top |
|
arjones85
Joined: 12 Oct 2009
Posts: 39
|
| Posted: Sun Mar 07, 2010 2:28 pm Post subject: |
|
|
| Is there any benefit to running ntpd vs just setting up a cronjob to do "ntpdate pool.ntp.org" every month or something? |
|
| Back to top |
|
kbrantley
Joined: 21 Sep 2007
Posts: 77
|
| Posted: Sun Mar 07, 2010 3:00 pm Post subject: |
|
|
glg wrote: do a search in the forums, it's pretty well known. Just grab ntpdate.
I did, and found... very very little. Search for "clock AND drift" and you'll see why I posted this.
arjones85 wrote: Is there any benefit to running ntpd vs just setting up a cronjob to do "ntpdate pool.ntp.org" every month or something?
The advantage of ntpd is that it continually keeps your clock accurate. Instead of jumping forward (or whatever) a large number of seconds, it continually makes minor adjustments to keep the clock in sync. ntpd also (with most distro-provided config files) syncs with several NTP sources at once, which can help negate the effects of general internet latency.
I was seeing a drift of ~2.4186 seconds per day. ntpdate could have been run twice daily and it would help, but with ntpd, you can forget about it entirely.
Note that you can't run ntpd and ntpdate at the same time, as the NTP protocol uses port 139 exclusively on both the server and the client side. |
|
| Back to top |
|
hoopycat
Joined: 30 Aug 2008
Posts: 1134
|
| Posted: Sun Mar 07, 2010 3:09 pm Post subject: |
|
|
Please don't use ntpdate via cron to keep your clock synced.
Most importantly for your system, it will change your clock abruptly, which can make log timestamps ambiguous. If your clock is fast (its time is ahead of the reference time), a sudden jump backwards can freak things out.
It also causes a sudden and heavy load on the time servers at the top of the minute, even more so at the common cron times (*, */2, */3, etc). You're much better off investing the ~1MB of RAM and running ntpd, pointing it at 0.pool.ntp.org, 1.pool.ntp.org, and 2.pool.ntp.org.
This message brought to you by 0.19% of us.pool.ntp.org :-)
(Also, the clock decoupling appears to be a side effect of the mainline Linux Xen implementation. I believe we're seeing the physical host's inherent clock drift. The virtualization is THAT GOOD.) |
|
| Back to top |
|
jeffml
Joined: 03 Feb 2004
Posts: 22
|
| Posted: Sun Mar 07, 2010 4:37 pm Post subject: |
|
|
hoopycat wrote: It also causes a sudden and heavy load on the time servers at the top of the minute, even more so at the common cron times (*, */2, */3, etc).
The statistics geek in me would love to see a graph of requests for the top of the minute requests. It would seem that the spike would relate to the amount of drift in clocks for those that request them. Probably more like a prolonged 15 second spike on each side of the top of the minute? |
|
| Back to top |
|
kbrantley
Joined: 21 Sep 2007
Posts: 77
|
| Posted: Sun Mar 07, 2010 5:52 pm Post subject: |
|
|
hoopycat wrote: (Also, the clock decoupling appears to be a side effect of the mainline Linux Xen implementation. I believe we're seeing the physical host's inherent clock drift. The virtualization is THAT GOOD.)
So my host drifts more than one second per twelve hours? :/
I sure hope not...
*pats ntpd* |
|
| Back to top |
|
jed
Joined: 28 Mar 2009
Posts: 394
Location: New Jersey
|
| Posted: Sun Mar 07, 2010 6:20 pm Post subject: |
|
|
kbrantley wrote: So my host drifts more than one second per twelve hours? :/
I sure hope not...
I have a PC that drifts a minute a day. I've heard of worse. There is a threshold where NTP will tell you to get bent and no longer sync because you drift so much, and I've heard of it being hit, too. |
|
| Back to top |
|
kbrantley
Joined: 21 Sep 2007
Posts: 77
|
| Posted: Sun Mar 07, 2010 6:47 pm Post subject: |
|
|
jed wrote: kbrantley wrote: So my host drifts more than one second per twelve hours? :/
I sure hope not...
I have a PC that drifts a minute a day. I've heard of worse. There is a threshold where NTP will tell you to get bent and no longer sync because you drift so much, and I've heard of it being hit, too.
Sounds a bit like my router... (which has done that before) |
|
| Back to top |
|
hoopycat
Joined: 30 Aug 2008
Posts: 1134
|
| Posted: Sun Mar 07, 2010 7:21 pm Post subject: |
|
|
jeffml wrote: The statistics geek in me would love to see a graph of requests for the top of the minute requests. It would seem that the spike would relate to the amount of drift in clocks for those that request them. Probably more like a prolonged 15 second spike on each side of the top of the minute?
Rest assured that if I could remember where I've seen the graphs, I would have included them. :-) I figure they're somewhere in the timekeepers mailing list archives...
Also, never underestimate how accurate your clock will be if you run ntpdate every minute.
(Don't get me started on htpdate.)
kbrantley wrote: So my host drifts more than one second per twelve hours? :/
But of course. Your host's hardware clock oscillator deviates from an ideal reference oscillator by about 23 parts per million, or 0.0023%. If we assume a 32768Hz oscillator, it's off by less than 1Hz. That's an easy software fix.
Now, the hard thing to fix is the frequency drift: the rate of change of the frequency is a pain in the butt. ntpd is really good about handling that.
Fun reading: http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm |
|
| Back to top |
|
jtsage
Joined: 04 Jan 2009
Posts: 1
Location: Pittsburgh, PA
|
| Posted: Sun Mar 07, 2010 10:32 pm Post subject: Re: Clock drift! |
|
|
kbrantley wrote:
Hey, linode people! Any chance we could get an internal ntp server to sync to? :)
why yes, that is a possibility. This is the short list, there are others.
http://www.linode.com/wiki/index.php/Internal_Services |
|
| Back to top |
|
rsk
Joined: 24 Nov 2009
Posts: 283
|
| Posted: Mon Mar 08, 2010 10:33 am Post subject: |
|
|
Urm, if I can ask for clarification... is NTP necessary only when using paravirt kernels, or with the "oldstyle Xen" 2.6.18.8 too?
ISTR that one doesn't let me mess with the clock at all... |
|
| Back to top |
|
Guspaz
Joined: 26 May 2009
Posts: 1030
Location: Montreal, QC
|
| Posted: Mon Mar 08, 2010 10:36 am Post subject: |
|
|
| I like syncing to ntp.canonical.com... from my Windows boxes :P |
|
| Back to top |
|
| |