[time-nuts] Does GPS time reception work everywhere all of the time?

[mike cook]

  Hal's response was quite complete, but he missed one essential and 
often missed parameter. Redundency.
I have been working for years in operating system support for major 
international computer constructors and it amazes me how often I came 
across equally large international clients who had no idea about 
constructing robust time distribution networks.  As an example, it was 
commonplace to find whole sites  with multiple MC systems relying on 
just one or two NTP time servers and having those two peering !!!!!  So 
finding mission critical systems with 5 or ten minutes drift from wall 
clock was far from unusual. Even when this was pointed out, getting them 
to correct it was very difficult, which meant that repeated downtimes 
for incorrectly executed  corrective action taken by their admins 
sometimes hade me tearing my hair out.

  So always make sure that there is redundant service. There are plenty 
of references about how to do this in a belt and braces manner. How far 
you go (5 independent clocks should be sufficient) depends on how much 
your downtime costs.

Le 31/08/2010 04:42, Bill Hawkins a écrit :
> Hal,
>
> Many thanks for taking the time for that excellent reply.
>
> A true data diode has no leakage. You can make one by clipping the receive
> wires in a switch. The control network sends predictable UDP messages with
> its data. No concern is taken for harm from the Internet, and it is not
> possible to do MS updates to the Windows operator consoles. If it worked
> during process startup, then no improvement is required.
>
> This is not a challenging application for time accuracy, unless a human
> (like the CIO) wants to have better accuracy than his (it's a testosterone
> thing) competition. Process valves have a 1 second time constant, for the
> most part. Programmable logic controllers and their actuators can't react
> faster than 1/2 of a power line cycle. Millisecond precision is adequate.
>
> Flakey? MS Windows may be used for human interfaces, where redundancy is
> economically possible as 1 for N rather than 1 for 1. Nobody uses Windows
> for the deterministic control calculations, which repeat at fixed intervals.
>
> The software genius who built the network software for the control system
> that I designed came up with a simple 256 step adjustment for the crystal
> clock that acted like the fast/slow adjustment on a watch. Double precision
> was not required. You simply added or subtracted a clock interrupt every
> n/256 interrupts. That didn't make him a genius, it was the tiny floating
> point package that beat anything else on the market.
>
> I had hoped for a line frequency clock like the PDP-11, but hardware gave
> me a crystal. Yes, the line frequency is within +/- 0.1% of true time, but
> that was enough in 1985. And it was always back in sync with WWV at 7 AM.
>
> My concern is with the facilities that don't have Internet connections
> and have to use satellite links. Do facilities that are far apart need to
> have one millisecond resolution? That's a testosterone question.
>
> How much am I willing to pay? I dunno, how much are you willing to offer?
>
> Bill Hawkins
>
>
> -----Original Message-----
> From: Hal Murray
> Sent: Monday, August 30, 2010 5:39 AM
>
> bill at iaxs.net said:
>> I'm involved with time synchronization of control system computers for
>> multi-national businesses. GPS springs to mind as a way to synchronize
> time
>> anywhere. Or is it? What about monsoon rains?
>> The Internet is available almost everywhere that control computers are
> used,
>> but many users prefer to use a data diode between them and the Internet.
>> Control computers are now essential for manufacturing processes. Some of
> the
>> processes run constantly for years without stopping for any kind of
> security
>> update. Some of the downtimes cost millions of dollars per day.
> How much does your "diode" leak?
>
> Can you get command packets (or acks) back to your control systems, or are
> they flying deaf?
>
>
>> A GPS time system allows the control systems to be synchronized in time,
> so
>> that messages sent periodically through the data diodes will have the
>> correct time stamp on various events that occur in the process.
>> But does that work everywhere all of the time? Where can I find answers?
> How accurate do you need your clocks to be?
> How reliable do you need/want them to be?
> How much are you willing to pay?
>
> How vulnerable/flaky is the OS on your control systems?  (Are they running
> Windows?)
>
> Note that there are 2 dimensions to accuracy.  One is RMS error, or
> something
> like that.  The other is how often do you go outside some error bars, how
> long is the tail?
>
> ----------
>
> The geeks I hang out with would put a firewall between the big bad internet
> and the system to be protected.  If you are sufficiently paranoid (which
> doesn't take much) you don't allow marketing people or visitors to connect
> their likely-to-be-infected laptop to the subnet with the protected systems.
>
> (You can setup another subnet for them.)
>
> If you need sub-ms accuracy, you probably need a local GPS receiver/system.
>
> For 10s of ms accuracy, you can run ntp on the firewall, or on a separate
> system right behind the firewall.  The accuracy depends on the network load.
>
> There isn't a simple answer.
>
> If all you need is to be within a second or three, you can probably get that
>
> by tweaking the timekeeping fudge factor, possibly by hand.  This would take
>
> occasional monitoring and tuning.
>
> -----------
>
> Consider timekeeping on a computer.  They are all based on a crystal.  It's
> either something like "every interrupt, bump the clock by 1 ms" or it's
> "that
> register ticks at 1 MHz".
>
> The problem is that the crystal usually doesn't tick at exactly the
> specified
> frequency.  It's off by a few to hundreds of PPM.  Low cost crystals are
> generally accurate to within 100 or 50 PPM.  The watch crystal behind the
> CMOS/TOY clock is probably much better but it's hard to use for fine grained
>
> timing.
>
> There are 86400 seconds per day so 1 PPM is 1 second in 11.5 days, 10 PPM is
>
> (roughly) 1 second per day...
>
>
> If you monitor a system for a week or two, you can calculate how far off the
>
> crystal is.  That's by hand, just looking at the printout from something
> like
> the date command, waiting a week and doing it again.
>
> If your OS cooperates, you can tell it the fudge factor so that it will keep
>
> (much) better time.  (You probably need double-precision arithmetic deep
> inside the timekeeping routines.)
>
> That will get rid of most of the systematic error.  You can tune things by
> watching it drift over following weeks/months.
>
>
> One complication is that the crystal frequency is slightly temperature
> dependent.  Ballpark numbers are 1 PPM per 10 F.  (or maybe 10C?)    A
> system
> will keep better time if there is good air-conditioning and/or you don't
> prop
> open the door very often.
>
> Another possible complication is software bugs/quirks.  Current Linux
> kernels
> get different answers each time they reboot and calibrate the TSC clock.
> The
> differences can be 200 PPM.  A kernel patch would fix that, but that's a
> pain
> if you don't otherwise need to maintain your own kernel.  I've been told
> it's
> been fixed.  I haven't tested it yet.
>
> ------------
>
> Story mode:
>
> Shortly after I got to Xerox in 1976, Ed Taft fixed a bug in the Alto
> timekeeping routines.
>    http://en.wikipedia.org/wiki/Alto_%28computer%29
> The Alto was designed to run at 170 ns.  You buy crystals in MHz rather than
>
> ns.  They got 5.88 MHz crystals.  When the software guys wrote the
> timekeeping code, they used 170 ns rather than 170.068.  That's off by about
>
> 400 PPM, enough to be annoying.  Tweaking one constant was enough to fix
> that.  (I forget if he had to rewrite that code to use double precision.)
>
>