[time-nuts] Question on LH

[Mark Sims]

It depends upon your lat and lon,  but figure about 10 feet.   Basically earth circumference is 24,000 miles * 5280 feet per mile divided by (2**24) (bits in the sign+significand of a 32 bit float).  The math works out to 7.5 feet,  but one always has to pay some imprecise math tax...

One can see the quantization effect with Heather V5.   Heather can calculate and display a lat/lon scattergram of the receiver fixes as they come in.  The default is a 10x10 grid at 3 meters per division.  The lat/lon coordinates are "plotted" in an internal bit map.  The  coordinates are also stored as 32-bit floats in the plot queue.   If one tells Heather to change the scale factor (meters/division) of the scattergram,  it is re-created from the low-res plot queue data.

gpsd1.gif is from the full-res double precision data the receiver sends,  gpsd2.gif is from the single precision floating point plot queue data.  The center points of two scattergrams are slightly different because it was changed to the current GPS coordinate when the plot was re-generated.  The pixel colors change every hour.

--------------------

>  What does the chop-off translate to in terms of distance?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gpsd1.gif
Type: image/gif
Size: 54333 bytes
Desc: gpsd1.gif
URL: <http://www.febo.com/pipermail/time-nuts/attachments/20161208/7ff4c236/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gpsd2.gif
Type: image/gif
Size: 13511 bytes
Desc: gpsd2.gif
URL: <http://www.febo.com/pipermail/time-nuts/attachments/20161208/7ff4c236/attachment-0003.gif>