[chrony-users] Requesting some insight regarding preferred servers and RTCSYNC/RTCFILE

[Martin Janse]

Hi all,



I have recently taken over the development of an embedded project. It runs a custom Linux build on a Colibri T30 system, including Chrony.

The device has an RTC on-board and is also connected to a LAN with multiple of the device. However, the LAN is not necessarily connected to internet.

The main purpose of the device is monitoring and logging, applying some statistical analysis on data and wrapping it into accessible data formats. For this purpose, somewhat accurate time (a few minutes off max) is necessary. The devices are placed in hard-to-reach locations and should be able work autonomously for years at a time. The devices can reboot on a moments notice with unknown intervals, at least once every 48 hours.


The device with its current settings seems to work as expected but I have my doubts that the device is really working as expected or if it just seems to be close enough. I have two main concerns I would like to have straightened out that I could not clearly do away with by reading the manual and some installation guides. The current configuration file is as follows:



refclock SOCK /var/run/chrony.DUM1.sock refid DUM1
refclock SOCK /var/run/chrony.RTC.sock refid RTCC delay 1e-3

pool 0.nl.pool.ntp.org iburst
pool 1.nl.pool.ntp.org iburst
pool 2.nl.pool.ntp.org iburst
pool 3.nl.pool.ntp.org iburst

allow all
manual
local stratum 10
makestep 300 -1
logdir /var/log/chrony
log measurements statistics tracking refclocks rtc
logchange 60
cmdallow 127.0.0.1
pidfile /var/run/chronyd.pid




My first concern lies with the RTC. In the current configuration, neither RTCSYNC nor RTCFILE are defined. Does Chrony still correct the system time while it's running? Or only at boot? Or never?


Secondly, a new feature needs to be added to the system. Other than syncing to the RTC or other devices on the network, an NTP server needs to be added that has precedence over any other source. I saw in the manual that the parameter 'prefer' can be added to the server, causing Chrony to prefer that server. However, is there a scenario where Chrony still picks a non-preferred source? Is there a degree of 'quality' that the preferred server needs to have in order to have guaranteed precedence?



I understand that the current configuration is not exactly 'as it should be'. The development of this embedded system was really messy and the engineer who made this is no longer working for the company. He left no documentation behind on this part, so I can only guess what exactly the reasons were for setting stratum to 10 or the makestep to -1. But I don't want to change anything that is not necessary because failure of logging is unacceptable.


If anyone can address my concerns and/or explain why this configuration works, I would be very grateful.


kind regards,



Martin Janse

[Miroslav Lichvar]

Post by Martin Janse
My first concern lies with the RTC. In the current configuration, neither RTCSYNC nor RTCFILE are defined. Does Chrony still correct the system time while it's running? Or only at boot? Or never?

If no rtcfile or rtcsync is in the config, chronyd will not care about
the RTC at all. It will not track its frequency offset relative to the
system time, and correct for drift with the -s option, or tell the
kernel to synchronize it. If the RTC drifts, the initial time on start
will be wrong.

I guess that's what you want as you seem to synchronize the system
clock to (the same?) RTC using a SOCK refclock.

Post by Martin Janse
Secondly, a new feature needs to be added to the system. Other than syncing to the RTC or other devices on the network, an NTP server needs to be added that has precedence over any other source. I saw in the manual that the parameter 'prefer' can be added to the server, causing Chrony to prefer that server. However, is there a scenario where Chrony still picks a non-preferred source? Is there a degree of 'quality' that the preferred server needs to have in order to have guaranteed precedence?

chronyd can select a non-prefered source if it doesn't agree with
other sources (falseticker). If you had only two sources, an RTC and
an NTP server, it would be easy for them to not agree and chronyd
would stop synchronization.

To always select the source, it needs to be specified with the "trust"
option.

[Bill Unruh]

William G. Unruh __| Canadian Institute for|____ Tel: [phone]
Physics&Astronomy _|___ Advanced Research _|____ Fax: [phone]
UBC, Vancouver,BC _|_ Program in Cosmology |____ ***@physics.ubc.ca
Canada V6T 1Z1 ____|____ and Gravity ______|_ www.theory.physics.ubc.ca/

Post by Miroslav Lichvar

Post by Martin Janse
My first concern lies with the RTC. In the current configuration, neither RTCSYNC nor RTCFILE are defined. Does Chrony still correct the system time while it's running? Or only at boot? Or never?

If no rtcfile or rtcsync is in the config, chronyd will not care about
the RTC at all. It will not track its frequency offset relative to the
system time, and correct for drift with the -s option, or tell the
kernel to synchronize it. If the RTC drifts, the initial time on start
will be wrong.
I guess that's what you want as you seem to synchronize the system
clock to (the same?) RTC using a SOCK refclock.

No idea why youu would want to do that. Your rtc drifts, and in particular
drifts when the sytem is off.
On boot chrony reads the rtc when chrony is first started and sets the system
time to that. After that the rtc is ignored except that the system will
calculate the rate and offset of the rtc. The rtc typically is a pretty shitty
source for time. It can drift badly.

Post by Miroslav Lichvar

Post by Martin Janse
Secondly, a new feature needs to be added to the system. Other than syncing to the RTC or other devices on the network, an NTP server needs to be added that has precedence over any other source. I saw in the manual that the parameter 'prefer' can be added to the server, causing Chrony to prefer that server. However, is there a scenario where Chrony still picks a non-preferred source? Is there a degree of 'quality' that the preferred server needs to have in order to have guaranteed precedence?

chronyd can select a non-prefered source if it doesn't agree with
other sources (falseticker). If you had only two sources, an RTC and

I think that sentence is open to confusion. If chrony sees that majority of
the clocks show one time and a minority show substantially different times (ie outside
the error bodget of the majority) , then that minority are false tickers. If
the prefer souce is in that minority, then chrony does not trust it and uses
one of the majority as its source.

are

Post by Miroslav Lichvar
an NTP server, it would be easy for them to not agree and chronyd
would stop synchronization.

I would expect chrony to choose the prefer option in that case. Is that wrong?

Post by Miroslav Lichvar
To always select the source, it needs to be specified with the "trust"
option.

Or just have it as the only source.

Post by Miroslav Lichvar
--
Miroslav Lichvar
--
with "unsubscribe" in the subject.
with "help" in the subject.

[Bill Unruh]

With bad sources, chrony cannot give good time. RTC tends to be bad -- it can
have large drifts (100 PPM is not unheard of which would put it out be seconds
per day, which would just grow on successive reboots). Inaccessible locations
also suggests to me varying temperatures which would make the above worse
(more unpredictable). And in specific, the various rtcs on the various devices
are apt to have wildly different drift rates.

You say it only may be connected to the internet. Thus your pool sources are
highly unreliable. You do not say what the source DUM1 is. Is it anything?

You do not say how your system can query other devices on the lan.At present
it is either the internet, the unkown source DUM1, or the RTC.
It does not query any of the other devices. You need to put a reliable source
of time onto the lan, and have the machines query that, or those (eg have
three GPS disciplined devices on the lan and have the machines query those.

Post by Martin Janse
Hi all,
I have recently taken over the development of an embedded project. It runs a custom Linux build
on a Colibri T30 system, including Chrony.
The device has an RTC on-board and is also connected to a LAN with multiple of the device.
However, the LAN is not necessarily connected to internet.
The main purpose of the device is monitoring and logging, applying some statistical analysis on
data and wrapping it into accessible data formats. For this purpose, somewhat accurate time (a
few minutes off max) is necessary. The devices are placed in hard-to-reach locations and should
be able work autonomously for years at a time. The devices can reboot on a moments notice with
unknown intervals, at least once every 48 hours.
The device with its current settings seems to work as expected but I have my doubts that the
device is really working as expected or if it just seems to be close enough. I have two main
concerns I would like to have straightened out that I could not clearly do away with by reading
refclock SOCK /var/run/chrony.DUM1.sock refid DUM1
refclock SOCK /var/run/chrony.RTC.sock refid RTCC delay 1e-3
pool 0.nl.pool.ntp.org iburst
pool 1.nl.pool.ntp.org iburst
pool 2.nl.pool.ntp.org iburst
pool 3.nl.pool.ntp.org iburst
allow all
manual
local stratum 10
makestep 300 -1
logdir /var/log/chrony
log measurements statistics tracking refclocks rtc
logchange 60
cmdallow 127.0.0.1
pidfile /var/run/chronyd.pid
My first concern lies with the RTC. In the current configuration, neither RTCSYNC nor RTCFILE
are defined. Does Chrony still correct the system time while it's running? Or only at boot? Or
never? 
Secondly, a new feature needs to be added to the system. Other than syncing to the RTC or other
devices on the network, an NTP server needs to be added that has precedence over any other
source. I saw in the manual that the parameter 'prefer' can be added to the server, causing
Chrony to prefer that server. However, is there a scenario where Chrony still picks a
non-preferred source? Is there a degree of 'quality' that the preferred server needs to have in
order to have guaranteed precedence?
I understand that the current configuration is not exactly 'as it should be'. The development
of this embedded system was really messy and the engineer who made this is no longer working
for the company. He left no documentation behind on this part, so I can only guess what exactly
the reasons were for setting stratum to 10 or the makestep to -1. But I don't want to change
anything that is not necessary because failure of logging is unacceptable. 
If anyone can address my concerns and/or explain why this configuration works, I would be very
grateful.
kind regards,
Martin Janse

[Martin Janse]

Thank you Miroslav and Bill for your responses.



I will try to elaborate as much as I can.


First off, I do understand that keeping time on a closed system without a dedicated, reliable time source is basically impossible and simply means that the time of that network will be unreliable. It is not clear to me yet, if defining RTCSYNC would improve the reliability by some degree. As far as I understand, RTCSYNC does not actually change the RTC time but rather compensates for the drift of the RTC when setting the local time. Since the system has an uptime of >99%, would this help?


Your assumptions on the device seem correct as well; the device is mostly applied offshore. Temperatures (measured on the Colibri Chip) can range from 20 to 70 degrees C. The pool sources are indeed unreliable because the network is most likely only connected to the terminal of the operator. However, some of the networks do have their own time source. I think it is important to tell the customers that having a dedicated time source is basically a must and not a part of our package.


As for the querying of other devices; unfortunately, I do not know how this works either. I can see that the DUM1 socket is defined together with the RTCC socket, but what they are used for I do not know. I imagine the RTCC is the hardware RTC on the device, but it seems the DUM1 socket is not used, because on every device I have looked, it has a maximum frequency skew and standard deviation. It is not being used by any device, no matter the network circumstances; it's either a source from the pool or the RTCC. I know that the system was designed to have the following priority:


1. pool, if internet is available

2. time from other embedded device

3. RTC on device itself


and the request is to expand this list by having a customer-defined source at the top.


Therefore I think adding a customer-defined source (max 1) with the 'trust' parameter would address the request of the customer sufficiently. What I can tell from your responses and the manual text is that this would have no consequences on the rest of the Chrony operations or the RTC. I cannot set the client source as the only source, because of redundancy reasons. Eliminating the pool however, will probably make no difference at all. Still, I'd rather not mess with these settings unless I really have to. Having poorly documented code and system design is already hard enough to deal with.


Thanks again for your time. I think I can make a decently informed proposal out of this!


regards,



Martin Janse

________________________________
From: Bill Unruh <***@physics.ubc.ca>
Sent: Monday, October 29, 2018 3:16:53 PM
To: chrony-***@chrony.tuxfamily.org
Subject: Re: [chrony-users] Requesting some insight regarding preferred servers and RTCSYNC/RTCFILE

With bad sources, chrony cannot give good time. RTC tends to be bad -- it can
have large drifts (100 PPM is not unheard of which would put it out be seconds
per day, which would just grow on successive reboots). Inaccessible locations
also suggests to me varying temperatures which would make the above worse
(more unpredictable). And in specific, the various rtcs on the various devices
are apt to have wildly different drift rates.

You say it only may be connected to the internet. Thus your pool sources are
highly unreliable. You do not say what the source DUM1 is. Is it anything?

You do not say how your system can query other devices on the lan.At present
it is either the internet, the unkown source DUM1, or the RTC.
It does not query any of the other devices. You need to put a reliable source
of time onto the lan, and have the machines query that, or those (eg have
three GPS disciplined devices on the lan and have the machines query those.

Post by Martin Janse
Hi all,
I have recently taken over the development of an embedded project. It runs a custom Linux build
on a Colibri T30 system, including Chrony.
The device has an RTC on-board and is also connected to a LAN with multiple of the device.
However, the LAN is not necessarily connected to internet.
The main purpose of the device is monitoring and logging, applying some statistical analysis on
data and wrapping it into accessible data formats. For this purpose, somewhat accurate time (a
few minutes off max) is necessary. The devices are placed in hard-to-reach locations and should
be able work autonomously for years at a time. The devices can reboot on a moments notice with
unknown intervals, at least once every 48 hours.
The device with its current settings seems to work as expected but I have my doubts that the
device is really working as expected or if it just seems to be close enough. I have two main
concerns I would like to have straightened out that I could not clearly do away with by reading
refclock SOCK /var/run/chrony.DUM1.sock refid DUM1
refclock SOCK /var/run/chrony.RTC.sock refid RTCC delay 1e-3
pool 0.nl.pool.ntp.org iburst
pool 1.nl.pool.ntp.org iburst
pool 2.nl.pool.ntp.org iburst
pool 3.nl.pool.ntp.org iburst
allow all
manual
local stratum 10
makestep 300 -1
logdir /var/log/chrony
log measurements statistics tracking refclocks rtc
logchange 60
cmdallow 127.0.0.1
pidfile /var/run/chronyd.pid
My first concern lies with the RTC. In the current configuration, neither RTCSYNC nor RTCFILE
are defined. Does Chrony still correct the system time while it's running? Or only at boot? Or
never?
Secondly, a new feature needs to be added to the system. Other than syncing to the RTC or other
devices on the network, an NTP server needs to be added that has precedence over any other
source. I saw in the manual that the parameter 'prefer' can be added to the server, causing
Chrony to prefer that server. However, is there a scenario where Chrony still picks a
non-preferred source? Is there a degree of 'quality' that the preferred server needs to have in
order to have guaranteed precedence?
I understand that the current configuration is not exactly 'as it should be'. The development
of this embedded system was really messy and the engineer who made this is no longer working
for the company. He left no documentation behind on this part, so I can only guess what exactly
the reasons were for setting stratum to 10 or the makestep to -1. But I don't want to change
anything that is not necessary because failure of logging is unacceptable.
If anyone can address my concerns and/or explain why this configuration works, I would be very
grateful.
kind regards,
Martin Janse

[Miroslav Lichvar]

Post by Martin Janse
First off, I do understand that keeping time on a closed system without a dedicated, reliable time source is basically impossible and simply means that the time of that network will be unreliable. It is not clear to me yet, if defining RTCSYNC would improve the reliability by some degree. As far as I understand, RTCSYNC does not actually change the RTC time but rather compensates for the drift of the RTC when setting the local time. Since the system has an uptime of >99%, would this help?

rtcsync just tells chronyd to tell the kernel that it should keep the
RTC synchronized to the system time. If the system clock was
synchronized to the RTC using the SOCK refclock, it would create a
synchronization loop. That would be bad.

Post by Martin Janse
As for the querying of other devices; unfortunately, I do not know how this works either. I can see that the DUM1 socket is defined together with the RTCC socket, but what they are used for I do not know.

There used to be a bug where chronyd didn't work with multiple SOCK
refclocks. A workaround was to specify dummy reflocks before the real
ones. Maybe this is a remnant of the workaround.