Dr. Hugh Melvin, Dept. of IT, NUI,G 1
Clock Synchronisation for RTS
Dr. Hugh Melvin, Dept. of IT, NUI,G 2
Importance of RTS Clocks
• RealTime implies need for accurate timekeeping
• Examples– Hard RTS
• Distributed Control Systems• Power System / Fly-by-wire
– Soft/Firm RTS• TDM within GSM/POTS
– POTS : SONET/SDH
» Synchronous Opt. Network /Synch. Digital Hierarchy
• MM applications
Dr. Hugh Melvin, Dept. of IT, NUI,G 3
Power System Control
• AS station– Token Bus Synchronisation via Master Clock
• Critical for chronological data logging / fault diagnosis
– Timeslicing for token management– Synchronising 2v3 voter systems
• Need to deliver verdicts simultaneously
• Fault Diagnosis– Impossible without Chronological Data
• Generator Earth Fault / Overcurrent .. – Which came first .. msec level data required
• Power Line Fault Monitoring– Noise burst travels in both directions .. usec level synch
Dr. Hugh Melvin, Dept. of IT, NUI,G 4
Token Bus : Master Clock
U/IA U/IB U/IA U/IB U/IA U/IB U/IA U/IB U/IA U/IBU/IA U/IB U/IA U/IBU/IA U/IB
101N8
AS220E
102N8
AS220E
103N8
AS220E
104N8
AS220E
105N8
AS220E
106N8
AS220E
107N8
AS220E
108N8
AS220E
U/IA U/IB U/IA U/IB U/IA U/IB U/IA U/IB U/IA U/IBU/IA U/IB
126N-BKBus 0
123N-UHRM-Clock
121N16
OS254
112N8
AS220E
111N8
AS220E
110N8
AS220E
109N8
AS220E
U/IA U/IB U/IA U/IB U/IA U/IB U/IA U/IBU/IA U/IB U/IA U/IB
160NS5NAT
PG750
133N8
AS EHF
132N8
AS EHF
131N8
AS EHF
128N8
AS231
125N16R30
141NAT-24Synogate
U/IA U/IB
127 N-BK Bus 1
MasterClock
Dr. Hugh Melvin, Dept. of IT, NUI,G 5
Soft-Firm RTS• POTS operation based on TDM
– PCME1E2..E4 SDH/SONET– Precise synchronisation reqd throughout the network for
correct system operation
• GSM : FDM + TDM– Each FDM channel divided out to 8 users via TDM
• Multimedia Applications– Delay / Jitter Measurement increasingly imp in packet (IP)
networks– More advanced QoS through synchronised time
• Recall G.1010
– Basis of SLA measurement important– Skew Issues between various system/media clocks
Dr. Hugh Melvin, Dept. of IT, NUI,G 6
Dr. Hugh Melvin, Dept. of IT, NUI,G 7
Dr. Hugh Melvin, Dept. of IT, NUI,G 8
Audio-System Clock Skew
Dr. Hugh Melvin, Dept. of IT, NUI,G 9
Computer Clocks• Most commonly consist of quartz crystal and a
counter• Crystal oscillates at defined rate (Hz) which
generates a consistent tick and increments a software counter
• Counter value translated to time standard– UTC (Univ. Coord. Time) .. Based on GMT
• Primary Source: Atomic Clocks TAI (International Atomic Time)
– But requires leap seconds every few years!– UTC = TAI + Leap_Seconds
• Crystal Quality described by Accuracy & Stability
Dr. Hugh Melvin, Dept. of IT, NUI,G 10
Computer Clocks
• Accuracy relates to how close the crystal freq is to rated value– Determined by manufacturing process
• Get what you pay for!
• Stability relates to how frequency varies – Influenced by parameters such as:
• Temperature .. Eg. 2ppm /C• Ageing
– Eg. Cesium Beam: 3 x 10-12 / year
• Noise
Dr. Hugh Melvin, Dept. of IT, NUI,G 11
Computer Clocks• Improved Quality Timekeeping ?
– Option A: Stick with crystals• Precision manufacturing costly• Temperature Compensated Crystal Osc.(TCXO)• Oven Controlled Crystal Osc.(OCXO)
– Option B : • Buy an Atomic Clock
– .. or GPS Receiver (based on atomic clock)
• Most popular approach to providing accurate/stable time
– Option C : Cheaper Approach• Software based approach to discipline cheap crystal
clocks
Dr. Hugh Melvin, Dept. of IT, NUI,G 12
Clock Terminology
• Confusion with terms in literature– Paxson/Mills terminology used here– Offset
• Difference between time reported by clock C, C(t) and true clock (UTC) at true time t.
• Also relative offset between clocks C1and C2 – C1(t) - C2(t)
– Skew• Difference in frequency between clock C and a true clock
(UTC) , C’(t)• Defined in ppm (usec per sec)• +/-12 ppm approx = +/- 1 sec/day• Also relative skew between clocks C1and C2
– C1’(t) - C2
’(t)
Dr. Hugh Melvin, Dept. of IT, NUI,G 13
Dr. Hugh Melvin, Dept. of IT, NUI,G 14
Clock Terminology• Skew
– A large skew rate rapidly increasing offset frequent resynchronisation
– If specify max abs skew rate for clock C of
– Clock should operate within cone of acceptability
• Drift– Rate of change of frequency C’’(t)
• Eg. Ageing influence or change in temperature
– Not usually that significant except over long timescales
– Note linear relationship in previous slide
))(1()()())(1(
121212tttCtCtt
Dr. Hugh Melvin, Dept. of IT, NUI,G 15
Cone of Acceptability
Real Time
Clock Time
Slope = 1 = True Clock
Slope = 1 -
Slope = 1 +
Dr. Hugh Melvin, Dept. of IT, NUI,G 16
Clock Synchronisation
• Perfect clocks do not exist • Eg. PC System Clock NTP Server GPS Receiver
GPS Atomic Clock GPS Master Atomic Clock ??
• Examine two separate scenarios• Localised Cluster of Clocks
– Eg. Power System Control / Fly-by-wire Systems
– Also widely distributed clocks over deterministic network
» Propagation time known (can be compensated for)
» Eg. POTS
• Widely distributed clocks over non-deterministic network– More difficult scenario
– Eg. Internet Synchronisation
Dr. Hugh Melvin, Dept. of IT, NUI,G 17
Clock Synchronisation• Some General Principles
– Fault Tolerance critical• Identify and isolate faulty clocks• Note: A faulty clock is one that does not operate within
cone of acceptability– Cf Clock Quality: May be stable but inaccurate
– Avoid setting clocks backward– Event processing nightmare– OS problems eg. Timers / timeslicing
– Avoid large step changes• Amortize the required change (+/-) over a series of short
intervals (eg. over multiple ticks)
Dr. Hugh Melvin, Dept. of IT, NUI,G 18
Localised Cluster of Clocks• Hardware-based Phase Locked Loops (PLL)
– Oscillator output is aligned to the input signal.– Input signal can come from a
• Master Clock • Combination of outputs from all other clocks
– Input signal used to drive its PLL– Can also compensate for Propagation Delay
variations– Expensive but precise approach
• Similar approach used in widely distributed scenario– GPS / POTS / GSM all use variants of this
approach
Dr. Hugh Melvin, Dept. of IT, NUI,G 19
PLL
VCOComparatorInput Signal
VCO = Voltage Controlled Oscillator
Freq controlled by applied input voltage
Dr. Hugh Melvin, Dept. of IT, NUI,G 20
Widely Distributed Clocks
• More difficult environment if underlying network non deterministic
• Expense of hardware based approach cannot be justified for many Soft-Firm RTS
• Cheap software based approach– Network Time Protocol (NTP)– RFC 1305 (www.ietf.org)
Dr. Hugh Melvin, Dept. of IT, NUI,G 21
Clock Synchronisation : NTP• Network Time Protocol (NTP) synchronises
clocks of hosts and routers in the Internet• Increasingly deployed in the Internet
– Increased need for time synchronisation– Facilitated via always-on Internet connection
• Provides nominal accuracies of low milliseconds on WANs, submilliseconds on LANs, and submicroseconds on workstations using a precision time source such as a cesium oscillator or GPS receiver
• Unix-based NTP daemon now ported to most OS
Dr. Hugh Melvin, Dept. of IT, NUI,G 22
NTPThe NTP architecture, protocol and algorithms have
evolved over the last twenty years to the latest NTP Version 4
• Internet standard protocol for time synchronisation and coordinated time distribution using UTC
• Fault tolerant protocol – automatically selects the best of several available time sources to synchronise with
• Highly scalable – nodes form a hierarchical structure with reference clock(s) at the top– Stratum 0: Time Reference Source
• GPS / GOES (GeoSat) / LORC (LoranC) / ATOM / DTS
– Stratum 1: Primary Time Server
Dr. Hugh Melvin, Dept. of IT, NUI,G 23
Timing Signal
Tim in g S ig n als
G PS Satell ite
G PS/Rad io C lock
N T P S e c o nd ary S e rve rS tratum 3
N T P S e c o nd ary S e rve r(S tra tum 3 )
N T P S e c o nd ary S e rve rS tratum 3
N T P S e c . S e rve rS trat. 2
N T P S e c .S e rve r S tra t. 2
N T P P rim ary S e rve rS tratum 1
NTP System
N T P
Dr. Hugh Melvin, Dept. of IT, NUI,G 24
NTP Operation
• Complex Software comprising various algorithms• Filtering Alg.• Clustering and Intersection Alg.• Combining Alg.• Clock Discipline
NTP Messages
Peer 1
Peer 2
Filter 1
Peer 3
Filter 2
Filter 3
Intersectionand
ClusteringAlgorithms
CombiningAlgorithm
Loop Filter
VFO
P/F-Lock Loop
Dr. Hugh Melvin, Dept. of IT, NUI,G 26
Client Server Mode
• UDP/IP packets for data transfer– Several packet exchanges between client/server– Client
• originate timestamp A within packet being sent.
– Server receives such a packet:• receive timestamp B• transmit timestamp C
– Client• Processes A,B,C as well as final packet arrival D• Determine offset and Round Trip Delay (RTD) • Note: RTD != RTT
Dr. Hugh Melvin, Dept. of IT, NUI,G 27
NTP Operation
C 3.59.022
D 3.59.032
B 3.59.020
A 3.59.000
15 ms 15 ms
Symmetric Network : 15 ms each way (actual delay)
RTD = (D - A) – (C – B) = 32 – 2 = 30 msec (RTT =?)
Offset = ½[(B-A) - (D-C)] = (20 – 10)/2 = 5 ms
Dr. Hugh Melvin, Dept. of IT, NUI,G 35
Clock Discipline
• Recall– No time reversal!– Avoid step changes
• Hybrid phase/frequency-lock (PLL/FLL) feedback loop
• PLL/FLL Mode: Depends on polling interval
Dr. Hugh Melvin, Dept. of IT, NUI,G 37
Clock Models
• Unix Clock Model• settimeofday( ), adjtime( )• Kernel variables tick , tickadj• adjtime adjusts clock every tick
– Can amortise reqd change gradually by making adjustment every tick eg. every 10 msec
– Note: Newer Unix/Linux kernels 1000Hz 1msec
• 3 clock rates– Normal rate .. Add 10 msec every tick (100 Hz)– Normal Rate +/- tickadj – Eg. If tickadj = 5us Normal Rate +/- 500 ppm
Dr. Hugh Melvin, Dept. of IT, NUI,G 38
NTP Operation • NTP adjusts every sec via adjtime
– Eg. If clock skew is +100 ppm & tickadj=5us• NTP will operate to keep clock effectively running at
correct rate– Normal Rate - 500 ppm over 0.2 sec
– Normal Rate for 0.8 sec Effective skew = 0 ppm
– Results in sawtooth – pattern
• Newer Unix Kernels have advanced NTP features– ntp_adjtime( ), ntp_gettime()– Eliminates the sawtooth pattern
Dr. Hugh Melvin, Dept. of IT, NUI,G 39
NTP Implementation
• Install NTP • Set up ntp.conf file
– List of servers that you wish to connect to– Redundancy & Path Diversity & Low RTD
• Start up NTP daemon ntpd• File ntp.drift records clock skew• Other utilities
– ntpq, ntpdate– See www.ntp.org
Dr. Hugh Melvin, Dept. of IT, NUI,G 40
Refid:
DCF: 77.5 KHz Radio Signal
PTB: German time signal
Dr. Hugh Melvin, Dept. of IT, NUI,G 41
Dr. Hugh Melvin, Dept. of IT, NUI,G 42
Time difference
Dr. Hugh Melvin, Dept. of IT, NUI,G 43
Server Details
• when: no of sec since last response
• poll : interval between queries
• reach : Reachability in octal– 11111111 = 3778 = max
– 11101110 = 3568 last + 5th probe lost
• Symbol to LHS of server– * : Synch Source – survivor with smallest dispersion
– + :other candidates included in final combination alg
– - : Discarded by clustering alg
– x : Falseticker acc to intersection alg
Dr. Hugh Melvin, Dept. of IT, NUI,G 44
Dr. Hugh Melvin, Dept. of IT, NUI,G 45
NTP Robustness Issues
• Redundancy
• Path Diversity
• Symmetric Networks
• Proximity to Primary Reference Sources– See results
• OS & Network Load– Platform Dependencies
Dr. Hugh Melvin, Dept. of IT, NUI,G 46
NTP Operation : Asymmetry
C 3.59.017
D 3.59.032
B 3.59.015
A 3.59.000
10 ms 20 ms
Offset still 5 ms but Asymmetric Network
RTD = (D - A) – (C – B) = 32 – 2 = 30 msec
Offset = ½[(B-A) - (D-C)] = (15 – 15)/2 = 0 ms .. Error
Dr. Hugh Melvin, Dept. of IT, NUI,G 47
NTP Operation : Asymmetry
C 3.59.017
D 3.59.032
B 3.59.015
A 3.59.000
15 ms 15 ms
NTP’s Symmetric view of Asymmetric Network
RTD = (D - A) – (C – B) = 32 – 2 = 30 msec
Offset = ½[(B-A) - (D-C)] = (15 – 15)/2 = 0 ms !
Exercise: What is the maximum error in this calculation?
Dr. Hugh Melvin, Dept. of IT, NUI,G 48
Dr. Hugh Melvin, Dept. of IT, NUI,G 49
Dr. Hugh Melvin, Dept. of IT, NUI,G 50
Server Offsets: Problem?