Date post: | 02-Jul-2015 |
Category: |
Technology |
Upload: | romain-jacotin |
View: | 425 times |
Download: | 4 times |
Agenda
• MARZULLO’ agreement algorithm
– Description
– Golang source code
• DTSS intersection algorithm (NTP)
– Description
– Golang source code
• References
MARZULLO's agreement algorithm
B
D
C
A
10 30 38 46 60 1229678
Offset
Marzullo's algorithm, invented by Keith Marzullo for his Ph.D. dissertation in 1984, is an agreement algorithm used to select sources for estimating accurate time from a number of noisy time sources.
The Marzullo's algorithm is also used to compute the relaxed intersection of n boxes (or more generally n subsets of Rn), as required by several robust set estimation methods.
Marzullo's algorithm is efficient in terms of time for producing an optimal value from a set of estimates with confidence intervals where the actual value may be outside the confidence interval for some sources. In this case the best estimate is taken to be the smallest interval consistent with the largest number of sources.
http://cseweb.ucsd.edu/~marzullo/
Keith MARZULLO’s algorithmMarzullo's algorithm begins by preparing a table of the sources, sorting it and then searching (efficiently) for the intersections of intervals.For each source there is a range [c−r,c+r] defined by c ± r.For each range the table will have two tuples of the form {offset,type} :• One tuple will represent the beginning of the range, marked with type = −1 as {c−r,−1}• and the other will represent the end with type = +1 as {c+r,+1}
The description of the algorithm uses the following variables: best (largest number of overlapping intervals found), cnt (current number of overlapping intervals), beststart and bestend (the beginning and end of best interval found so far), i (an index), and the table m of tuples.
1. Build m the table of tuples.2. Sort the table m by the offset. (If two tuples with the same offset but opposite types exist, indicating that one interval
ends just as another begins, then a method of deciding which comes first is necessary. Such an occurrence can be considered an overlap with no duration, which can be found by the algorithm by putting type −1 before type +1. If such pathological overlaps are considered objectionable they can be avoided by putting type +1 before −1 in this case.)
3. [initialize] best=0 cnt=04. [loop] go through each tuple in the table m in ascending order
• [current number of overlapping intervals] cnt=cnt−type[i]• if cnt>best then best=cnt beststart=offset[i] bestend=offset[i+1]
5. [end loop] return [beststart,bestend] as optimal interval. The number of false sources (ones which do not overlap the optimal interval returned) is the number of sources minus the value of best.
commentary: the next tuple, at [i+1], will either be an end of an interval (type=+1) in which case it ends this best interval, or it will be a beginning of an interval (type=−1) and in the next step will replace best.
ambiguity: unspecified is what to do if best=cnt. This is a condition of a tie for greatest overlap. The decision can either be made to take the smaller of bestend−beststart or offset[i+1]−offset[i] or just take an arbitrary one of the two equally good entries.
Confidence intervals
B
D
C
A
10 30 38 46 60 1229678
Offset
-1
-1
-1
-1+1
+1
+1
+1
Golang algorithm
type tuple struct {
segname string // optional : ID of the interval
offset int // the offset value of the start or end interval
tp int // is -1 for a start interval and +1 for an end interval
}
type tuples []tuple
// functions to give sort capability to tuples with the “sort” package
func (ts tuples) Len() int {
return len(ts)
}
func (ts tuples) Swap(i, j int) {
ts[i], ts[j] = ts[j], ts[i]
}
func (ts tuples) Less(i, j int) bool {
return ts[i].offset < ts[j].offset
}
m := tuples{
{"A", 10, -1},
{"A", 30, 1},
{"B", 38, -1},
{"B", 78, 1},
{"C", 46, -1},
{"C", 96, 1},
{"D", 60, -1},
{"D", 122, 1},
}
B
D
C
A
10 30 38 46 60 1229678
Offset
-1
-1
-1
-1+1
+1
+1
+1
Golang algorithm
// 0. Build the table of tuples
m := tuples{
{"A", 10, -1},
{"A", 30, 1},
{"B", 38, -1},
{"B", 78, 1},
{"C", 46, -1},
{"C", 96, 1},
{"D", 60, -1},
{"D", 122, 1},
}
// 1. Sort the table by the offset.
sort.Sort(m)
// 2. [initialize] best=0 cnt=0
best := 0
cnt := 0
Golang algorithm
// 3. [loop] go through each tuple in the table in ascending order
// 4. [current number of overlapping intervals] cnt=cnt−type[i]
// 5. if cnt>best then best=cnt beststart=offset[i] bestend=offset[i+1]
var beststart int
var bestend int
for i, t := range m {
cnt = cnt - t.tp
if cnt > best {
best = cnt
beststart = t.offset
bestend = m[i+1].offset
}
}
// 6. [end loop] return [beststart,bestend] as optimal interval.
// The number of false sources (ones which do not overlap the optimal interval
// returned) is the number of sources minus the value of best.
fmt.Println(”Best source quantity = ", best)
fmt.Println("False source quantity = ", len(m)/2 - best)
fmt.Println("Beststart = ", beststart)
fmt.Println("Bestend = ", bestend)
Golang algorithm
Agenda
• MARZULLO’ agreement algorithm
– Description
– Golang source code
• DTSS intersection algorithm (NTP)
– Description
– Golang source code
• References
DTSS intersection algorithm (NTP)
B
D
C
A
10 30 38 46 60 1229678
Offset
The intersection algorithm is an agreement algorithm used to select sources for estimating accurate time from a number of noisy time sources, it forms part of the modern Network Time Protocol (NTP).It is a modified form of Marzullo's algorithm.
While Marzullo's algorithm will return the smallest interval consistent with the largest number of sources, the returned interval does not necessarily include the center point (calculated offset) of all the sources in the intersection.
The Intersection algorithm returns an interval that includes that returned by Marzullo's algorithm but may be larger since it will include the center points. This larger interval allows using additional statistical data to select a point within the interval, reducing the jitter in repeated execution.
2058
70 91
NTP
DTSS
DTSS intersection algorithm (NTP)
Given M intervals of the form c ± r (which means [c−r,c+r]), the algorithm seeks to find an interval with M−fsources. The value f is referred to as the number of falsetickers, those sources which are in error (the actual value is outside the confidence band). The best estimate is that which assumes the least number of falsetickers, f. The results will be considered valid if f < M/2, otherwise the algorithm will return failure instead of an interval.
The intersection algorithm begins by creating a table of tuples {offset,type}. For each interval there are three entries: the lower endpoint, the midpoint and the upper endpoint, labelled with types −1, 0 and +1 respectively. Thus the interval c ± r results in the entries {c−r,−1}, {c,0} and {c+r,+1}. These entries are then sorted by offset.
Variables: This algorithm uses f as number of false tickers, endcount and midcount are integers. lower and upperare values of offsets, i and j (an index), and the table m of tuples.
1. The algorithm begins by initializing a value f and counters endcount and midcount to zero.2. Then, starting from the lowest offset of the sorted table of tuples, for each tuple {offset, type} the value
of type is subtracted from the counter endcount, which is the number of intersections. If type is zero, increment the value of midcount, which is the number of falsetickers. If endcount >= M-f for some entry, offset of that entry becomes the lower offset of the intersection; otherwise, f is increased by one and the above procedure is repeated.
3. Without resetting f or midcount, a similar procedure is used to find the upper offset, except that the value of type is added to the midcounter counter.
4. If after both offsets have been determined midcount <= f, the procedure continues having found M-ftruechimers; otherwise, f is increased by one and the entire procedure is repeated.
Confidence intervals
B
D
C
A
10 20 30 38 4660
58 70 91 1229678
Offset
-1
-1
-1
-1+1
+1
+1
+1
0
0
0
0
Golang algorithm
type tuple struct {
segname string // optional : ID of the interval
offset int // the offset value of the start or end interval
tp int // lower, midpoint, upper endpoint are types −1, 0, +1
}
type tuples []tuple
// functions to give sort capability to tuples with the “sort” package
func (ts tuples) Len() int {
return len(ts)
}
func (ts tuples) Swap(i, j int) {
ts[i], ts[j] = ts[j], ts[i]
}
func (ts tuples) Less(i, j int) bool {
return ts[i].offset < ts[j].offset
}
B
D
C
A
10 20 30 38 4660
58 70 91 1229678
m := tuples{
{"A", 10, -1},
{"A", 20, 0},
{"A", 30, 1},
{"B", 38, -1},
{"B", 58, 0},
{"B", 78, 1},
{"C", 46, -1},
{"C", 70, 0},
{"C", 96, 1},
{"D", 60, -1},
{"D", 91, 0},
{"D", 122, 1},
}
Golang algorithm
Offset
-1
-1
-1
-1+1
+1
+1
+1
0
0
0
0
// Build the table of tuples and Sort the table by the offset
var endcount, midcount, lower, upper, f, M int
m := tuples{
{"A", 10, -1},
{"A", 20, 0},
{"A", 30, 1},
{"B", 38, -1},
{"B", 58, 0},
{"B", 78, 1},
{"C", 46, -1},
{"C", 70, 0},
{"C", 96, 1},
{"D", 60, -1},
{"D", 91, 0},
{"D", 122, 1},
}
sort.Sort(m)
M = len(m)/3
Golang algorithm
// calling all truechimers
for f = 0; f < M/2; f++ {
endcount = 0
midcount = 0
// find low endpoint
for _, t := range m {
endcount -= t.tp
lower = t.offset
if endcount >= (M - f) { break }
if t.tp == 0 { midcount++ }
}
endcount = 0
// find high endpoint
for j := len(m)-1; j >= 0; j-- {
endcount += m[j].tp
upper = m[j].offset
if endcount >= (M-f) { break }
if m[j].tp == 0 { midcount++ }
}
// continue until all falsetickers found
if midcount <= f { break }
}
Golang algorithm
/ do we found an intersection ?
if lower <= upper {
fmt.Println("Best source quantity = ", M-f)
fmt.Println("False source quantity = ", f)
fmt.Println("Beststart = ", lower)
fmt.Println("Bestend = ", upper)
} else {
fmt.Println("Failed because false source quantity = ", f)
}
Golang algorithm
Agenda
• MARZULLO’ agreement algorithm
– Description
– Golang source code
• DTSS intersection algorithm (NTP)
– Description
– Golang source code
• References
References
Keith MARZULLOhttp://cseweb.ucsd.edu/~marzullo/
Maintaining the time in a distributed sytem (MARZULLO/OWICKI) http://infolab.stanford.edu/pub/cstr/reports/csl/tr/83/247/CSL-TR-83-247.pdf
Marzullo’s algorithmhttp://en.wikipedia.org/wiki/Marzullo's_algorithm
DTSS intersection algorithm (Digital Time Synchronization Service)http://en.wikipedia.org/wiki/Intersection_algorithmhttp://www.eecis.udel.edu/~mills/ntp/html/select.html
Network Time Protocol (NTP)http://tools.ietf.org/pdf/rfc5905.pdfhttp://www.ntp.org/
The Go Programming Languagehttp://www.golang.org/