Alberto Pace Director of the CERN School of Computing Welcome to iCSC2014 the seventh edition of the Inverted School, “Where Students turn into Teachers”. At regular CERN Schools of Computing, the sum of the knowledge of the students often exceeds the one of lecturer teaching, and that it is frequent to find in the room real experts on particular topics. This is the idea behind iCSCs. During the main school, students give lightning talks and make proposals via an electronic discussion forum, from which a programme for the inverted school is inspired. This year’s programme relies from contribution selected from the main school in Nicosia, Cyprus in August 2013. It focuses challenging and innovative topics: “LAN programming for building distributed systems”, “Web API”, “Read-out electronics: where data come from”, “Machine learning and data mining” and a topic to bridge physics with computing on “A journey from quark to jet”. We are indebted to the lecturers at the main CSC who acted as mentors. My main thanks go to all those who developed ideas and proposals and to those actually lecturing. This is their school and I am confident all will go very well. It is already the seventh edition and do not hesitate, you the attendees, to comment and advise us on how to improve it. Enjoy the school. 1
Transcript
Alberto Pace Director of the CERN School of Computing
Welcome to iCSC2014 the seventh edition of the Inverted School, “Where Students turn into Teachers”.
At regular CERN Schools of Computing, the sum of the knowledge of the students often exceeds the one of lecturer teaching, and that it is frequent to find in the room real experts on particular topics. This is the idea behind iCSCs.
During the main school, students give lightning talks and make proposals via an electronic discussion forum, from which a programme for the inverted school is inspired.
This year’s programme relies from contribution selected from the main school in Nicosia, Cyprus in August 2013. It focuses challenging and innovative topics: “LAN programming for building distributed systems”, “Web API”, “Read-out electronics: where data come from”, “Machine learning and data mining” and a topic to bridge physics with computing on “A journey from quark to jet”.
We are indebted to the lecturers at the main CSC who acted as mentors. My main thanks go to all those who developed ideas and proposals and to those actually lecturing. This is their school and I am confident all will go very well. It is already the seventh edition and do not hesitate, you the attendees, to comment and advise us on how to improve it.
Enjoy the school.
1
2
Contents
Foreword 1
History of iCSCs 5
Schedule 7
iCSC 2014 Lecturers 9
iCSC 2014 Mentors Introduction to the inverted CSC
11
13
Lecture 1 LAN Programming - The basics 15
Lecture 2 Is your web API truly RESTful (and does it matter) 25
Lecture 3 Building highly distributed systems within 5 minutes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
39
Lecture 4 From Quark to Jet: A Beautiful Journey Beauty physics, tracking and large-scale distributed computing in HEP
53
Lecture 5 Read-Out Electronics: where data come from 65
Lecture 6 From Quark to Jet: A Beautiful Journey Making a jet, classifying a jet, and personal scale computing in HEP
77
Lecture 7 Read-Out Electronics: where data come from 85 Lecture 8 Lecture 9
Introduction to machine learning and data mining Self organizing maps. A visualization technique with data dimension reduction
95
107
3
4
History of iCSCs iCSC2005 23-25 February 2005, CERN
Data Management and DataBase Technologies
Advanced Software Development & Engineering
Web Services in Distributed Computing
iCSC2006 6-8 March 2006, CERN
Computational Intelligence for HEP Data Analysis
The Art of Designing Parallel Applications
Software Testing: Fundamentals and Best Practices
iCSC2008 3-5 March 2008, CERN
Towards Reconfigurable High-Performance Computing
Special topics
iCSC2010 8-9 March 2010, CERN
Software management and optimization
Visualisation
System monitoring
iCSC2011 3-4 March 2011, CERN
Virtualization and clouds
Cryptography
Modern software engineering
5
iCSC2013 25-26 February 2013, CERN
GPU computing and its applications in HEP
Introduction to Computer Vision Testing methods and tools for
large scale distributed systems
How the LHC experiment have interpreted the Grid distributed computing model
6
iCSC2014 Schedule
Monday 24 February
31/3-004
14:00-14:10
Welcome
14:10-14:20
Introduction to the inverted CSC
14:20-15:20
Lecture 1 LAN Programming – The basics Jonas Kunze
15:20-15:50
Coffee break
15:50-16:50
Lecture 2 Is your web API truly RESTful
(and does it matter) Josef Hammer
16:50-17:50
Lecture 3 Building highly distributed systems within 5 minutes Jonas Kunze
Tuesday 25 February
31/3-004
09:00-10:00
Lecture 4 From Quark to Jet: A Beautiful Journey – Lecture 1 Tyler Mc Millan Dorland
10:00-11:00
Lecture 5 Read-Out Electronics: where data come from – Lecture 1 Francesco Messi
11:00-11:30
Coffee break
11:30-12:30
Lecture 6 From Quark to Jet: A Beautiful Journey – Lecture 2 Tyler Mc Millan Dorland
12:30-13:30
Lunch
13:30-14:30
Lecture 7 Read-Out Electronics: where data come from – Lecture 2 Francesco Messi
14:30-15:30
Lecture 8 Introduction to machine learning and data mining Juan Lopez Gonzalez
15:30-16:00
Coffee break
16:00-17:00
Lecture 9 Self organizing maps
A visualization technique with data dimension reduction Juan Lopez Gonzalez
7
8
iCSC2014 Lecturers Biographies
DORLAND Tyler DESY, Hamburg - Germany
My first exposure to working with computers came from blowing into the cartridge of my Nintendo back in 1989. A few years later I had upgraded and was learning the intricacies of my 486 I spent all summer saving for. Since then, I have fiddled around with careers in music, football, and ultimate, but I have always had a computer at my side. I graduated from the University of Colorado with degrees in physics and saxophone performance (BA,BM). I continued my studies in physics at the University of Notre Dame (M.Sc.), and finished them at the University of Washington where I defended my search for the associated production of a vector and Higgs Boson, research conducted as a member of the D-Zero collaboration, as my doctoral thesis. Now I am a fellow at DESY working on the CMS experiment in Top quark studies and hardware upgrades. As always, my trusty computer aids me at every step, I am looking forward to learning more about it!
HAMMER Josef CERN, Geneva - Switzerland
Having studied computer science in Austria and Australia (with a focus on eCommerce and mobile and distributed systems), I obtained my MSc in 2007. Following nine months of traveling around the world, I zeroed in again on computer networks with a project on automatic Quality of Service management for home networks. In 2009 I joined CMS as a PhD student and created the SOAP-based Interconnection Test Framework for the distributed CMS L1-Trigger environment. Since 2012 I am working as a fellow for the PH-CMG-CO group, moving our monolithic JSP/Servlet-based system to RESTful web applications. I have been involved in the design of distributed systems for more than a decade, using a variety of technologies and tools ranging from C++ and CORBA over Zeroconf, Java EE and SOAP-based Web Services to REST and Python.
KUNZE Jonas Johannes Gutenberg Universität Mainz - Germany
I am a German physicist and PhD student working for the NA62 experiment at CERN and as Linux administrator at the University of Mainz, Germany. During my diploma thesis I’ve developed the PC-farm and implemented the associated software framework for the online trigger of NA62. Through this work and private projects I gained a lot of experience in network and parallel programming. Within my current work I’m analyzing the feasibility of using commodity processors for low-level triggers at high-energy physics experiments like NA62, LHCb and ATLAS.Privately I have developed the social network www.metalcon.de which has become the biggest German database concerning heavy metal music. As a part of the current redesign I’ve developed a fast autocompletion service which will run as a parallel web service distributed all over the world.
9
LÓPEZ GONZÁLEZ Juan CERN, Geneva - Switzerland
I work for GS-AIS-GDI in development and maintenance of the applications of the group (Phonebook, public outreach, e-groups, roles, foundation data...) I am also responsible of the maintenance and administration of the deployment/build/monitoring infrastructure in GDI section. Programming languages: C++, C#, Java, Groovy (Grails) and Lisp. Others: Maven, Bamboo, Nexus, Atlassian tools.
MESSI Francesco Rheinische Friedrich-Wilhelms-Universität, Bonn - Germany
I graduated in Physics working on the time-adjustment of the Front-End electronics for the muon chambers of the LHCb experiment in the electronics group of the university of Rome "La Sapienza". I worked one year in the didactic group of the university of Rome "Tor Vergata" developing didactical instrumentation for the high school. I started my PhD in 2009 at the "Rheinische Friedrich-Wilhelms-Universität" in Bonn on the Tagger Detector of the BGO-OD experiment at the ELSA accelerator. The main focus of my work is the development of the Front-End electronics (Amplifier, Dual Threshold Discriminator and Shaper). This includes the design of the PCB, the development of the firmware for the microcontroller and of the User Interface of the board, the commissioning of the boards and the analysis for the in-beam characterization of the prototype detector. I also contributed to the construction of the detector and to the implementation of the analysis tools. At present I am writing my thesis.
10
iCSC2014 Mentors The preparation of each iCSC lecture has been followed by one or two CSC lecturer(s) acting as mentor(s).
Mentor Lecture Lecturer
Francois Fluckiger LAN Programming – The basics
Building highly distributed systems within 5 minutes Jonas Kunze
Sebastian Lopienski Is your web API truly RESTful (and does it matter) Joseph Hammer
Sebastian Lopienski LAN Programming – The basics
Building highly distributed systems within 5 minutes Jonas Kunze
Giuseppe Lo Presti Is your web API truly RESTful (and does it matter) Joseph Hammer
Pere Mato Vila Read-Out Electronics: where data come from Francesco Messi
Lorenzo Moneta Introduction to machine learning and data mining
Self organizing maps – A visualization technique with data dimension reduction
Juan Lopez Gonzalez
Andrzej Nowak Introduction to machine learning and data mining
Self organizing maps – A visualization technique with data dimension reduction
Juan Lopez Gonzalez
Alberto Pace Read-Out Electronics: where data come from Francesco Messi
Ivica Puljak From Quark to Jet: A Beautiful Journey Tyler Mc Millan Dorland
Are Strandlie From Quark to Jet: A Beautiful Journey Tyler Mc Millan Dorland
11
12
Intr
oduc
tion
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
1iC
SC
2014
, 24
-25
Feb
ruar
y 20
14,
CE
RN
Wel
com
e …
iCS
C
Inve
rted
CE
RN
Sch
oo
l of
Co
mp
uti
ng
, 24-
25 F
ebru
ary
2014
2iC
SC
2014
, 24
-25
Feb
ruar
y 20
14,
CE
RN
iCS
C
"Whe
re s
tude
nts
turn
into
tea
cher
s"
Invo
lvin
g fo
rmer
CS
C p
artic
ipan
ts t
o de
liver
adv
ance
d ed
ucat
ion
3iC
SC
2014
, 24
-25
Feb
ruar
y 20
14,
CE
RN
The
CE
RN
Sch
ool o
f Com
putin
g
A
ims
at c
reat
ing
a c
om
mo
n c
ult
ure
in s
cien
tifi
c co
mp
uti
ng
am
on
g y
ou
ng
sci
enti
sts
and
en
gin
eers
invo
lved
in
par
ticl
e p
hys
ics
or
oth
er s
cien
ces,
as
a st
rate
gic
dir
ecti
on
to
fav
or
mo
bili
ty a
nd
to
fac
ilita
te t
he
dev
elo
pm
ent
of
larg
e co
mp
uti
ng
-o
rien
ted
tra
nsn
atio
nal
pro
ject
s.
ht
tp://
cern
.ch/
csc
P
arti
cip
ants
co
me
fro
m w
orl
dw
ide
lab
ora
tori
es a
nd
u
niv
ersi
ties
wit
h t
ypic
ally
of
15 t
o 3
0 d
iffe
ren
t n
atio
nal
itie
s (6
0 d
iffe
ren
t n
atio
nal
itie
s o
ver
the
pas
t 10
yea
rs).
http
://ce
rn.c
h/cs
c/al
umni
4iC
SC
2014
, 24
-25
Feb
ruar
y 20
14,
CE
RN
The
inve
rted
CS
C
A
t th
e en
d o
f ea
ch m
ain
sch
oo
l, w
e ca
ll st
ud
ents
pre
sen
t to
m
ake
pro
po
sals
. Wh
en w
e re
ceiv
e su
ffic
ien
t p
rop
osa
ls o
f ap
pro
pri
ate
qu
alit
y, w
e o
rgan
ize
an in
vert
ed s
cho
ol.
T
he
stu
den
ts c
om
bin
e th
eir
skill
s an
d e
lab
ora
te o
n C
SC
re
late
d s
ub
ject
s.
A
t re
gu
lar
CS
Cs,
th
e su
m o
f th
e kn
ow
led
ge
of
the
stu
den
ts
oft
en e
xcee
ds
the
on
e o
f le
ctu
rer
teac
hin
g, a
nd
th
at it
is
freq
uen
t t
o f
ind
in t
he
roo
m r
eal
exp
erts
on
par
ticu
lar
top
ics.
T
his
is t
he
idea
beh
ind
iC
SC
.
13
Intr
oduc
tion
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
5iC
SC
2014
, 24
-25
Feb
ruar
y 20
14,
CE
RN
Thi
s ye
ar p
rogr
amm
e
T
op
ics
LA
N P
rogr
amm
ing,
bui
ldin
g di
strib
uted
sys
tem
s
Web
AP
I
A jo
urne
y fr
om q
uark
to je
t
Rea
d-O
ut E
lect
roni
cs: w
here
dat
a co
me
from
M
achi
ne le
arni
ng a
nd d
ata
min
ing
S
pea
kers
fro
m C
SC
201
3, N
ico
sia
F
ranc
esco
, Jon
as, J
osef
, Jua
n Lo
pez,
and
Tyl
er
P
rin
ted
bo
okl
et a
vaila
ble
to
reg
iste
red
par
tici
pan
ts
6iC
SC
2014
, 24
-25
Feb
ruar
y 20
14,
CE
RN
Be
just
and
fear
not
Sh
akes
pea
re
14
LECTURE 1 LAN Programming – The basics
Monday 24 February
14:20-15:20
Description Distributed computing enables you to implement scalable systems with a higher total computing power while it also increases the availability. As the components of a distributed system have to communicate via networks, it is important to understand the underlying protocols to use them efficiently. In this lecture we will go into details about (10) Gigabit Ethernet as this is still the most common LAN technology. In this context the most important Internet Protocols will be discussed and compared. As nowadays networking performance is typically limited by the operating system, we will also discuss how Internet sockets are implemented in Linux and how alternatives based on Ethernet and other technologies look like.
Jonas Kunze
Johannes-Gutenberg
Universität Mainz - DE
Audience The attendees will understand the capabilities of the main Internet Protocols and how they perform on a Local Area Network. They will also get to know high performance network drivers significantly increasing throughput and latency. This lecture targets anyone who is interested in network programming and wants to understand how the underlying architectures work and which are the most common bottlenecks. Pre-requisite A basic programming knowledge is recommended to follow this lecture.
15
16
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
1iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Net
wor
k P
rogr
amm
ing
Lect
ure
1
LA
N P
rog
ram
min
g –
Th
e B
asic
s
Jon
as K
un
ze
Un
iver
sit
y o
f M
ain
z –
NA
62
Inve
rted
CE
RN
Sch
oo
l of
Co
mp
uti
ng
, 24-
25 F
ebru
ary
2014
LA
N P
rog
ram
min
g –
Th
e B
asic
s
2iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
R
ecap
of
the
TC
P/IP
mo
del
IS
O/O
SI a
nd T
CP
/IP
Use
r D
atag
ram
Pro
toco
l (U
DP
)
Tra
nsm
issi
on C
ontr
ol P
roto
col (
TC
P)
N
etw
ork
prog
ram
min
g w
ith B
SD
Soc
kets
C
ode
snip
pets
P
erfo
rman
ce
A
ltern
ativ
es to
BS
D S
ocke
ts
Net
wor
k P
roto
cols
in U
ser
Spa
ce
LA
N P
rog
ram
min
g –
Th
e B
asic
s
3iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
The
ISO
/OS
I ref
eren
ce m
odel
C
om
mu
nic
atio
ns
pro
toco
ls a
re d
ivid
ed i
nto
ind
epen
den
t la
yers
E
very
lay
er o
ffer
s a
serv
ice
to t
he
ove
rlyi
ng
lay
er
Brok
er 1
spea
ks ja
pane
se
Inte
rpre
ter
spea
ks ja
pane
se
and
engl
ish
Tech
nici
anre
cogn
izes
lett
ers
and
mor
ses
them
Brok
er 2
spea
ks fr
ench
Inte
rpre
ter
spea
ks fr
ench
an
d en
glis
h
Tech
nici
anre
ceiv
es le
tter
s an
dw
rites
sen
tenc
es
Idea
s ab
out b
itcoi
ns
Tran
slat
ed s
ente
nces
with
out k
now
ledg
e of
bitc
oins
Non
-inte
rpre
ted
char
acte
rsin
the
corr
ect o
rder
I S O
I S O
LA
N P
rog
ram
min
g –
Th
e B
asic
s
4iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Inte
rpla
y be
twee
n O
SI l
ayer
s
E
very
lay
er e
nca
psu
late
s th
e m
essa
ge
into
a P
roto
col D
ata
Un
it (
PD
U)
P
DU
s ty
pica
lly c
onsi
st o
f a H
eade
r an
d a
Dat
a se
ctio
n
C
om
mu
nic
atio
n p
artn
ers
exch
ang
e P
DU
s b
y u
sin
g t
he
nex
t lo
wer
laye
r
R
ecei
ver
un
pac
ks P
DU
s in
rev
erse
ord
er (
like
a st
ack)
n-PD
U
(n+
1)-P
DU
HH
Data
HDa
ta
HDa
ta
HDa
taLa
yer 3
Laye
r 2
Laye
r 1
17
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
5iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
The
TC
P/IP
mod
el
T
he
ISO
/OS
I mo
del
is ju
st a
th
eore
tica
l mo
del
wit
h a
lmo
st n
o
imp
lem
enta
tio
n
T
he
mo
st c
om
mo
n c
om
mu
nic
atio
ns
pro
toco
ls a
re p
art
of
the
Inte
rnet
Pro
toco
l Su
ite
(TC
P/IP
mo
del
)
Som
e IS
O/O
SI
laye
rs a
re m
erge
d
No
stric
t se
para
tion
betw
een
laye
rs
LA
N P
rog
ram
min
g –
Th
e B
asic
s
6iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Use
r D
atag
ram
Pro
toco
l (U
DP
)
U
DP
is c
onne
ctio
nles
s an
d un
relia
ble
like
IP
S
ourc
e-P
ort:
The
por
t of t
he p
roce
ss s
endi
ng th
e da
tagr
am
D
estin
atio
n-P
ort
: Th
e p
ort
nu
mb
er
the
da
tag
ram
sh
ou
ld b
e fo
rwa
rde
d
to
Le
ngth
: The
leng
th o
f th
e w
ho
le P
DU
in B
yte
s (
8 <
len
gth
< 6
55
35
)
C
heck
sum
: Cal
cula
ted
with
the
who
le P
DU
and
dat
a fr
om th
e IP
h
ea
de
r
LA
N P
rog
ram
min
g –
Th
e B
asic
s
7iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Tra
nsm
issi
on C
ontr
ol P
roto
col (
TC
P)
M
uch
mo
re p
ow
erfu
l an
d c
om
ple
x co
mm
un
icat
ion
ser
vice
th
an U
DP
Im
po
rtan
t ap
plic
atio
n la
yer
pro
toco
ls b
ased
on
TC
P
Wor
ld W
ide
Web
(H
TT
P)
E
mai
l (S
MT
P)
LA
N P
rog
ram
min
g –
Th
e B
asic
s
8iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Tra
nsm
issi
on C
ontr
ol P
roto
col (
TC
P)
T
CP
is r
elia
ble
:
Err
or-f
ree:
frag
men
ts a
re r
etra
nsm
itte
din
cas
e th
ey d
id n
ot
arriv
e at
the
dest
inat
ion
(tim
eout
)
pre
serv
ing
ord
erw
ithou
t d
up
licat
es
T
CP
is c
on
nec
tio
n o
rien
ted
C
onne
ctio
n es
tabl
ishm
ent n
eces
sary
bef
ore
data
can
be
sent
C
onne
ctio
n de
fined
by
IP a
nd p
ort
nu
mb
er (
like
UD
P)
of
sour
ce a
nd d
estin
atio
n
Con
nect
ions
are
alw
ays
po
int-
to-p
oin
tan
d fu
ll-du
plex
It
impl
emen
ts f
low
co
ntr
ol a
ndco
ng
esti
on
avo
idan
ce
D
ata
is tr
ansm
itted
as
an u
nst
ruct
ure
d b
yte
stre
am
18
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
9iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P d
ata
flow
A
sen
ds
fram
e w
ith
SY
N a
nd
ran
do
m
Seq
uen
ce n
um
ber
X
B
ack
no
wle
dg
es w
ith
AC
K=
X+
1 an
d
ran
do
m S
equ
ence
nu
mb
er Y
A
ack
no
wle
dg
es t
he
rece
pti
on
A
sen
ds
Z b
ytes
B
incr
ease
s th
e se
qu
ence
by
Z t
o
ackn
ow
led
ge
the
dat
a re
cep
tio
n
D
isco
nn
ecti
on
wo
rks
like
con
nec
tio
n
esta
blis
hm
ent
bu
t w
ith
FIN
inst
ead
of
SY
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
10iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Flo
w C
ontr
ol a
nd C
onge
stio
n A
void
ance
F
ram
es a
re o
nly
rar
ely
dro
pp
ed b
ecau
se o
f tr
ansm
issi
on
er
rors
(e.
g. b
it f
lip)
C
onne
ctio
ns a
re ty
pica
lly e
ither
wor
king
with
out
tran
smis
sion
er
rors
or
not a
t all
M
ain
reas
on fo
r dr
oppe
d fr
ames
are
ove
rload
s of
the
rece
iver
or
the
net
wo
rk
TC
P im
ple
men
ts t
wo
mec
han
ism
s to
avo
id o
verl
oad
ing
:
F
low
co
ntr
ol:
Avo
ids
over
load
ing
of th
e re
ceiv
er
C
on
ges
tio
n a
void
ance
: R
educ
es th
e se
ndin
g ra
te in
cas
e th
at
frag
men
ts a
re d
ropp
ed b
y th
e ne
twor
k
LA
N P
rog
ram
min
g –
Th
e B
asic
s
11iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P's
Flo
w C
ontr
ol:
Slid
ing
Win
dow
E
ach
no
de
ha
s a
re
ceiv
ing
an
d se
ndi
ng b
uff
er
In
eac
h se
gmen
t a n
ode
spec
ifies
how
man
y by
tes
it ca
n re
ceiv
e
Rec
eive
r w
indo
w s
ize:
Num
ber
of f
ree
byte
s in
the
rece
ivin
g bu
ffer
If
a n
ode
has
sent
as
man
y u
nac
kno
wle
dg
edby
tes
as th
e w
indo
w
size
is la
rge
it w
ill s
top
send
ing
and
wa
it fo
r th
e ne
xt a
ckno
wle
dgm
ent
W
ith
eac
h a
ckn
ow
led
gm
ent
the
win
do
w s
lides
to
th
e ri
gh
t
LA
N P
rog
ram
min
g –
Th
e B
asic
s
12iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P's
Con
gest
ion
Avo
idan
ce
C
on
ges
tio
n w
ind
ow
: Spe
cifie
s th
e m
axim
um n
umbe
r of
byt
es
that
may
be
sent
with
out
ackn
owle
dgm
ent
dep
end
ing
on
th
e n
etw
ork
cap
acit
y
M
ax b
ytes
th
at m
ay b
e se
nt
= m
in(s
lidin
g w
in, c
on
ges
tio
n w
in)
Th
e co
ng
esti
on
avo
idan
ce a
lgo
rith
m:
In
itial
ize
the
cong
estio
n w
indo
w t
o ty
pica
lly 2
x M
SS
(sl
ow s
tart
)
Sen
d un
til o
ne o
f the
two
win
dow
s ar
e fil
led
If
a se
gmen
t is
ackn
owle
dged
: In
crea
se th
e co
nges
tion
win
dow
D
oubl
ed u
ntil
thre
shol
d re
ache
d, t
hen
linea
rly
If
ackn
owle
dgm
ent
timed
out
(fr
ame
drop
ped
by n
etw
ork)
:
Set
thr
esho
ld t
o ha
lf th
e cu
rren
t co
nges
tion
win
dow
and
go
back
to s
low
st
art
19
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
13iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P's
Con
gest
ion
Avo
idan
ce
LA
N P
rog
ram
min
g –
Th
e B
asic
s
14iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Sen
ding
Buf
fer
W
hen
an
ap
plic
atio
n s
end
s d
ata
chu
nks
to
th
e T
CP
sta
ck t
wo
d
iffe
ren
t ap
pro
ach
es c
an b
e ap
plie
d:
1.Lo
w la
tenc
y
Dat
a ch
unks
sen
t di
rect
ly a
s th
ey a
re
D
isad
vant
age:
Man
y sm
all I
P p
acke
ts w
ill b
e tr
ansm
itted
(lo
w e
ffici
ency
)
2.H
igh
thro
ughp
ut
Buf
fer
data
and
sen
d la
rger
seg
men
ts
H
ighe
r la
tenc
y bu
t m
ore
effic
ient
LA
N P
rog
ram
min
g –
Th
e B
asic
s
15iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Nag
le’s
Alg
orith
m
A
n a
lgo
rith
m t
o r
each
th
e h
igh
th
rou
gh
pu
t ap
pro
ach
:
Sen
d fir
st c
hunk
of d
ata
arriv
ing
at th
e T
CP
sta
ck d
irect
ly
Fill
sen
ding
buf
fer
with
new
inco
min
g da
ta w
ithou
t se
ndin
g
If th
e bu
ffer
reac
hes
the
MS
S :
Sen
d a
new
fra
me
clea
ring
the
buffe
r
If al
l sen
t seg
men
ts a
re a
ckno
wle
dged
: S
end
a ne
w f
ram
e cl
earin
g th
e bu
ffer
N
agle
’s a
lgo
rith
m is
use
d in
alm
ost
all
TC
P im
ple
men
tati
on
s
Can
be
deac
tivat
ed to
red
uce
late
ncy
(e.g
. for
X11
app
licat
ions
)
LA
N P
rog
ram
min
g –
Th
e B
asic
s
16iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Sw
itch
off N
agle
'sA
lgor
ithm
T
his
is o
nly
rar
ely
nec
essa
ry!
W
ith
in y
ou
r p
rog
ram
:
S
yste
m w
ide:
intf
lag
= 0
;se
tsoc
kopt
(so
cket
,/*
soc
ket a
ffec
ted
*/IP
PR
OT
O_T
CP,
/* s
et o
ptio
n at
TC
P le
vel *
/T
CP_
NO
DE
LA
Y,
/*
nam
e of
opt
ion
*/(c
har
) &
flag
,/*
the
actu
al v
alue
*/
size
of(i
nt))
;
/* le
ngth
of
opti
on v
alue
*/
echo
1 >
/pro
c/sy
s/ne
t/ip
v4/t
cp_l
ow_l
aten
cy
20
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
17iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P v
sU
DP
T
CP
: A
lo
t o
f b
oo
kkee
pin
g a
nd
ad
dit
ion
al d
ata
tran
smis
sio
n
for
ackn
ow
led
gm
ents
U
DP
: Ju
st s
end
s th
e d
ata
as i
t is
Bu
t...
T
CP
: F
low
co
ntr
ol,
con
ges
tio
n a
void
ance
, N
agle
's a
lgo
rith
m
Typ
ical
ru
le o
f th
um
b:
T
CP
fo
r h
igh
th
rou
gh
pu
t, r
elia
bili
ty a
nd
/or
con
ges
tio
n
avo
idan
ce
U
DP
fo
r lo
w la
ten
cy a
nd
bro
adca
sts/
mu
ltic
asts
(n
ot
po
ssib
le
wit
h T
CP
)
LA
N P
rog
ram
min
g –
Th
e B
asic
s
18iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
A Q
uick
RT
T T
est
Thi
s te
st w
as
perf
orm
ed w
ith
hpcb
ench
:hp
cben
ch.s
ourc
efor
ge.n
et
405060708090100
110
120
130
140
0200
400
600
800
1000
1200
1400
1600
Latency[μs]
Chunk
size
[B]
UDPvs
TCProundtrip
times
UDP
TCP
TCPnodelay
LA
N P
rog
ram
min
g –
Th
e B
asic
s
19iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
R
ecap
of t
he T
CP
/IP m
odel
IS
O/O
SI a
nd T
CP
/IP
Use
r D
atag
ram
Pro
toco
l (U
DP
)
Tra
nsm
issi
on C
ontr
ol P
roto
col (
TC
P)
N
etw
ork
pro
gra
mm
ing
wit
h B
SD
So
cket
s
Cod
e sn
ippe
ts
Per
form
ance
In
terr
upt
Coa
lesc
ing
N
AP
I
A
ltern
ativ
es to
BS
D S
ocke
ts
Net
wor
k P
roto
cols
in U
ser
Spa
ce
LA
N P
rog
ram
min
g –
Th
e B
asic
s
20iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
BS
D S
ocke
ts
L
inu
x su
pp
ort
s T
CP
/IP a
s it
s n
ativ
e n
etw
ork
tra
nsp
ort
B
SD
So
cket
s is
a li
bra
ry w
ith
an
inte
rfac
e to
imp
lem
ent
net
wo
rk c
om
mu
nic
atio
ns
usi
ng
an
y T
CP
/IP la
yer
bel
ow
th
e ap
plic
atio
n la
yer
Im
po
rtan
t fu
nct
ion
s
sock
et()
ope
ns a
new
soc
ket
bi
nd()
ass
igns
soc
ket t
o an
add
ress
lis
ten(
) pr
epar
es s
ocke
t for
inco
min
g co
nnec
tions
ac
cept
() c
reat
es n
ew s
ocke
t for
inco
min
g co
nnec
tion
co
nnec
t()
conn
ects
to a
rem
ote
sock
et
send
() /
writ
e()
send
s da
ta
recv
() /
read
() r
ecei
ves
data
21
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
21iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P C
ode
Sni
ppet
Sim
ple
TC
P s
ocke
t acc
eptin
g co
nnec
tions
and
rec
eivi
ng d
ata:
Com
plet
e ex
ampl
es t
o be
fou
nd a
t: ht
tp://
gith
ub.c
om/J
onas
Kun
ze
sock
et =
sock
et(A
F_I
NE
T, S
OC
K_S
TR
EA
M, 0
);se
rv_a
ddr.s
in_f
amil
y=
AF
_IN
ET
;se
rv_a
ddr.s
in_p
ort=
hto
ns(8
080)
;se
rv_a
ddr.s
in_a
ddr.s
_add
r=
IN
AD
DR
_AN
Y;
bind
(soc
ket,
(str
ucts
ocka
ddr
*) &
serv
_add
r, si
zeof
(ser
v_ad
dr))
;lis
ten(
sock
et, 5
);co
nnec
tion
Soc
ket=
acc
ept(
sock
et, (
stru
ctso
ckad
dr*)
&cl
i_ad
dr, &
clil
en);
recv
(con
nect
ionS
ocke
t, bu
ffer
, siz
eof(
buff
er),
0);
LA
N P
rog
ram
min
g –
Th
e B
asic
s
22iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P v
s U
DP
: Thr
ough
put
Sin
gle
thre
aded
blo
ckin
g se
nder
and
rec
eive
r, re
liab
le n
etw
ork
●Sm
all f
ram
es in
duce
hig
hC
PU
load
→ p
acke
t los
s●T
CP
achi
eves
hig
her
thro
ughp
ut
UD
P re
ceiv
e ra
teU
DP
send
rat
eTC
P re
ceiv
e ra
te
100
1 k
10 k
chun
k si
ze [B
]
100
1 k
10 k
chun
k si
ze [B
]
CPU load [%]140 12
010
0 80 60 40 20 0
Datarate [Gbps]
12 10 8 6 4 2 0
UD
P CP
U lo
adU
DP
pack
et lo
ssTC
P CP
U lo
ad
LA
N P
rog
ram
min
g –
Th
e B
asic
s
23iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Dow
n to
the
Ker
nel
W
hen
dat
a ar
rive
s at
th
e N
IC:
D
ata
cop
ied
to k
erne
l spa
ce
(DM
A)
N
IC s
ends
inte
rru
pt
K
erne
l co
pie
sda
ta t
o th
e co
rres
pond
ing
user
spa
ce b
uffe
r (s
ocke
t)
Ker
nel i
nfor
ms
user
spa
ce
appl
icat
ion
LA
N P
rog
ram
min
g –
Th
e B
asic
s
24iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Inte
rrup
t C
oale
scin
g
T
ech
niq
ue
to r
edu
ce in
terr
up
t lo
ad
In
terr
up
ts a
re h
eld
bac
k u
nti
l...
…
a c
erta
in n
umbe
r of
fram
es h
ave
been
rec
eive
d...
…
or
a tim
er ti
mes
out
N
ow
th
e ke
rnel
can
pro
cess
sev
eral
fra
mes
at
on
ce
Hig
her
effic
ienc
y w
ith ju
st li
ttle
incr
ease
of l
aten
cy
# pr
int c
urre
nt s
etti
ngs
etht
ool-
c et
h0
# ch
ange
set
ting
set
htoo
l-C
eth
0 rx
-use
cs0
# 0
is a
dapt
ive
mod
e fo
r m
any
driv
ers
etht
ool-
C e
th0
rx-f
ram
es 1
2
22
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
25iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Inte
rrup
t C
oale
scin
g
S
mal
l va
lues
ove
rlo
ad t
he
CP
U →
Pac
ket
loss
H
igh
val
ues
lea
d t
o b
uff
er o
verf
low
→ P
acke
t lo
ss
Fir
st b
in s
how
sad
apti
ve m
ode
LA
N P
rog
ram
min
g –
Th
e B
asic
s
26iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
NA
PI
A
n a
lter
nat
ive
to in
terr
up
ts is
po
llin
g:
K
erne
l per
iodi
cally
che
cks
for
new
dat
a in
the
NIC
buf
fer
H
igh
polli
ng f
requ
enci
es in
duce
hig
h m
emor
y lo
ads
Lo
w p
ollin
g fr
eque
ncie
s le
ad t
o hi
gh la
tenc
ies
and
pack
et lo
ss
N
AP
I: L
inu
x u
ses
bo
th
Inte
rrup
ts p
er d
efau
lt
Pol
ling
in c
ase
of h
igh
data
rat
es in
com
ing
The
ker
nel s
till n
eeds
to c
opy
inco
min
g da
ta!
LA
N P
rog
ram
min
g –
Th
e B
asic
s
27iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
R
ecap
of t
he T
CP
/IP m
odel
IS
O/O
SI a
nd T
CP
/IP
Use
r D
atag
ram
Pro
toco
l (U
DP
)
Tra
nsm
issi
on C
ontr
ol P
roto
col (
TC
P)
N
etw
ork
prog
ram
min
g w
ith B
SD
Soc
kets
C
ode
snip
pets
P
erfo
rman
ce
A
lter
nat
ives
to
BS
D S
ock
ets
N
etw
ork
Pro
toco
ls in
Use
r S
pace
E
xam
ple:
pf_
ring
DN
A
R
elia
bilit
y on
top
of
UD
P?
R
elia
bilit
y w
ithou
t ac
know
ledg
men
t
LA
N P
rog
ram
min
g –
Th
e B
asic
s
28iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Net
wor
k P
roto
cols
in U
ser
Spa
ce
F
ollo
win
g a
pp
roac
h c
an b
e im
ple
men
ted
in
th
e u
ser
spac
e to
avo
id d
ou
ble
co
pie
s
NIC
cop
ies
inco
min
g da
ta to
a u
ser
spac
e bu
ffer
(DM
A)
T
he u
ser
spac
e ap
plic
atio
n po
lls th
e bu
ffer
T
he u
ser
spac
e ap
plic
atio
n m
ay e
nabl
e in
terr
upts
for
low
dat
a ra
tes
T
he k
erne
l is
only
use
d fo
r th
e in
itial
izat
ion
0%
CP
U u
sed
fo
r ac
cess
ing
th
e d
ata
23
Net
wo
rk P
rog
ram
min
g
Lec
ture
1iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
LA
N P
rog
ram
min
g –
Th
e B
asic
s
29iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Exa
mpl
e: p
f_rin
gD
NA
P
rop
riet
ary
use
r sp
ace
dri
ver
by
nto
p
D
oes
no
t im
ple
men
t an
y p
roto
col
Y
ou n
eed
to im
plem
ent
them
: ET
H, I
P, U
DP
, TC
P, A
RP
, IG
MP
...
C
om
pat
ible
wit
h a
ll 1
Gb
Ean
d 1
0 G
bE
NIC
s ru
nn
ing
on
PC
I-E
F
ull
line
rate
(1-
10 G
bE
) w
ith
an
y fr
ame
size
R
ou
nd
tri
p t
ime
bel
ow
5 µ
s
H
ard
war
e fi
lter
ing
(o
nly
In
tel a
nd
Sili
com
NIC
s)
Ver
y ef
ficie
nt In
trus
ion
prev
entio
n sy
stem
s po
ssib
le (
Sno
rt)
O
ther
use
rsp
ace
dri
vers
: N
etm
ap, I
nte
l DP
DK
, Op
enO
nlo
ad
LA
N P
rog
ram
min
g –
Th
e B
asic
s
30iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Rel
iabi
lity
on t
op o
f U
DP
?
A
t C
ER
N e
xper
imen
ts m
ost
dat
a se
nd
ers
are
FP
GA
s
Ver
y fa
st in
par
alle
l job
s
Typ
ical
ly f
ully
load
ed b
y al
gorit
hms
S
omet
imes
the
re's
no
spac
e le
ft fo
r a
fully
impl
emen
ted
TC
P/IP
sta
ck
I'v
e se
en m
any
gro
up
s im
ple
men
tin
g r
elia
ble
pro
toco
ls o
n t
op
o
f IP
In m
ost c
ases
the
resu
lt w
as T
CP
with
out
flow
and
con
gest
ion
cont
rol
B
ein
g c
om
pat
ible
wit
h T
CP
/UD
P r
elie
ves
the
soft
war
e d
evel
op
ers
Y
ou d
on't
need
to im
plem
ent
the
prot
ocol
on
the
rece
iver
sid
e
Inst
ead
you
can
use
stan
dard
libr
arie
s
LA
N P
rog
ram
min
g –
Th
e B
asic
s
31iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Rel
iabi
lity
with
out
ackn
owle
dgm
ent
S
om
etim
es it
's n
ot
even
po
ssib
le t
o s
tore
dat
a u
nti
l th
e ac
kno
wle
dg
men
t is
rec
eive
d
You
sho
uld
use
pure
UD
P in
this
cas
e
A
s so
on
as
dat
agra
ms
are
sen
t o
ut
you
hav
e to
tru
st t
he
net
wo
rk
Mak
e su
re th
at y
ou d
on't
over
load
sw
itche
s/ro
uter
s/re
ceiv
er
node
s
Che
ck e
very
nod
e w
heth
er f
ram
es a
re d
ropp
ed
Switc
h/R
oute
r:sh
ow in
terf
aces
...
Lin
ux:
cat /
proc
/net
/udp
LA
N P
rog
ram
min
g –
Th
e B
asic
s
32iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Sum
mar
y
T
CP
is m
ore
th
an ju
st r
elia
ble
It
impl
emen
ts a
max
imum
effi
cien
t dat
a tr
ansm
issi
on
B
SD
so
cket
s p
rovi
de
a n
ice
AP
I fo
r si
mp
le n
etw
ork
p
rog
ram
min
g
For
mor
e co
mpl
ex a
rchi
tect
ures
net
wor
king
libr
arie
s ar
e re
com
men
ded
L
inu
x' n
etw
ork
so
cket
s ar
e n
ot
as e
ffic
ien
t as
th
ey c
ou
ld b
e
Hig
h pe
rfor
man
ce n
etw
ork
driv
ers
prov
ide
effic
ient
alte
rnat
ives
to
BS
D s
ocke
ts b
ut th
ey g
ener
ate
addi
tiona
l wor
k fo
r th
e de
velo
per
team
24
LECTURE 2 Is your web API truly RESTful (and does it matter)
Monday 24 February
15:50-16:50
Description After a brief introduction to the history of web services, this lecture will cover the basics of REST (Representational State Transfer) and provide you with an understanding of essential terms and constraints. We will have a look at the API design process, and think about what you should consider when designing a scalable web service. Furthermore, we will explore how striving for a resource-oriented client architecture helps to reap the benefits of REST without sacrificing user experience.
Josef Hammer
CERN
Audience This lecture targets everyone (mainly computer scientists) involved in web development, in particular those responsible for the API design of the back-end. After this lecture, the attendees are expected to have a comprehensive overview of the REST principles and its benefits for modern, scalable web development. Furthermore, the participants will be aware of potential pitfalls and ways to address them.
Pre-requisite No special prior knowledge is required to follow this lecture. However, a basic understanding of HTTP and web development will prove beneficial.
LECTURE 3 Building highly distributed systems within 5 minutes
Monday 24 February
16:50-17:50
Description Highly distributed systems are typically very complex. Traditionally, it took a long time to design the dataflow and it may have taken even more time to implement the necessary communication interfaces. But using modern libraries to outsource the communication significantly reduces development and implementation time. During this lecture several communication patterns will be discussed and a selection of libraries for different application areas will be introduced: Boost.Asio is a C++ library that provides you a consistent way to develop asynchronous communication and therefore makes it easy to develop a highly parallel program. ØMQ is a library for many different programming languages. It provides the distribution of messages with several patterns and therefore clearly facilitates the development of distributed systems. Apache Thrift is a framework enabling Remote Procedure Calls (RPCs) between many different languages. It generates source code for the server and client based on a given interface description file.
Jonas Kunze
Johannes-Gutenberg
Universität Mainz - DE
Audience This lecture is addressed to physicists and engineers that need to implement distributed systems. During this lecture the attendees will get an overview of different modern approaches to implement distributed systems.
Pre-requisite A basic programming knowledge is recommended to follow this lecture. Additionally a basic understanding of the main Internet Protocols is advisable. This knowledge can be obtained during the first lecture.
39
40
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
1iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Net
wor
k P
rogr
amm
ing
Lect
ure
2
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s
Wit
hin
5 M
inu
tes
Jon
as K
un
ze
Un
iver
sit
y o
f M
ain
z –
NA
62
Inve
rted
CE
RN
Sch
oo
l of
Co
mp
uti
ng
, 24-
25 F
ebru
ary
2014
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
2iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
M
oti
vati
on
B
oo
st.A
sio
M
essa
ge
Pas
sin
g
Ø
MQ
A
pac
he
Th
rift
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
3iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
TC
P in
C c
ode
T
he
BS
D s
ock
et A
PI i
s m
inim
alis
tic
N
o in
trin
sic
mul
tithr
eadi
ng s
uppo
rt
Han
dlin
g m
ultip
le c
onne
ctio
ns t
ypic
ally
via
fork
()
N
o da
ta m
anag
emen
t (m
essa
ging
)
Con
figur
atio
n a
bit
awkw
ard
T
her
e is
no
exc
epti
on
han
dlin
g o
r O
OP
in C
T
her
e is
no
C+
+ s
ock
et A
PI i
n t
he
std
libra
ry
std:
:soc
ket w
ill n
ever
com
e
#inc
lude
<st
dio.
h>#i
nclu
de <
netin
et/in
.h>
#inc
lude
<sy
s/so
cket
.h>
#inc
lude
<sy
s/ty
pes.
h>
intm
ain(
int
argc
, cha
r *a
rgv[
] )
{in
tsoc
kfd,
new
sock
fd, p
ortn
o, c
lilen
;ch
ar b
uffe
r[25
6];
stru
ctso
ckad
dr_i
nse
rv_a
ddr,
cli_
addr
;in
tn;
sock
fd=
soc
ket(
AF_
INE
T,
SOC
K_S
TR
EA
M, 0
);
bzer
o((c
har
*) &
serv
_add
r, s
izeo
f(se
rv_a
ddr)
);po
rtno
= 1
324;
serv
_add
r.si
n_fa
mily
= A
F_IN
ET
;se
rv_a
ddr.
sin_
addr
.s_a
ddr
= I
NA
DD
R_A
NY
;se
rv_a
ddr.
sin_
port
= h
tons
(por
tno)
;bi
nd(s
ockf
d, (
stru
ctso
ckad
dr*)
&se
rv_a
ddr,
size
of(s
erv_
addr
));
liste
n(so
ckfd
,5);
clile
n=
siz
eof(
cli_
addr
);w
hile
(1)
{ne
wso
ckfd
= a
ccep
t(so
ckfd
,(s
truc
tso
ckad
dr*)
&cl
i_ad
dr, &
clile
n);
pid
= f
ork(
);if
(pi
d=
= 0
) {
clos
e(so
ckfd
);do
som
ethi
ng(n
ewso
ckfd
);ex
it(0)
;}
else
{cl
ose(
new
sock
fd);
}}
}
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
4iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
M
otiv
atio
n
B
oo
st.A
sio
A
sync
hron
ous
oper
atio
ns
Con
curr
ency
with
out
thre
ads
M
ultit
hrea
ding
M
essa
ge P
assi
ng
Ø
MQ
A
pach
e T
hrift
41
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
5iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Boo
st.A
sio
B
oost
.Asi
ois
a C
++
libr
ary
for
low
-leve
l I/O
pro
gram
min
g w
ith a
co
nsis
tent
asy
nch
ron
ou
sm
odel
incl
udin
g a
BS
D s
ocke
tint
erfa
ce
BS
D S
ock
et A
PI
(Lin
ux)
Eq
uiv
alen
tsin
Bo
ost
.Asi
o
sock
et d
escr
ipto
r–
int
For
TC
P:
ip::
tcp
::so
cke
tF
orU
DP
: ip
::u
dp
::so
cke
t
sock
addr
_in,
soc
kadd
r_in
6F
orT
CP
: ip
::tc
p::e
ndpo
int
For
UD
P:
ip::u
dp::e
ndpo
int
acce
pt()
For
TC
P:
ip::
tcp:
:acc
epto
r::a
ccep
t()
bind
()F
or T
CP
: ip
::tc
p::s
ocke
t::b
ind(
)F
or U
DP
: ip
::ud
p::s
ocke
t::b
ind(
)
...
...
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
6iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Boo
st.A
sio
B
oo
st.A
sio
use
s an
ob
ject
as
an in
terf
ace
to t
he
op
erat
ing
sy
stem
: io
_ser
vice
T
he
io_s
ervi
ceo
bje
ct is
pas
sed
to
I/O
ob
ject
s lik
e tc
p::
sock
et
T
he
I/O o
bje
cts
will
fo
rwar
d r
equ
ests
to
th
e io
_ser
vice
ob
ject
io
_ser
vice
runs
the
requ
ired
sysc
alls
boos
t::a
sio:
:io_
serv
ice
io_s
ervi
ce;
boos
t::a
sio:
:ip:
:tcp
::so
cket
sock
et(io
_ser
vice
);bo
ost:
:asi
o::i
p::t
cp::
reso
lver
res
olve
r(io
_ser
vice
); //
get
host
byna
me
wra
pper
sock
et.c
onne
ct(*
reso
lver
.res
olve
({ho
stna
me,
por
tNum
}));
sock
et.se
nd(b
oost
::as
io::
buff
er("
mes
sage
"));
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
7iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Boo
st.A
sio:
Asy
nchr
onou
s op
erat
ions
I/O
ob
ject
s im
ple
men
t n
on
-blo
ckin
g/a
syn
chro
no
us
op
erat
ion
s
E.g
. boo
st::a
sio:
:ip::t
cp::s
ocke
t::as
ync_
conn
ect
C
om
ple
tio
n h
and
ler
fun
ctio
n p
asse
d t
o a
syn
c_ f
un
ctio
ns
io
_ser
vice
.ru
n()
cal
ls t
he
com
ple
tio
n h
and
ler
as s
oo
n a
s re
sult
s o
f as
ync_
fu
nct
ion
s ar
e av
aila
ble
Your
Pro
gram
Com
plet
ion
Hand
ler
io_s
ervi
ce
Oper
atin
g Sy
stem
Exec
utes
sysc
alls
Read
s res
ults
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
8iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Boo
st.A
sio:
Asy
nchr
onou
s op
erat
ions
S
imp
le T
CP
co
nn
ecti
on
exa
mp
le:
E
ven
sim
ple
r w
ith
C+
+11
usi
ng
a la
mb
da
fun
ctio
n:
void
MyC
lass
::han
dle_
conn
ect(
cons
tboo
st::
syst
em::
erro
r_co
de&
error
) {
if (
!err
or)
{do
Som
ethi
ng()
; }} ... so
cket
.asy
nc_c
onne
ct(s
ocke
t, *r
esol
ver.r
esol
ve({
host
nam
e, p
ortN
um})
,bo
ost:
:bin
d(&
MyC
lass
::han
dle_
conn
ect,
this
, boo
st::
asio
::pl
aceh
olde
rs::
erro
r));
wor
kWhi
leC
onne
ctin
g();
io_s
ervi
ce.r
un()
; // R
uns h
andl
e_co
nnec
tas s
oon
as th
e co
nnec
tion
is e
stab
lishe
d
sock
et.a
sync
_con
nect
(*re
solv
er.r
esol
ve({
host
nam
e, p
ortN
um})
,[t
his]
(boo
st::
syst
em::
erro
r_co
deer
ror,
tcp:
:res
olve
r::i
tera
tor)
{if
(!e
rror
) {
doS
omet
hing
(); }
});
wor
kWhi
leC
onne
ctin
g();
io_s
ervi
ce.r
un()
;
42
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
9iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Con
curr
ency
with
out
thre
ads
Han
dlin
g m
ultip
le T
CP
con
nect
ions
O
ne
io_s
ervi
ceca
n h
and
le s
ever
al I
/O o
bje
cts
and
asy
nc_
o
per
atio
ns
io
_ser
vice
::ru
n()
will
blo
ck u
nti
l all
req
ues
ts h
ave
bee
n
han
dle
d
sock
1.as
ync_
read
_som
e(re
adB
uffe
r, []
(boo
st::
syst
em::
erro
r_co
deer
ror,
std:
:siz
e_t)
{if
(!e
rror
) {s
td::
cout
<<
"S
ocke
t 1 r
ecei
ved
som
ethi
ng"
<<
std
::en
dl;}
});
sock
2.as
ync_
read
_som
e(re
adB
uffe
r, []
(boo
st::
syst
em::
erro
r_co
deer
ror,
std:
:siz
e_t)
{if
(!e
rror
) {s
td::
cout
<<
"S
ocke
t 2 r
ecei
ved
som
ethi
ng"
<<
std
::en
dl;}
});
io_s
ervi
ce.r
un()
;co
ut<
< “
Bot
h so
cket
s re
ceiv
ed s
omet
hing
” <
< e
ndl;
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
10iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Mul
tithr
eadi
ng
io
_ser
vice
::ru
n()
can
be
calle
d b
y m
ult
iple
th
read
s si
mu
ltan
eou
sly
as
ync_
op
erat
ion
s w
ill b
e d
istr
ibu
ted
am
on
g t
hes
e th
read
s
A
co
mm
on
ap
pro
ach
is t
o la
un
ch a
th
read
po
ol r
un
nin
g t
he
wh
ole
life
tim
e
N th
read
s sp
awne
d at
the
begi
nnin
g ha
ndlin
g al
l asy
nc_
ope
ratio
ns
Rec
ursi
ve c
alls
of a
syn
c_ o
pera
tions
(io
_ser
vice
::run
() n
ever
ret
urns
)
hand
le_r
ead
do s
omet
hing
hand
le_w
rite
calls
asyn
c_w
rite
asyn
c_re
ad
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
11iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Ser
ver
with
a T
hrea
d P
ool
boos
t::a
sio:
:io_
serv
ice
io_s
ervi
ce;
Ech
oSer
ver
s(io
_ser
vice
, 123
4); /
/ cal
ls so
cket
.asy
nc_r
ead.
..
std:
:vec
tor<
std:
:thr
ead>
thre
adP
ool;
for
(std
::si
ze_t
i= 0
; i<
std
::th
read
::ha
rdw
are_
conc
urre
ncy(
); +
+i)
{th
read
Poo
l.pus
h_ba
ck(
std:
:thr
ead(
[&](
) {
io_s
ervi
ce.r
un()
;})
);} fo
r(au
to&
thre
ad :
thre
adP
ool)
{th
read
.join
();
}
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
12iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
M
otiv
atio
n
B
oost
.Asi
o
M
essa
ge
Pas
sin
g
Ø
MQ
A
pach
e T
hrift
43
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
13iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Mes
sage
pas
sing
via
TC
P
T
CP
on
ly o
ffer
s a
con
tin
uo
us
dat
a st
ream
A
lthou
gh d
ata
is ty
pica
lly s
ent t
o so
cket
s in
chu
nks,
the
rece
iver
m
ay s
ee d
iffer
ent c
hunk
s (s
calin
g w
indo
w)
T
he a
pplic
atio
n la
yer
prog
ram
has
to s
plit
the
stre
am in
to
mes
sage
s
T
her
e ar
e th
ree
po
ssib
le a
pp
roac
hes
to
ind
icat
e m
essa
ges
in
the
stre
am:
P
roto
col d
efin
es th
e m
essa
ge le
ngth
im
plic
itly
T
he m
essa
ge le
ngth
is e
xplic
itly
spec
ified
in a
mes
sage
he
ader
L
ine-
Bas
ed: M
essa
ges
in th
e st
ream
are
sep
arat
ed b
y de
limite
rs
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
14iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Mes
sage
pas
sing
via
TC
P
L
ine-
Bas
ed a
pp
roac
h e
asily
im
ple
men
ted
wit
h B
oo
st.A
sio
:
O
ther
ap
pro
ach
es a
re m
uch
mo
re e
ffic
ien
t
But
als
o ha
rd w
ork
to im
plem
ent
Ø
MQ
imp
lem
ents
fas
t m
essa
ge
pas
sin
g u
sin
g t
he
exp
licit
fo
rmatbo
ost:
:asi
o::r
ead_
until
(soc
ket,
msg
Buf
fer,
"\r\
n");
No
need
to r
einv
ent t
he w
heel
!
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
15iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
M
otiv
atio
n
B
oost
.Asi
o
M
essa
ge P
assi
ng
Ø
MQ
M
essa
ging
Pat
tern
s
Bro
ker
M
ultit
hrea
ding
A
pach
e T
hrift
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
16iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q
W
hat
ØM
Q s
ays
abo
ut
thei
r so
cket
lib
rary
(h
ttp
://z
gu
ide.
zero
mq
.org
/pag
e:al
l):
We
took
a n
orm
al T
CP
soc
ket,
inje
cted
it w
ith a
mix
of r
adio
activ
e is
otop
es s
tole
n fr
om a
sec
ret S
ovie
t ato
mic
res
earc
h pr
ojec
t, bo
mba
rded
it w
ith 1
950-
era
cosm
ic r
ays
(...)
It's
soc
kets
on
ster
oids
.TC
P So
cket
ØMQ
sock
etZA
P!PO
W!!
Span
dex
Cosm
ic ra
ysIll
egal
radi
oiso
tope
s fro
mse
cret
Sov
iet a
tom
ic c
ity
BOOM
!
44
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
17iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q
Ø
MQ
off
ers
a u
nif
orm
AP
I (Ø
MQ
so
cket
s) t
o t
ran
spo
rt
mes
sag
es o
ver
dif
fere
nt
chan
nel
s:
TC
P, m
ultic
ast,
IPC
(pr
oces
s to
pro
cess
), in
proc
(thr
ead
to
thre
ad)
C
ross
Pla
tfo
rm (
Lin
ux,
Win
do
ws,
Mac
, etc
...)
Im
ple
men
tati
on
s in
man
y(!!
!) d
iffe
ren
t la
ng
uag
es:
C
/C+
+, J
ava,
Pyt
hon,
Rub
y, P
HP
, Per
l,N
ode.
js,
C#,
Clo
jure
, C
L, D
elph
i, E
rlang
, F
#, F
elix
, G
o, H
aske
ll, H
axe,
Lu
a, O
bjec
tive-
C,
Q,
Rac
ket,
Sca
la..
.
O
pen
So
urc
e
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
18iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Mes
sagi
ng P
atte
rns
Ø
MQ
so
cket
s ex
pre
ss s
ever
al m
essa
gin
g p
atte
rns
R
EQ
and
RE
P
PU
Ban
d S
UB
P
US
H a
ndP
UL
L
RE
Q a
nd R
OU
TE
R
DE
ALE
R a
nd R
EP
D
EA
LER
and
RO
UT
ER
D
EA
LER
and
DE
ALE
R
RO
UT
ER
and
RO
UT
ER
P
AIR
and
PA
IR
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
19iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
RE
Q-R
EP
C
lien
ts “
con
nec
t” t
o a
ser
vice
an
d s
end
a R
EQ
ues
tm
essa
ge
T
he
serv
er R
EP
lies
to e
ach
req
ues
t w
ith
a s
ing
le m
essa
ge
S
end
ing
is d
on
e as
ynch
ron
ou
sly
in t
he
bac
kgro
un
d
Use
r w
rites
sim
ple
non-
bloc
king
cod
e
If th
e re
mot
e en
dpoi
nt is
dow
n th
e m
essa
ge w
ill b
e se
nt la
ter
T
his
rep
rese
nts
a r
emo
te p
roce
du
re c
all p
atte
rn
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
20iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Sim
ple
RE
Q C
lient
intm
ain(
) {
zmq:
:con
text
_tco
ntex
t(1)
;zm
q::s
ocke
t_ts
ocke
t(co
ntex
t, ZM
Q_R
EQ
);so
cket
.con
nect
("tc
p://R
EPS
erve
rHos
tNam
e:55
55")
;
zmq:
:mes
sage
_tre
ques
t(6)
;m
emcp
y((v
oid
*) r
eque
st.d
ata(
), "
Hel
lo",
5);
sock
et.se
nd(r
eque
st);
zmq:
:mes
sage
_tre
ply;
sock
et.r
ecv(
&re
ply)
;re
turn
0;
}
45
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
21iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Sim
ple
RE
P S
erve
r
intm
ain(
) {
zmq:
:con
text
_tco
ntex
t(1)
; // S
imila
r to
io_s
ervi
cezm
q::s
ocke
t_ts
ocke
t(co
ntex
t, ZM
Q_R
EP)
;so
cket
.bin
d("t
cp://
*:55
55")
;
whi
le (
true
) {
zmq:
:mes
sage
_tre
ques
t;so
cket
.rec
v(&
requ
est)
;
zmq:
:mes
sage
_tre
ply(
5);
mem
cpy(
(voi
d *)
rep
ly.d
ata(
), "
Wor
ld",
5);
sock
et.se
nd(r
eply
);} re
turn
0;
}
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
22iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
RE
Q-R
EP
Not
es
T
he
RE
Q-R
EP
so
cket
pai
r is
in lo
ckst
ep
Ser
ver
and
clie
nt h
ave
to c
all s
end
and
recv
alte
rnat
ely
S
erve
r au
tom
atic
ally
sen
ds to
the
node
it g
ot th
e la
st m
essa
ge
(rec
v) fr
om
All
the
conn
ectio
n ha
ndlin
g is
don
e by
ØM
Q
T
he
con
nec
tio
n c
an b
e es
tab
lish
ed f
rom
bo
th s
ides
(tr
ue
for
all p
atte
rns)
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
23iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
PU
B-S
UB
S
erve
r P
UB
lish
esd
ata
to a
ll co
nn
ecte
d c
lien
ts
C
lien
ts S
UB
scri
be
to t
he
dat
a b
y co
nn
ecti
ng
to
th
e se
rver
S
ub
scri
pti
on
to
mes
sag
es b
y d
ata
pre
fix
(filt
er)
If
no
clie
nt
is c
on
nec
ted
th
e d
ata
will
be
lost
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
24iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Pip
elin
e
V
enti
lato
r: P
rod
uce
s ta
sk t
hat
can
be
pro
cess
ed in
par
alle
l
T
hes
e ta
sks
are
then
PU
SH
edev
enly
to
th
e co
nn
ecte
d
Wo
rker
s
A
fter
pro
cess
ing
th
e ta
sks
the
Wo
rker
s p
ush
th
e re
sult
s to
a
Sin
k
B
asic
load
bal
anci
ng
46
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
25iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Pip
elin
e W
orke
r
intm
ain(
) {
zmq:
:con
text
_tco
ntex
t(1)
;zm
q::s
ocke
t_tv
enti
lato
rSoc
ket(
cont
ext,
ZMQ
_PU
LL
);ve
ntil
ator
Soc
ket.c
onne
ct("
tcp:
//ve
ntil
ator
:555
7");
zmq:
:soc
ket_
tsin
kSoc
ket(
cont
ext,
ZMQ
_PU
SH);
sink
Soc
ket.c
onne
ct("
tcp:
//si
nk:5
558"
);
whi
le (
1) {
zmq:
:mes
sage
_tta
sk;
vent
ilato
rSoc
ket.r
ecv(
&ta
sk);
// P
UL
Lzm
q::m
essa
ge_t
resu
lt =
doS
omeW
ork(
task
);si
nkSo
cket
.send
(res
ult)
; // P
US
H}
}
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
26iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
N-t
o-M
com
mun
icat
ion
S
o f
ar w
e h
ad N
wo
rker
s p
ulli
ng
fro
m o
ne
ven
tila
tor
It
is p
oss
ible
to
co
nn
ect
on
e Ø
MQ
so
cket
to
sev
eral
en
dp
oin
ts
T
he
mes
sag
es w
ill b
e sc
hed
ule
d f
airl
y fr
om
all
ven
tila
tors
vent
ilat
orS
ocke
t.con
nect
("tc
p://
vent
ilat
or1:
5557
");
vent
ilat
orS
ocke
t.con
nect
("tc
p://
vent
ilat
or2:
5557
");
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
27iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Bro
ker
W
ith
th
e la
st d
esig
n t
he
wo
rker
s n
eed
to
kn
ow
all
ven
tila
tors
(h
ost
nam
es)
If
a n
ew v
enti
lato
r is
ad
ded
all
the
wo
rker
s h
ave
to c
on
nec
t (e
vtl.
Res
tart
)
O
ne
easy
des
ign
to
fix
th
is:
Ad
d a
cen
tral
bro
ker
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
28iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Bro
ker
T
his
is e
asily
im
ple
men
ted
wit
h a
zm
q_p
roxy
forw
ard
ing
m
essa
ges
:
zmq:
:con
text
_tco
ntex
t(1)
;
// S
ocke
t fac
ing
vent
ilat
ors
zmq:
:soc
ket_
tfro
nten
d(co
ntex
t, ZM
Q_P
UL
L);
fron
tend
.bin
d("t
cp:/
/*:5
556"
);
// S
ocke
t fac
ing
wor
kers
zmq:
:soc
ket_
tbac
kend
(con
text
, ZM
Q_P
USH
);ba
cken
d.bi
nd("
tcp:
//*:
5557
");
// Pa
ss m
essa
ges f
rom
ven
tilat
ors t
o w
orke
rszm
q_pr
oxy(
fron
tend
, bac
kend
, NU
LL
);
47
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
29iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Bro
ker
N
ow
yo
u o
nly
hav
e to
ch
ang
e o
ne
line
in t
he
ven
tila
tor:
A
nd
co
nn
ect
the
wo
rker
to
th
e b
roke
r in
stea
d o
f th
e ve
nti
lato
rs
A
nd
ag
ain
yo
u c
an s
tart
ven
tila
tors
, wo
rker
s, b
roke
r an
d s
ink
in w
hat
ever
ord
er y
ou
lik
e:
Mes
sag
es a
re q
ueu
ed a
s cl
ose
to
th
e re
ceiv
er a
s p
oss
ible
vent
ilat
or.c
onne
ct("
tcp:
//ve
ntil
ator
1:55
57")
;ve
ntil
ator
.con
nect
("tc
p://
vent
ilat
or2:
5557
"); …
Tur
ns to
:ve
ntil
ator
.con
nect
("tc
p://
brok
er:5
557"
);
sock
et.b
ind(
"tcp
://*
:555
9");
→ s
ocke
t.con
nect
("tc
p://
brok
er:5
559"
);
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
30iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
IPC
S
o f
ar w
e u
sed:
sock
et.b
ind
("tc
p:/
/*:5
555"
);
T
o r
un
th
e sa
me
pro
gra
ms
loca
lly o
ne
sho
uld
use
:
sock
et.b
ind(
"ip
c:///
tmp/
hello
Wor
ld")
; //
For
pro
cess
es
sock
et.b
ind(
"in
pro
c:///
hello
Wor
ld")
; //
For
thre
ads
S
tart
dev
elo
pin
g y
ou
r so
ftw
are
wit
h m
any
mo
du
les
com
mu
nic
atin
g w
ith
IPC
T
hen
ou
tso
urc
e h
eavy
lo
aded
ser
vice
s to
ext
ern
al b
oxe
s ju
st
by
chan
gin
g
in
pro
c/ip
c://.
.. →
tcp
://..
.
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
31iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Mul
tithr
eadi
ng
Ø
MQ
so
cket
s ar
e n
ot
thre
ad s
afe!
B
ut
they
are
ext
rem
ely
ligh
twei
gh
t
Cre
ate
one
(or
mor
e) s
ocke
ts p
er th
read
U
se th
ese
ØM
Q s
ocke
ts to
exc
hang
e m
essa
ges
betw
een
the
thre
ads
U
se a
pro
xy t
o d
istr
ibu
te w
ork
am
on
g t
he
thre
ads
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
32iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Mul
tithr
eade
d W
orke
r
void
wor
kerT
hrea
d(zm
q::c
onte
xt_t
& c
onte
xt)
{zm
q::s
ocke
t_tv
enti
lato
rPro
xy(c
onte
xt, Z
MQ
_PU
LL
);ve
ntil
ator
Pro
xy.c
onne
ct("
inpr
oc://
wor
kers
");
zmq:
:soc
ket_
tsin
k(co
ntex
t, Z
MQ
_PU
SH
);si
nk.c
onne
ct("
tcp:
//si
nk:5
558"
);
whi
le (
1) {
zmq:
:mes
sage
_tta
sk;
vent
ilat
orP
roxy
.rec
v(&
task
);
zmq:
:mes
sage
_tre
sult
= d
oSom
eWor
k(ta
sk);
sink
.sen
d(re
sult
);}
}
48
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
33iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Mul
tithr
eade
d W
orke
rin
tmai
n()
{zm
q::c
onte
xt_t
cont
ext(
1);
zmq:
:soc
ket_
tven
tila
torP
roxy
(con
text
, ZM
Q_P
UL
L);
vent
ilat
orP
roxy
.con
nect
("tc
p://
loca
lhos
t:55
60")
;zm
q::s
ocke
t_tw
orke
rs(c
onte
xt, Z
MQ
_PU
SH
);w
orke
rs.b
ind(
"inp
roc:
//wor
kers
");
std:
:vec
tor
< s
td::
thre
ad >
thre
adP
ool;
for
(std
::si
ze_t
i= 0
; i<
std
::th
read
::ha
rdw
are_
conc
urre
ncy(
); +
+i)
{th
read
Poo
l.pus
h_ba
ck(s
td::
thre
ad([
&](
) {
wor
kerT
hrea
d(co
ntex
t); /
/ will
con
nect
with
inpr
oc://
wor
kers
}));
} zmq:
:pro
xy(v
enti
lato
rPro
xy, w
orke
rs, N
UL
L);
}
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
34iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
ØM
Q –
Not
es
W
ith
ØM
Q m
essa
ges
sti
ll n
eed
to
be
tran
slat
ed t
o p
roce
du
re
exec
uti
on
s
O
bje
ct s
eria
lizat
ion
has
to
be
imp
lem
ente
d o
n t
op
of
ØM
Q
T
her
e's
mu
ch m
ore
fu
nct
ion
alit
y in
ØM
Q!
R
ead
th
e g
reat
gu
ide:
htt
p:/
/zg
uid
e.ze
rom
q.o
rg
T
he
exam
ple
s in
th
is le
ctu
re a
re b
ased
on
th
e ex
amp
les
fro
m
the
zgu
ide
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
35iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Out
line
M
otiv
atio
n
B
oost
.Asi
o
M
essa
ge P
assi
ng
Ø
MQ
A
pac
he
Th
rift
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
36iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Apa
che
Thr
ift
R
emo
te P
roce
du
re C
alls
(R
PC
s):
Exe
cuti
ng
su
bro
uti
nes
(fu
nct
ion
s, m
eth
od
s) o
n a
pro
gra
m
run
nin
g r
emo
tely
T
hri
ft is
a s
cala
ble
cro
ss-l
ang
uag
e R
PC
fra
mew
ork
dev
elo
ped
b
y F
aceb
oo
k
It im
plem
ents
the
mis
sing
obj
ect s
eria
lizat
ion
It
does
not
offe
r
It
's a
n o
pen
so
urc
e p
roje
ct in
th
e A
pac
he
So
ftw
are
Fo
un
dat
ion
49
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
37iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Apa
che
Thr
ift
T
he
dev
elo
per
def
ines
ser
vice
s in
an
Inte
rfac
e D
efin
itio
n
Lan
gu
age
(ID
L)
file
T
hri
ft g
ener
ates
co
de
(In
terf
aces
) to
be
use
d t
o c
all t
hes
e se
rvic
es r
emo
tely
E
.g. c
allin
g a
Java
Met
hod
from
a P
HP
scr
ipt r
unni
ng o
n a
rem
ote
host
Serv
iceD
efini
tion.
thrif
t
Serv
er A
pplic
atio
n
Thrif
tSer
vice
Serv
er.c
pp
Clie
nt A
pplic
atio
n
Thrif
tSer
vice
Clie
nt.c
pp
thrift --gen cpp
RPC
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
38iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Thr
ift –
Inte
rfac
e D
efin
ition
In
terf
ace
Def
init
ion
Lan
gu
age
(.th
rift
) fi
les
D
efin
e na
mes
pace
, dat
a st
ruct
ures
, typ
es, m
etho
ds, s
ervi
ces
S
imila
r to
C s
ynta
x
Bas
ic ty
pes
are
bool
, byt
e, i1
6/32
/64,
dou
ble,
str
ing,
m
ap<
t1,t2
>, l
ist<
t1>
, set
<t1
>
nam
espa
cecp
pch
.cer
n.ic
sc14
enum
Ope
rati
on {
AD
D =
1,
SU
BT
RA
CT
= 2
,M
ULT
IPLY
= 3
,D
IVID
E =
4}
stru
ctW
ork
{1:
i32
num
1,2:
i32
num
2,3:
Ope
rati
on o
p} se
rvic
eC
alcu
lato
r {
i32
calc
ulat
e(1:
Wor
k w
)}
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
39iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Thr
ift –
Com
pilin
g T
hrift
File
s
T
hri
ft c
om
pile
s th
e ID
L f
iles
to s
erve
r (a
nd
clie
nt)
so
urc
e co
de
It
gen
erat
es t
ho
usa
nd
s o
f lin
es o
f co
de
wit
h p
lace
ho
lder
s
C
alcu
lato
r_se
rver
.ske
leto
n.c
pp
:us
ing
nam
espa
ce :
:ch:
:cer
n::ic
sc14
;cl
ass
Cal
cula
torH
andl
er: v
irtu
al p
ubli
c C
alcu
lato
rIf
{pu
blic
:C
alcu
lato
rHan
dler
() {
// Y
our
initi
aliz
atio
n go
es h
ere
} in
t32_
t cal
cula
te(c
onst
Wor
k& w
) {
// Yo
ur im
plem
enta
tion
goes
her
e} };
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
40iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Thr
ift –
Doc
umen
tatio
n
T
her
e is
on
ly v
ery
littl
e d
ocu
men
tati
on
on
line
U
sefu
l lin
ks:
ht
tp://
wik
i.apa
che.
org/
thrif
t/Thr
iftU
sage
ht
tp://
thrif
t-tu
toria
l.rea
dthe
docs
.org
/
http
://w
ww
.slid
esha
re.n
et/d
virs
ky/in
trod
uctio
n-to
-thr
ift
http
://di
wak
ergu
pta.
gith
ub.io
/thrif
t-m
issi
ng-g
uide
Goo
d L
uck!
50
Net
wo
rk P
rog
ram
min
g
Lec
ture
2iC
SC
2014
24
-25
Feb
ruar
y 20
14, C
ER
N
Bu
ildin
g H
igh
ly D
istr
ibu
ted
Sys
tem
s W
ith
in 5
Min
ute
s
41iC
SC
2014
, Jo
nas
Ku
nze
, U
niv
ersi
ty o
f M
ain
z –
NA
62
Sum
mar
y
T
her
e is
no
nat
ive
C+
+ li
bra
ry f
or
net
wo
rk p
rog
ram
min
g
T
her
e ar
e m
any
dif
fere
nt
libra
ries
fo
r d
iffe
ren
t p
urp
ose
s
Boo
st.A
sio
for
easy
asy
nch
ron
ou
san
d m
ult
ith
read
edso
cket
pr
ogra
mm
ing
Ø
MQ
add
ition
ally
pro
vide
s m
essa
ge
pas
sin
gan
d he
lpfu
l p
atte
rns
A
pach
e T
hrift
pro
vide
s an
effi
cien
t RP
C f
ram
ewo
rk
A
ll th
ese
libra
ries
are
cro
ss-p
latf
orm
cap
able
Ø
MQ
an
d T
hri
ft p
rovi
de
inte
rfac
es f
or
man
y la
ng
uag
es
Vis
it ht
tps:
//gith
ub.c
om/J
onas
Kun
zefo
r co
de s
nipp
ets
and
thes
e sl
ides
51
52
LECTURE 4 From Quark to Jet: A Beautiful Journey Beauty physics, tracking and large-scale distributed computing in HEP
Tuesday 25 February
09:00-10:00
Description The beauty, or bottom, quark is an extremely powerful object in high energy physics. Distinctive characteristics of the decay of the quark have motivated the design high-energy physics detectors in the quest to reconstruct the quark. Each component of the detector is used as a tool to identify these characteristics and present unique challenges that are solved with a combination of engineering, physics, and computing. These two lectures will explore the tools used and the different physics and computing environments they are used in, and the important interplay between the two environments and limitations they set on each will be considered. When all components are reassembled, the quark is described as a jet This lecture will briefly introduce beauty physics and it's role in HEP. Attention will be paid to the distinctive features of the decay of the beauty quark, such as secondary vertices, leptonic decay, and hadronization features that require all subsystems of a detector to be used in it's reconstruction. Then an introduction to a track reconstruction algorithm with particular attention paid to the performance and precision of different seeding and/or iterative methods. The implementation of the algorithms into an experiment wide software infrastructure, and the requirements of large-scale computing, will be explored. The performance of these on large scale computing infrastructures will be presented. Finally, the expected impact of future LHC running and the challenges this presents to current algorithms and performance, along with the concepts used to solve these problems are presented.
Tyler Mc Millan
Dorland
DESY - Hamburg
Audience The target audience of this lecture is broad between physicists and computer scientists, but perhaps more focused on computer scientists with only some knowledge of object reconstruction. The benefits of following this lecture is the understand why the b quark is an exceptional object used in HEP, cursory understanding of differences between and complexities of track reconstruction algorithms, understanding of the implementation of a physical algorithm into a large computing software infrastructure and the importance of the tracking step in the overall performance of event reconstruction, and the techniques used and concessions made by both physics performance and computing time in preparing for the high luminosity phase of Run II. Pre-requisite As prerequisite this lecture will not require specific knowledge, but will include some high-level explanations for those with extensive knowledge of physics detectors or distributed computing.
53
54
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
1iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Fro
m Q
uark
to
Jet:
A B
eaut
iful J
ourn
eyLe
ctur
e1
Bea
uty
Ph
ysic
s, T
rack
ing
, an
d
Dis
trib
ute
d C
om
pu
tin
g
Tyl
er D
orl
and
Deu
tsch
esE
lekt
ron
en-S
ynch
rotr
on
(D
ES
Y)
Inve
rted
CE
RN
Sch
oo
l of
Co
mp
uti
ng
, 24-
25 F
ebru
ary
2014
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
2iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
3iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Hug
e nu
mbe
rs o
f co
mpl
ex e
quat
ions
Ent
irel
y S
imul
ated
, pa
rtic
les
are
subj
ecte
d to
de
cay
cond
ition
s
Det
ecto
r si
mul
atio
n,A
lgor
ithm
ic r
econ
stru
ctio
n
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
4iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Hug
e nu
mbe
rs o
f co
mpl
ex e
quat
ions
Ent
irel
y S
imul
ated
, pa
rtic
les
are
subj
ecte
d to
de
cay
cond
ition
s
Det
ecto
r si
mul
atio
n,A
lgor
ithm
ic r
econ
stru
ctio
n
Dis
trib
uted
Com
putin
g, L
apto
ps, L
ocal
Ser
vers
55
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
5iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Hug
e nu
mbe
rs o
f co
mpl
ex e
quat
ions
Ent
irel
y S
imul
ated
, pa
rtic
les
are
subj
ecte
d to
de
cay
cond
ition
s
Det
ecto
r si
mul
atio
n,A
lgor
ithm
ic r
econ
stru
ctio
n
Dis
trib
uted
Com
putin
g, L
apto
ps, L
ocal
Ser
vers
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
6iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Ask
Que
stio
ns h
ere
U
se t
heo
ry t
o m
ake
pre
dic
tio
ns
for
ob
serv
able
s o
f p
arti
cles
D
esig
n d
etec
tors
to
det
ect
thes
e o
bse
rvab
les
A
lgo
rith
ms
to r
emak
e th
e o
bje
cts
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
7iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Bea
uty
Phy
sics
-T
heor
y
B
eau
ty q
uar
k d
isco
vere
d
in 1
977
at F
erm
ilab
F
irst
th
ird
gen
erat
ion
q
uar
k
B
eau
ty (
and
ch
arm
) q
uar
ks
hav
e a
lifet
ime
that
allo
ws
for
dec
ay l
eng
ths
of
a fe
w
mill
imet
ers
T
op is
too
shor
t, up
/dow
n/ch
arm
is to
o lo
ng
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
8iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Bea
uty
Phy
sics
-T
heor
y
b
-jet
s ar
e ex
trem
ely
po
wer
ful b
ackg
rou
nd
re
du
cers
M
any
impo
rtan
t si
gnal
s ha
ve b
-qua
rks
H
ug
e o
rder
of
mag
nit
ud
e re
du
ctio
n f
rom
iden
tify
ing
b
-qu
arks
V
ery
imp
ort
ant
too
l
> 10 orders of magnitude
2orders of magnitude
pp
->an
yth
ing
pp
->b
eau
ties
10 x
pp
->X
-> b
eau
ties
56
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
9iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Had
roni
zatio
n
M
ost
cal
cula
tio
ns
are
con
fin
ed t
o s
imp
le e
lem
ents
W
hat
we
actu
ally
mea
sure
is m
uch
mo
re c
om
plic
ated
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
10iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Har
doni
zatio
n
W
e kn
ow
th
at a
s q
uar
ks
get
fu
rth
er a
way
fro
m
each
oth
er t
hey
mak
e p
airs
wit
h o
ther
qu
arks
T
hese
are
cal
led
hadr
ons
H
adro
niz
atio
nd
epen
ds
on
man
y ex
per
imen
tally
ad
just
ed f
acto
rs
M
ost
imp
ort
antl
y w
e ca
n
beg
in t
o lo
ok
at e
ven
t to
po
log
y
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
11iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Bea
uty
Phy
sics
–P
artic
le L
evel
If
a b
-qu
ark
is p
aire
d
wit
h a
n s
-qu
ark
the
resu
ltin
g m
eso
n, B
_s,
has
a lo
ng
life
tim
e, a
nd
so
me
very
in
tere
stin
g
dec
ay s
ign
atu
res
W
e u
se t
hes
e p
arti
cula
r d
ecay
sig
nat
ure
s to
d
eter
min
e w
hat
ex
per
imen
tal s
ign
atu
re
we
wan
t to
see
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
12iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
erim
enta
l sig
natu
re
N
ow
we
hav
e a
dis
tin
ct
sig
nat
ure
to
sea
rch
fo
r
A s
econ
dary
ver
tex
Je
t
Dis
plac
ed tr
ack
Le
pton
R
are,
bu
t n
ot
un
iqu
e
We
will
use
diff
eren
t te
chni
ques
to c
lass
ify
E
ssen
tially
a p
roba
bilit
y th
e je
t cam
e fr
om a
b-
quar
k
57
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
13iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Bea
uty
Phy
sics
–D
etec
tor
Leve
l
Sec
onda
ry v
erte
xJe
t
Dis
plac
ed tr
acks
Lept
on
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
14iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
U
se t
heo
ry t
o m
ake
pre
dic
tio
ns
for
ob
serv
able
s o
f p
arti
cles
D
esig
n d
etec
tors
to
det
ect
thes
e o
bse
rvab
les
R
eco
nst
ruct
ion
alg
ori
thm
s to
re
mak
e th
e o
bje
cts
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
15iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Tra
ckin
g -
an In
trod
uctio
n
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
16iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Fitt
ing
A
n n
th d
egre
e p
oly
no
mia
l w
ill e
xact
ly f
it (
n+
1) p
oin
ts
T
her
efo
re, a
ny
thre
e p
oin
ts
can
be
fit
wit
h a
cir
cle
F
its
gen
eral
ly c
lass
ifie
d b
y d
ista
nce
s o
f p
oin
ts t
o f
itte
d
curv
e (c
hi-
squ
ared
)
F
or
nth
deg
ree
po
lyn
om
ial,
n+
2 …
n+
mp
oin
ts a
re
deg
rees
of
free
do
mP
olyn
omia
l fits
to a
sin
e cu
rve
58
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
17iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Tra
ck S
eedi
ng a
nd R
econ
stru
ctio
n
In
sid
e o
f th
e co
llisi
on
re
gio
n w
e w
ill h
ave
man
y h
its
we
can
ass
oci
ate
wit
h
a p
rim
ary
vert
ex
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
18iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Tra
ck S
eedi
ng a
nd R
econ
stru
ctio
n
C
ho
ose
an
init
ial s
et o
f la
yers
th
at w
e n
ame
the
“see
din
g la
yers
” th
at
pro
vid
e an
init
ial e
stim
ate
of
trac
k p
aram
eter
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
19iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Tra
ck S
eedi
ng a
nd R
econ
stru
ctio
n
C
ho
ose
an
init
ial s
et o
f la
yers
th
at w
e n
ame
the
“see
din
g la
yers
” th
at
pro
vid
e an
init
ial e
stim
ate
of
trac
k p
aram
eter
T
hen
co
llect
all
po
ssib
le
hit
s as
soci
ated
wit
h
dif
fere
nt
seed
s
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
20iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Tra
ck S
eedi
ng a
nd R
econ
stru
ctio
n
C
ho
ose
an
init
ial s
et o
f la
yers
th
at w
e n
ame
the
“see
din
g la
yers
” th
at p
rovi
de
an in
itia
l est
imat
e o
f tr
ack
par
amet
er
T
hen
co
llect
all
po
ssib
le h
its
asso
ciat
ed w
ith
dif
fere
nt
seed
s
U
sin
g t
ech
niq
ues
to
est
imat
e th
e g
oo
dn
ess
of
the
fit
we
can
th
en e
stim
ate
the
fin
al
trac
k p
aram
eter
s
59
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
21iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Fak
e R
emov
al
C
ho
ose
an
init
ial s
et o
f la
yers
th
at w
e n
ame
the
“see
din
g
laye
rs”
that
pro
vid
e an
init
ial
esti
mat
e o
f tr
ack
par
amet
er
T
hen
co
llect
all
po
ssib
le h
its
asso
ciat
ed w
ith
dif
fere
nt
seed
s
U
sin
g t
ech
niq
ues
to
est
imat
e th
e g
oo
dn
ess
of
the
fit
we
can
th
en e
stim
ate
the
fin
al
trac
k p
aram
eter
s
A
nd
rem
ove
hit
s n
ot
asso
ciat
ed w
ith
go
od
tra
cks
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
22iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Itera
tive
Tra
ckin
g
W
ith
iter
ativ
e tr
acki
ng
cer
tain
q
ual
ity
trac
ks c
an b
e ch
ose
n
and
th
en r
emo
ved
fro
m
furt
her
insp
ecti
on
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
23iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Itera
tive
Tra
ckin
g
W
ith
iter
ativ
e tr
acki
ng
cer
tain
q
ual
ity
trac
ks c
an b
e ch
ose
n
and
th
en r
emo
ved
fro
m
furt
her
insp
ecti
on
T
hen
use
th
e re
mai
nin
g h
its
to c
reat
e th
e re
mai
nin
g t
rack
s
A
fter
man
y it
erat
ion
s w
e en
d
wit
h t
he
fin
al s
et o
f tr
acks
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
24iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Rea
l dat
a ex
ampl
es
60
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
25iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Look
ing
Tow
ards
201
5 Cur
rent
alg
orith
ms
wer
e de
velo
ped
cons
ider
ing
the
run
cond
ition
s fo
r 20
11-2
012
whe
re
ther
e w
as a
n av
erag
e of
20
inte
ract
ions
per
bun
ch c
ross
ing
For
201
5, th
ere
coul
d be
ove
r 40
inte
ract
ions
on
aver
age
With
no
chan
ges,
the
com
putin
g po
wer
nee
ded
coul
d be
6 ti
mes
wha
t is
cur
rent
ly
used
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
26iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Tra
ckin
g
C
har
ged
par
ticl
es m
ake
curv
es in
m
agn
etic
fie
lds
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
27iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
CM
S C
ompu
ting
Net
wor
k
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
28iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
CM
S C
urre
nt E
vent
Mod
el
G
lob
al c
on
fig
ura
tio
ns
are
load
ed in
to m
emo
ry
The
n co
nfig
urat
ions
spe
cific
to th
e sp
ecifi
c tim
e of
run
ning
E
ven
ts t
hen
pro
cess
ed s
eria
lly
T
he
mo
st t
ime
inte
nsi
ve p
art
of
even
t re
pro
cess
ing
is t
rack
ing
61
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
29iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Am
dahl
’s L
aw
A
md
ahl’s
Law
is t
he
up
per
lim
it o
n t
he
spee
du
p g
ain
ed b
y a
nu
mb
er o
f p
roce
sso
rs
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
30iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
CM
S T
hrea
ded
Des
ign
E
ven
ts a
re n
ot
seen
glo
bal
ly
M
ult
iple
eve
nts
are
ru
n c
on
curr
entl
y
Less
bac
kup
from
ver
y co
mpl
icat
ed e
vent
s
S
trea
ms
still
pro
cess
ser
ially
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
31iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Con
curr
ent P
roce
ssin
g In
side
an
Eve
nt
C
urr
ent
even
t P
roce
ssin
g
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
32iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
T
hre
adin
g in
sid
e o
f a
mo
du
le
Con
curr
ent P
roce
ssin
g In
side
an
Eve
nt
62
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
33iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Per
form
ance
Res
ults
S
ing
le t
hre
aded
ru
ns
ou
t o
f m
emo
ry a
t 30
00 s
imu
ltan
eou
s ev
ents
D
efin
ite
imp
rove
men
t th
rou
gh
mu
ltit
hre
adin
g
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
34iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Con
clus
ions
B
eau
ty p
hys
ics
is a
ver
y d
iver
se a
nd
larg
e p
art
of
hig
h e
ner
gy
ph
ysic
s
B
-Had
ron
s h
ave
dis
tin
gu
ish
ing
tra
its
we
can
use
to
mak
e b
-je
ts v
ery
po
wer
to
ols
fo
r b
ackg
rou
nd
red
uct
ion
T
o m
ake
use
of
this
, we
mu
st u
se in
form
atio
n f
rom
man
y p
arts
of
the
det
ecto
r w
hic
h a
ll re
qu
ire
thei
r o
wn
re
con
stru
ctio
n a
lgo
rith
ms
and
dif
fere
nt
leve
ls o
f co
mp
uti
ng
re
sou
rces
B
y re
stru
ctu
rin
g t
he
even
t p
roce
ssin
g s
tru
ctu
re t
o
acco
mm
od
ate
thre
aded
ap
plic
atio
ns
we
can
mee
t th
e d
eman
ds
req
uir
ed f
or
trac
kin
g in
th
e fu
ture
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
35iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
1
36iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Spe
edup
from
63
64
LECTURE 5 Read-Out Electronics: where data come from
Tuesday 25 February
10:00-11:00
Description This lecture gives a general overview on the evolution of detectors used in HEP and on the necessity of electronics in modern set-ups. The second part of the lecture is focused on the Read-Out Electronics (ROE) as a transmission line: information are transported from the detector to the Data Storage; at high frequency it is important to take into account particular characteristics (such as signal propagation and integrity, interaction between signals and electromagnetic interferences) of the signal even if it is a digital one.
Francesco Messi
Rheinische Friedrich-Wilhelms University
Bonn - DE
Audience This lecture targets everyone interested in the basic concepts of transmission lines. After this lecture the attendees are expected to have acquired a basic knowledge of the important concepts, properties and limitations of transmission lines (bandwidth, impedance, crosstalk, etc...) and of the standard read-out electronics used in modern HEP experiments.
Pre-requisite This lecture can be followed by anyone having interest in the subject and basic knowledge in electronics and detectors. Nevertheless the main concepts explained in this lecture will be understandable by everyone.
65
66
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
1iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Rea
d-O
ut E
lect
roni
cs:
whe
re d
ata
com
e fr
om
Lect
ure
1
Fra
nce
sco
Mes
si
Rh
ein
isch
eF
ried
rich
-Wilh
elm
s-U
niv
ersi
tät
Bo
nn
–D
E
Inve
rted
CE
RN
Sch
oo
l o
f C
om
pu
tin
g,
24-2
5 F
ebru
ary
2014
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
2iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Out
look
W
hy
we
bu
ild d
etec
tors
R
es
ea
rch
in P
hys
ics
:
A lo
ok a
t yes
terd
ay
A lo
ok a
t tod
ay
T
he
ele
ctr
on
ics
T
he R
ead-
Out
Ele
ctro
nics
F
rom
the
anal
og to
the
digi
tal w
orld
C
hara
cter
istic
s/re
quire
men
ts o
f an
elec
tron
ics
chai
n
A
n e
xa
mp
le:
Pa
rtic
le ID
en
tifi
ca
tio
n(P
ID)
us
ing
th
e T
ime
of
Fli
gh
t (T
oF
)
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
3iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Out
look
A
s h
um
an
s,
we
are
cu
rio
us
: w
e w
an
t to
kn
ow
th
e p
lac
e w
e li
ve
in, w
e w
an
t to
un
de
rsta
nd
wh
at
the
ma
tte
r is
ma
de
of!
!!
F
or
tha
t, w
e b
uil
d m
ac
hin
es
(e
xp
eri
me
nts
an
d d
ete
cto
rs)
to
inve
sti
ga
te t
he
ma
tte
r
…it
is im
port
ant t
o us
e co
rrec
tly th
e to
ols
we
have
!!!
Why
do
we
buil
d de
tect
ors?
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
4iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Out
look
20.1
2.19
47 -
G.D
. Roc
hest
er &
C.C
. But
ler,
Evid
ence
for t
he
exis
tenc
e of
a n
ew u
nsta
ble
elem
enta
ry p
artic
les,
Nat
ure
160,
85
5
Why
do
we
buil
d de
tect
ors?
W
e w
ant
to s
tud
y w
ha
t w
e c
an
no
t s
imp
ly s
ee
wit
h o
ur
eye
s
F
or
tha
t w
e “
as
k”
the
de
tec
tors
to
re
co
rd a
sc
en
e
S
o t
ha
t w
e c
an
an
aly
ze it
An
even
t in
the
AT
LA
S d
etec
tor
toda
yye
ster
day
67
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
5iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Res
earc
h in
Phy
sics
a ve
ry g
ener
al o
verv
iew
of t
he e
volu
tion
of th
e de
tect
ors
used
in e
xper
imen
tal s
et-u
ps
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
6iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Res
earc
h in
Phy
sics
Nuc
lear
phy
sics
res
earc
h w
as b
ased
on
obse
rvat
ion
of tr
acks
fro
m c
harg
ed p
arti
cles
A lo
ok a
t yes
terd
ay…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
7iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Year
ParticleInstrument
1947
π+an
d π-
Nuc
lear
Em
ulsi
on
π0
Cou
nter
s an
d E
mul
sion
ΛC
loud
Cha
mbe
r
1947
K+an
d K
-N
ucle
ar E
mul
sion
K0
Clo
ud C
ham
ber
1953
Σ+N
ucle
ar E
mul
sion
Σ-C
loud
Cha
mbe
r
Σ0B
ubbl
e ch
ambe
r
Ξ-
Clo
ud C
ham
ber
Ξ0
Bub
leC
ham
ber
1958
Ant
i Λ0
Nuc
lear
Em
ulsi
on
Dis
cove
ry o
f “e
lem
enta
ry”
part
icle
s…
Res
earc
h in
Phy
sics
A lo
ok a
t yes
terd
ay…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
8iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Res
earc
h in
Phy
sics
N
uc
lea
r e
mu
lsio
n:
P
hoto
grap
hic
emul
sion
wer
e se
nd to
the
high
atm
osph
ere
In
tera
ctio
n w
ith t
he c
osm
ic r
ays
wou
ld le
ave
a tr
ack
S
tudy
ing
the
trac
ks it
is p
ossi
ble
to id
entif
y di
ffere
nt p
artic
les
and
deca
ys:
T
he g
rain
den
sity
is p
ropo
rtio
nal t
o th
e en
ergy
loss
by
ioni
zatio
n (B
ethe
-B
loch
)
dire
ctio
n an
d (~
)ene
rgy
A
big
dev
iatio
n fr
om a
traj
ecto
ry
inte
ract
ion
and
new
par
ticle
s
A
neu
tral
par
ticle
doe
s no
t lea
ve a
sig
natu
re
e.g.
use
d to
day
in th
e O
PE
RA
exp
erim
ent
(Osc
illa
tion
Pro
ject
wit
h E
mul
sion
-tR
acki
ngA
ppar
atus
)
A lo
ok a
t yes
terd
ay…
68
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
9iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
E
xp
eri
me
nta
l se
t-u
p m
ad
e o
f a
larg
e n
um
be
r o
f d
ete
cto
rs
Inve
stig
atio
n of
the
“con
stit
uent
s of
mat
ter”
Loo
king
for
the
“Fun
dam
enta
l pa
rtic
les”
e.
g. H
iggs
Bos
on
Exp
lora
tion
of
“rar
e ch
anne
ls”
Res
earc
h in
Phy
sics
A lo
ok a
t tod
ay…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
10iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
E
xp
eri
me
nta
l se
t-u
p m
ad
e o
f a
larg
e n
um
be
r o
f d
ete
cto
rs:
T
rack
er: d
etec
ting
the
pass
age
of a
cha
rge
part
icle
idea
lly w
ithou
t pe
rtur
batio
n of
its
traj
ecto
ry
(e.g
. sili
con
strip
s or
pix
els)
C
alor
imet
er: m
easu
ring
the
ener
gy r
elea
sed
by a
par
ticle
in th
e in
tera
ctio
n w
ith t
he d
etec
tors
(e
.g. i
norg
anic
sci
ntill
ator
s)
T
ime
of F
light
: sui
ted
to m
easu
re th
e tim
e of
flig
ht o
f the
par
ticle
Alw
ays
at le
ast i
n co
uple
: one
sta
rt a
nd o
ne s
top
time
need
ed
et
c…
E
ac
h d
ete
cto
r c
an
ac
qu
ire
da
ta w
ith
a r
ate
of
MH
z o
r m
ore
…
U
sa
ge
of
fas
t e
lec
tro
nic
s is
ne
ed
ed
to
re
ad
th
e d
ata
.
Res
earc
h in
Phy
sics
A lo
ok a
t tod
ay…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
11iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Res
earc
h in
Phy
sics
A b
rief
sum
mar
y…
In
th
e l
as
t c
en
tury
th
ere
wa
s a
big
evo
luti
on
of
de
tec
tors
N
uc
lea
r p
hys
ics
re
se
arc
h w
as m
ain
ly b
as
ed
on
th
e o
bs
erv
ati
on
o
f tr
ac
ks
fro
m c
ha
rge
d p
art
icle
s
D
ete
cto
rs w
ere
“pu
rely
an
alo
g”
de
vic
es
E
ac
h d
ete
cto
r w
as
bu
ilt
to p
erf
orm
th
e f
ull
ex
pe
rim
en
t
T
he
se
arc
h f
or
the
co
ns
titu
en
t o
f th
e m
att
er
req
uir
es
:
high
ene
rgie
s,
very
sho
rt ti
mes
,
“rar
e ch
anne
ls”
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
12iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
M
od
ern
ex
pe
rim
en
tal s
et-
up
s a
re m
ad
e o
f a
larg
e n
um
be
r o
f d
iffe
ren
t d
ete
cto
rs, e
ac
h lo
ok
ing
at
a p
art
icu
lar
as
pe
ct
M
od
ern
de
tec
tors
are
pro
du
cin
g a
na
log
ele
ctr
ica
l sig
na
ls
D
ete
cto
rs a
re “
talk
ing
” th
rou
gh
th
e e
lec
tro
nic
s
Res
earc
h in
Phy
sics
A b
rief
sum
mar
y…
69
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
13iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
ele
ctro
nics
The
Rea
d-O
ut E
lect
roni
cs, u
sed
to c
omm
unic
ate
with
th
e de
tect
ors,
is th
e to
ol u
sed
to tr
ansp
ort i
nfor
mat
ion
from
the
anal
og w
orld
(sp
oken
by
dete
ctor
s) to
the
digi
tal w
orld
(sp
oken
by
the
Dat
a S
tora
ge S
yste
m)
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
14iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
Rea
d-O
ut E
lect
roni
cs
Bin
ary
wor
ldH
igh
tran
smis
sion
rat
e
Fro
m a
litt
le, f
ast e
lect
rica
l sig
nal t
o a
bit-
stre
am o
f da
ta
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
15iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
Rea
d-O
ut E
lect
roni
csF
rom
a li
ttle
, fas
t ele
ctri
cal s
igna
l to
a bi
t-st
ream
of
data
Fro
nt-E
ndE
lect
roni
csP
re-A
naly
sis
Ele
ctro
nics
Ana
log
sign
alD
igit
al s
igna
lB
it-s
trea
m o
f da
ta
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
16iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
Rea
d-O
ut E
lect
roni
cs
①F
ron
t-E
nd
Ele
ctr
on
ics
: fr
om
th
e a
na
log
to
th
e d
igit
al w
orl
d•
Com
para
tors
•A
nalo
g to
Dig
ital C
onve
rter
s (A
DC
)•
Tim
e to
Dig
ital C
onve
rter
s (T
DC
)
②S
ign
al
pro
ce
ss
ing
: th
e F
PG
A•
Whe
re th
ey a
re u
sed…
•F
or w
hat…
•W
hy…
In tw
o m
ajor
ste
ps:
In le
ctur
e 2
70
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
17iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e an
alog
to th
e di
gita
l wor
ld…
Am
plitu
de c
an v
ary
to a
ny v
alue
Shap
e ca
n va
ryB
inar
y lo
gic:
“1”
or
“0”
Squ
are
sign
al
D
igit
al “
ON
E”
if t
he
sig
na
l is
ab
ove
a f
ixe
d t
hre
sh
old
A
pp
ling
a t
hre
sho
ld w
e d
ecid
e (d
iscr
imin
ate)
wh
at is
no
ise
and
wh
at is
a g
oo
d s
ign
al
1) C
ompa
rato
r
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
18iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e an
alog
to th
e di
gita
l wor
ld…
D
igit
al w
ord
pro
po
rtio
na
l to
th
e c
ha
rge
of
the
an
alo
g s
ign
al
* P
ictu
re f
rom
: htt
p://
dev.
emce
lett
roni
ca.c
om/
For
a d
ynam
ic s
igna
l: S
ampl
ing-
AD
C
2) A
nalo
g to
Dig
ital
Con
vert
er: A
DC
…ta
lkin
g ab
out s
ampl
ing…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
19iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e an
alog
to th
e di
gita
l wor
ld…
2) A
nalo
g to
Dig
ital
Con
vert
er: A
DC
B
an
dw
idth
giv
es
info
rma
tio
n a
bo
ut
ho
w f
as
t is
th
e d
evi
ce
S
amp
ling
Rat
e g
ives
in
form
atio
n a
bo
ut
ho
w a
ccu
rate
is t
he
dig
ita
liza
tio
n o
f th
e s
ign
al s
ee
n b
y th
e d
evi
ce
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
20iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e an
alog
to th
e di
gita
l wor
ld…
D
igit
al w
ord
pro
po
rtio
na
l to
th
e t
ime
of
the
sig
na
l
3) T
ime
to D
igit
al C
onve
rter
: TD
C
71
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
21iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e an
alog
to th
e di
gita
l wor
ld…
…an
d m
any
mor
e…
Info
rmat
ion
in b
inar
y lo
gic…
Fam
ily
On
e (V
)Z
ero
(V
)
TT
L2
to 5
0 to
0.8
NIM
3 to
12
-2 t
o 1.
5
EC
L-0
.81
to -
1.13
-1.4
8 to
-1.
95
LVD
Sp
1.27
to 2
.40
0.92
to
1.12
n0.
92 t
o 1.
121.
27 t
o 2.
4
D
igit
al s
ign
al
two
sta
tes
: p
res
en
t o
r a
bs
en
t, “
1”
or
“0”
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
22iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Cha
ract
eris
tics
of a
n el
ectr
onic
s ch
ain
to
re
ce
ive
th
e c
orr
ec
t in
form
ati
on
R
elia
bil
ity
to
re
ce
ive
th
e f
ull
info
rma
tio
n t
ran
sm
itte
d
Eff
icie
nc
y
to
un
der
stan
d t
he
mes
sag
e
No
dis
tort
ion
we
wan
t la
rge
ba
nd
wid
th
Wha
t do
we
wan
t fro
m o
ur e
lect
roni
cs c
hain
?
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
23iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
ele
ctro
nics
A b
rief
sum
mar
y…
E
lec
tro
nic
s is
ne
ed
ed
to
re
ad
ou
t th
e in
form
ati
on
fro
m d
ete
cto
rs
S
eve
ral s
tag
es
of
dif
fere
nt
ele
ctr
on
ics
are
ne
ed
ed
F
ina
l go
al i
s t
o c
on
vert
info
rma
tio
n f
rom
a li
ttle
, fa
st
an
alo
g
sig
na
l in
a b
it-s
tre
am
of
da
ta
T
he
ele
ctr
on
ics
ch
ain
mu
st:
P
repr
oces
s th
e si
gnal
s w
/o d
isto
rtio
n
Be
relia
ble
B
e ef
ficie
nt
Be
fast
B
e ca
libra
ted
to th
e ex
perim
ent!!
!
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
24iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple…
Par
ticle
iden
tific
atio
n ca
n be
mad
e in
man
y w
ays
and
with
diff
eren
t set
-ups
; in
this
exa
mpl
e pr
oton
s an
d po
sitiv
e pi
ons
are
iden
tifie
d by
the
mea
sure
of t
he
Tim
e of
Flig
ht in
cor
rela
tion
with
the
ir m
omen
tum
.
The
ele
ctro
nics
use
d fo
r th
e m
easu
rem
ent o
f the
tim
e ca
n be
cru
cial
for
the
kind
of p
hysi
cs o
ne w
ants
to
inve
stig
ate…
72
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
25iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple:
PID
usi
ng T
oF
Im
ag
ine
yo
u n
ee
d t
o id
en
tify
Pro
ton
s f
rom
Pio
ns
(+
)
W
e k
no
w f
rom
pa
rtic
le p
hys
ics
th
at,
fo
r a
fix
ed
mo
me
ntu
m, t
he
ve
loc
ity
of
the
tw
o p
art
icle
s a
re d
iffe
ren
t
we
ca
n i
de
nti
fy t
he
pa
rtic
les
me
as
uri
ng
th
e T
ime
of
Fli
gh
t
Par
ticl
e ID
enti
fica
tion
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
26iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
W
e m
ea
su
re t
he
To
Fo
f th
e p
art
icle
s
W
e p
lot
the
m o
n h
isto
gra
ms
W
e m
us
t b
e a
ble
to
dis
tin
gu
ish
th
e t
wo
tim
e d
istr
ibu
tio
ns
An
exam
ple:
PID
usi
ng T
oFP
arti
cle
IDen
tifi
cati
on
Sep
arat
ion
Pow
er
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
27iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple:
PID
usi
ng T
oFS
et-u
p pr
inci
ple
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
28iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple:
PID
usi
ng T
oF
L
et’
s f
oc
us
on
ly o
n t
he
Tim
e o
f F
ligh
t:
W
e n
ee
d a
sta
rt d
ete
cto
r
E.g
. a ta
gger
for
the
inco
min
g ph
oton
s
W
e n
ee
d a
sto
p d
ete
cto
r
E.g
. a T
oFw
all
T
ime
of
Flig
ht
= s
top
_ti
me
–s
tart
_ti
me
Set
-up
prin
cipl
e
73
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
29iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple:
PID
usi
ng T
oF
It
is a
sim
ple
me
as
ure
men
t, b
ut
invo
lve
s a
lot
of
ele
ctr
on
ics
Set
-up
prin
cipl
e
H
ow
mu
ch
do
es
th
e t
ime
re
so
luti
on
of
the
sto
p d
ete
cto
r in
flu
en
ce
th
e m
ea
su
re?
le
t’s
sim
ula
te d
iffe
ren
t re
so
luti
on
s…
Det
ecto
r 1
Det
ecto
r 2
Am
p.A
mp.
Dis
cr.
Dis
cr.
TD
Cst
art
stop
Tim
e of
Fli
ght
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
30iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Par
ticle
mom
entu
m [G
eV/c
]0.
51
1.5
22.
53
TrackTime [ns]
-100
102030405060
Tra
ckT
ime
[ns]
-20
24
68
Number of Entries
0
50
100
150
200
250
300
350
Pro
ject
ionY
of b
inx=
[150
,159
]
Tra
ckT
ime
[ns]
-3-2
-10
12
34
56
Number of Entries
0
20406080
100
120
Pro
ject
ionY
of b
inx=
[290
,299
]
SP
= 1
4.3
SP
= 9
.7
An
exam
ple:
PID
usi
ng T
oF1)
Ide
al e
lect
roni
cs: n
o sp
read
of
tim
e du
e of
the
TD
C
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
31iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple:
PID
usi
ng T
oF
Tra
ckT
ime
[ns]
-3-2
-10
12
34
56
Number of Entries
0
102030405060708090
Pro
ject
ionY
of b
inx=
[290
,299
]
SP
= 8
.9S
P =
5.4
2) A
“go
od”
elec
tron
ics:
~48
0 ps
tim
e re
solu
tion
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
32iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Par
ticle
mom
entu
m [G
eV/c
]0
.51
1.5
22.
53
TrackTime [ns]
-100
102030405060
An
exam
ple:
PID
usi
ng T
oF
Tra
ckT
ime
[ns]
-20
24
68
Number of Entries
0
1020304050
Pro
ject
ionY
of b
inx=
[150
,159
]
Tra
ckT
ime
[ns]
-3-2
-10
12
34
56
Number of Entries
0
1020304050
Pro
ject
ionY
of b
inx=
[200
,209
]
SP
= 2
.9S
P =
???
3) A
“ba
d” e
lect
roni
cs: ~
1.9
ns ti
me
reso
luti
on
74
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
33iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
W
e w
ant
to i
de
nti
fy P
roto
ns
an
d
+ u
sin
g a
Tim
e o
f F
ligh
t
F
rom
th
e p
lot
“To
Fvs
Mo
me
ntu
m”
we
ca
n d
isti
ng
uis
h t
he
tw
o
pa
rtic
les
on
ly i
f w
e c
an
se
pa
rate
th
e t
wo
tim
e d
istr
ibu
tio
ns
T
he
tim
e r
es
olu
tio
n o
f th
e e
lec
tro
nic
s c
ha
in a
ffe
cts
wh
ich
kin
d o
f p
hys
ics
yo
u c
an
inve
sti
ga
te
Diff
eren
t tec
hnol
ogie
s al
low
for
diff
eren
t tim
e re
solu
tions
…
…bu
t bud
gets
are
lim
ited
Y
ou
mu
st
ch
oo
se
yo
ur
ele
ctr
on
ics
ta
kin
g in
to a
cc
ou
nt
you
r s
pe
cif
ic p
hys
ics
pro
gra
m!!
!
An
exam
ple:
PID
usi
ng T
oFC
oncl
usio
ns…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
34iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Rea
d-O
ut E
lect
roni
cs:
whe
re d
ata
com
e fr
om
lect
ure
1su
mm
ary
and
conc
lusi
ons
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 1
)
35iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Sum
mar
y an
d co
nclu
sion
s
T
o i
nve
sti
ga
te t
he
ma
tte
r, w
e b
uil
d d
ete
cto
rs
M
od
ern
de
tec
tors
are
pro
vid
ing
an
alo
g e
lec
tric
al s
ign
als
E
lec
tro
nic
s is
ne
ed
ed
to
“re
ad
th
e i
nfo
rma
tio
n”
fro
m t
he
d
ete
cto
rs
T
he
ele
ctr
on
ics
ch
ain
ha
s t
o:
P
repr
oces
s th
e si
gnal
s w
/o d
isto
rtio
n
Be
relia
ble
B
e ef
ficie
nt
Be
fast
B
e ca
libra
ted
to th
e ex
perim
ent!!
!
Lec
ture
1
75
76
LECTURE 6 From Quark to Jet: A Beautiful Journey Making a jet, classifying a jet, and personal scale computing in HEP
Tuesday 25 February
11:30-12:30
Description This lecture will begin by presenting the detecting systems needed for proper jet reconstruction. Then the jet reconstruction algorithms in detail are explained. A brief explanation of combination algorithms follows, completed by an examination of the datastructures used to store them. This will be followed by a brief description of a multivariate technique commonly used for the classification of these objects and the software package (TMVA) used to implement them. Particular attention is paid to which parts of these algorithms can be completed on university or personal level computing that is the common interface for analysts.
Tyler Mc Millan
Dorland
DESY - Hamburg
Audience The target audience of this lecture is broad between physicists and computer scientists, but perhaps more focused on physicists with some knowledge of different computing environments. The benefits of following this lecture is the understanding of jet reconstruction algorithms, understanding of personal/group level computing in HEP, understanding of a single multivariate analysis technique (most likely decision trees), understanding of data structures used in HEP (ROOT based), and new approaches in common software at an experiment wide level. Pre-requisite As prerequisite some knowledge of a particle will be helpful but not required. This lecture will be reasonably independent of the first lecture; however the physics motivation for b-quarks will not be revisited extensively.
77
78
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
1iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Fro
m Q
uark
to
Jet:
A B
eaut
iful J
ourn
eyLe
ctur
e2
Jet
Clu
ster
ing
, Cla
ssif
icat
ion
, an
d
Per
son
al C
om
pu
tin
g
Tyl
er D
orl
and
Deu
tsch
esE
lekt
ron
en-S
ynch
rotr
on
(D
ES
Y)
Inve
rted
CE
RN
Sch
oo
l of
Co
mp
uti
ng
, 24-
25 F
ebru
ary
2014
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
2iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
3iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Hug
e nu
mbe
rs o
f co
mpl
ex e
quat
ions
Ent
irel
y S
imul
ated
, pa
rtic
les
are
subj
ecte
d to
de
cay
cond
ition
s
Det
ecto
r si
mul
atio
n,A
lgor
ithm
ic r
econ
stru
ctio
n
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
4iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Hug
e nu
mbe
rs o
f co
mpl
ex e
quat
ions
Ent
irel
y S
imul
ated
, pa
rtic
les
are
subj
ecte
d to
de
cay
cond
ition
s
Det
ecto
r si
mul
atio
n,A
lgor
ithm
ic r
econ
stru
ctio
n
Dis
trib
uted
Com
putin
g, L
apto
ns, w
orkg
roup
ser
vers
79
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
5iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Exp
lain
ing
the
Titl
e: A
n ou
tline
T
heo
ry
Had
ron
izat
ion
R
eco
nst
ruct
ion
Qua
rks:
Mat
hem
atic
alR
epre
sent
atio
nM
atric
es,
oper
ator
s, e
tc.
Par
ticle
s:in
term
edia
te a
nd fi
nal
stat
e ob
ject
s
Jets
:E
nerg
y de
posi
ted
in d
etec
tor,
algo
rithm
s us
ed t
o re
crea
te
part
icle
s
Hug
e nu
mbe
rs o
f co
mpl
ex e
quat
ions
Ent
irel
y S
imul
ated
, pa
rtic
les
are
subj
ecte
d to
de
cay
cond
ition
s
Det
ecto
r si
mul
atio
n,A
lgor
ithm
ic r
econ
stru
ctio
n
Dis
trib
uted
Com
putin
g, L
apto
ns, w
orkg
roup
ser
vers
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
6iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Ask
Que
stio
ns h
ere
U
se t
heo
ry t
o m
ake
pre
dic
tio
ns
for
ob
serv
able
s o
f p
arti
cles
D
esig
n d
etec
tors
to
det
ect
thes
e o
bse
rvab
les
R
eco
nst
ruct
ion
alg
ori
thm
s to
re
mak
e th
e o
bje
cts
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
7iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Cal
orim
eter
s
C
alo
rim
eter
s ar
e d
esig
ned
to
cap
ture
th
e en
erg
y o
f p
arti
cles
T
wo
dif
fere
nt
typ
es a
re in
u
se a
t L
HC
H
omog
eneo
us –
Cap
ture
all
of th
e en
ergy
of t
he in
cide
nt
part
icle
S
ampl
ing
–ca
ptur
e a
port
ion
the
inci
dent
en
ergy
and
mak
e a
corr
ectio
n
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
8iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Sam
plin
g C
alor
imet
er
S
amp
ling
cal
ori
met
ers
hav
e a
sen
siti
ve l
ayer
sa
nd
wic
hed
bet
wee
n
to h
eavy
ab
sorb
er
laye
rs
Abs
orbe
r la
yers
us
eful
to c
reat
e sh
ower
s of
sec
onda
ypa
rtic
les
U
sefu
l fo
r h
adro
ns
bec
ause
a
ho
mo
gen
eou
s d
etec
tor
wo
uld
be
too
larg
e
Wor
se r
esol
utio
n,
thou
gh
80
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
9iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Jet c
lust
erin
g
A
few
th
eore
tica
l co
nsi
der
atio
ns
In
frar
ed S
afe
S
houl
d no
t be
sens
itive
to s
oft
radi
atio
n
C
ollin
ear S
afe
S
houl
d no
t be
sens
itive
to
colli
near
rad
iatio
n
Bec
ause
we
are
mea
surin
g de
cay
prod
ucts
we
mus
t fin
d a
way
to
clus
ter
them
toge
ther
to a
ccur
atel
y re
pres
ent t
he o
rigin
al p
artic
le
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
10iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Con
e A
lgor
ithm
s
“S
eed
” d
efin
es a
pp
roxi
mat
e je
t d
irec
tio
n
A
ll en
erg
y d
epo
sits
wit
hin
a
giv
en r
adiu
s ar
e p
ut
into
th
e je
t
T
he
cen
tro
id is
det
erm
ined
su
mm
ing
all
par
ticl
es w
ith
in
the
con
e
T
he
cen
tro
id b
eco
mes
th
e n
ew s
eed
Ite
rate
d un
til s
tabl
e
trac
ks o
r to
wer
s
seed
Rco
ne
Rco
ne
cent
roid
= n
ew s
eed
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
11iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Mid
-poi
nt c
one
algo
rithm
S
earc
h f
or
mis
sin
g je
ts u
sin
g t
he
mid
po
int
of
all j
et p
airs
as
a se
ed
If
th
ere
is a
sta
ble
co
ne
con
sid
er t
he
ener
gy
dep
osi
ts s
har
ed
bet
wee
n t
he
two
jets
(E
Sh
ared
)
T
ake
f =
ES
har
ed/E
jet2
If
f> 5
0% m
erge
the
jets
; els
e sp
lit th
e je
ts
mid
poin
t
JET
#1
p T
jet1
>p T
jet2
JET
#2
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
12iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
KT
algo
rithm
B
egin
wit
h a
list
of
hit
s an
d c
alo
rim
eter
to
wer
s
C
alcu
late
F
or e
ach
prec
lust
eri:
F
or e
ach
pair
(i,j):
F
ind
th
e m
inim
um
, d
min
, of
all d
ian
d d
i,j
If
dm
inis
a d
i,j,
rem
ove
pre
clu
ster
sia
nd
j fr
om
th
e lis
t an
d
rep
lace
wit
h a
new
mer
ged
pre
clu
ster
If
dm
inis
a d
i, p
recl
ust
erI i
s n
ot
“mer
gea
ble
” an
d c
an b
e ad
ded
to
th
e lis
t o
f je
ts
R
epea
t u
nti
l lis
t is
exh
aust
ed
81
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
13iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Som
e di
ffere
nt E
xam
ples
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
14iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Com
bini
ng M
easu
rem
ents
It
can
be
adva
nta
geo
us
to u
se d
iffe
ren
t p
arts
of
the
det
ecto
rs
for
dif
fere
nt
mea
sure
men
ts f
or
the
con
stit
uen
ts o
f th
e je
ts
F
or
exam
ple
, we
can
so
met
ime
rep
lace
a c
alo
rim
eter
m
easu
rem
ent
wit
h a
tra
cker
mea
sure
men
t as
soci
ated
to
it
For
low
mom
enta
, th
e tr
acke
r m
easu
rem
ents
are
mor
e pr
ecis
e
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
15iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Par
ticle
ID -
RIC
H
C
her
enko
v ra
dit
atio
nis
em
itte
d w
hen
a p
arti
cle
pas
ses
thro
ug
h a
m
ediu
m a
nd
is in
itia
lly
go
ing
fas
ter
than
th
e sp
eed
of
ligh
t in
th
at
med
ium
A
rin
g o
f lig
ht
is
emit
ted
th
at is
p
rop
ort
ion
al t
o t
he
mo
men
tum
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
16iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Par
ticle
ID -
RIC
H
B
y ch
oo
sin
g t
he
corr
ect
med
ia, w
e ca
n u
se t
his
as
a fo
r o
f p
arti
cle
iden
tifi
cati
on
Li
ght i
n C
1an
d C
2 =
Pio
n
Ligh
t in
just
C1
= K
aon
N
o lig
ht =
pro
ton
82
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
17iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Mul
tivar
iate
Tec
hniq
ues
F
inal
ly w
e ca
n c
om
bin
e al
l th
ese
mea
sure
men
ts in
to
ve
ry p
ow
er m
ult
ivar
iate
an
alys
is (
MV
A)
tech
niq
ues
T
hese
can
giv
e a
mea
sure
of
how
like
ly a
jet i
s to
be
a b-
jet
O
ne
tech
niq
ue
is a
dec
isio
n
tree
th
at m
akes
a s
erie
s o
f cu
ts o
n d
iffe
ren
t in
pu
t va
riab
les
T
hen
recl
assi
fied
by th
e G
inii
ndex
p(1
-p)
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
18iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Dec
isio
n tr
ees
-w
eigh
ting
T
he
tree
is t
rain
ed a
gai
nst
a k
no
wn
tru
th (
fro
m M
C)
M
iscl
assi
fied
eve
nts
are
giv
en a
larg
er w
eig
ht
then
ret
rain
ed
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
19iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Sam
ple
Res
ult
In
th
e en
d w
e g
et a
bet
ter
dis
crim
inat
or
than
an
y si
mp
le c
ut
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
20iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Inpu
ts
83
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful J
ou
rney
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
21iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Out
put
U
sin
g t
he
inp
uts
fro
m
mu
ltip
le p
arts
of
the
det
ecto
r w
e ca
n m
ake
a b
ette
r ju
dg
emen
to
n if
th
e je
t w
e ar
e m
easu
rin
g c
ame
fro
m a
b
-qu
ark
or
ano
ther
so
urc
e
Fro
m Q
uar
k to
Jet
: A
Bea
uti
ful
Jou
rney
Lec
ture
2
22iC
SC
2014
, T
yler
Do
rlan
d, D
ES
Y
Con
clus
ions
Je
t al
go
rith
ms
are
des
ign
ed t
o m
ake
up
fo
r th
e in
effi
cien
cies
cr
eate
d b
y fi
nit
e re
solu
tio
n o
f o
ur
calo
rim
eter
s
A
spec
ts f
rom
man
y d
iffe
ren
t p
ort
ion
s o
f th
e d
etec
tor
can
be
com
bin
ed u
sin
g s
tati
stic
al t
oo
ls s
uch
as
dec
isio
n t
rees
to
d
eter
min
e h
ow
like
ly i
t is
th
at a
par
ticu
lar
jet
cam
e fr
om
a b
-q
uar
k
T
hes
e al
go
rith
ms
are
no
t as
co
mp
uti
ng
inte
nsi
ve a
nd
wel
l su
ited
fo
r p
erso
nal
co
mp
uti
ng
D
ata
stru
ctu
res
mu
st b
e m
anip
ula
ted
in o
rder
fo
r th
ese
to b
e ru
n la
pto
ps
84
LECTURE 7 Read-Out Electronics: where data come from
Tuesday 25 February
13:30-14:30
Description This lecture will focus on electronics used in HEP: while the software tools available to program very performant firmware are more and more powerful, hardware limitations cannot be (easily) overcome. High speed digital signals, today required in HEP setups, need to be handled as analog signals: signal propagation, interaction between signals and electromagnetic interferences must be carefully considered. Moreover, the power consuming of ASICs and FPGAs strongly depends of the operations required and may be a limitation of the board.
Francesco Messi
Rheinische Friedrich-Wilhelms University
Bonn - DE
Audience This lecture targets everyone interested in how the signals are transported from a detector to the data storage, which could be the hardware limitation of the available electronics and which characteristics must be taken into account in the choice of a good electronics for the experiment. After this lecture the attendees are expected to have acquired a good understanding of the hardware limitation of electronics and the characteristics needed to preserve the integrity of the signal to be stored for the analysis of an experiment.
Pre-requisite Though all lectures are in principle independent, there might be some benefit for listeners to have attended the first lecture of the series.
85
86
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
1iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Rea
d-O
ut E
lect
roni
cs:
whe
re d
ata
com
e fr
om
Lect
ure
2
Fra
nce
sco
Mes
si
Rh
ein
isch
eF
ried
rich
-Wilh
elm
s-U
niv
ersi
tät
Bo
nn
–D
E
Inve
rted
CE
RN
Sch
oo
l o
f C
om
pu
tin
g,
24-2
5 F
ebru
ary
2014
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
2iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Out
look
P
rolo
gu
e
T
he
mag
ic k
ey:
Ele
ctro
nic
s
T
he
FP
GA
(F
ield
Pro
gra
mm
able
Gat
e A
rray
)
A
n e
xam
ple
: fe
atu
res
extr
acti
on
fro
m a
Sam
plin
g-A
DC
F
rom
th
e ch
ip t
o t
he
bo
ard
H
igh
Sp
eed
Dig
ital
Sig
nal
s
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
3iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Pro
logu
e
Mod
ern
set-
ups
are
mad
e of
sev
eral
diff
eren
t de
tect
ors;
eac
h de
tect
or is
gen
erat
ing
elec
tric
al
sign
als
that
hav
e to
be
pre-
proc
esse
d to
be
stor
ed.
Ele
ctro
nics
is a
cru
cial
key
in th
is p
roce
ss…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
4iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Pro
logu
e…M
oder
n ex
peri
men
tal s
et-u
ps
Mo
der
n e
xper
imen
tal s
et-u
ps
are
mad
e o
f a
larg
e n
um
ber
of
dif
fere
nt
det
ecto
rs, e
ach
loo
kin
g a
t a
par
ticu
lar
asp
ect
M
od
ern
det
ecto
rs a
re p
rod
uci
ng
an
alo
g e
lect
rica
l sig
nal
s
D
etec
tors
are
“ta
lkin
g”
thro
ug
h t
he
elec
tro
nic
s
87
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
5iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Pro
logu
e…
W
e w
ant:
to
rec
eive
the
corr
ect i
nfor
mat
ion
relia
bilit
y
to r
ecei
ve th
e fu
ll in
form
atio
n tr
ansm
itted
ef
ficie
ncy
to
und
erst
and
the
mes
sage
no
dis
tort
ion
R
ead
-Ou
t E
lect
ron
ics:
Fro
m a
littl
e, f
ast
anal
og s
igna
l to
a bi
t-st
ream
of d
ata
Ele
ctro
nics
as
a “t
rans
mis
sion
line
” fr
om th
e de
tect
or to
the
data
sto
rage
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
6iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Pro
logu
e…F
rom
the
anal
og to
the
digi
tal w
orld
Fam
ily
On
e (V
)Z
ero
(V
)
TT
L2
to 5
0 to
0.8
NIM
3 to
12
-2 t
o 1.
5
EC
L-0
.81
to -
1.13
-1.4
8 to
-1.
95
LVD
Sp
1.27
to 2
.40
0.92
to
1.12
n0.
92 t
o 1.
121.
27 t
o 2.
4
…an
d m
any
mor
e…
D
igit
al s
ign
al
two
sta
tes:
pre
sen
t o
r ab
sen
t, “
1” o
r “0
”
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
7iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Pro
logu
e…B
inar
y lo
gic:
fun
ctio
ns
AN
DO
RN
OT
XO
R
NA
ND
NO
RN
XO
R
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
8iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Pro
logu
e…B
inar
y lo
gic:
Flip
Flo
p
CK
SR
Qn
Qn
+1
0X
X0
0
0X
X1
1
10
00
0
10
01
1
10
10
0
10
11
0
11
00
1
11
01
1
11
1?
?
88
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
9iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
mag
ic k
ey: e
lect
roni
cs
wha
t do
we
gain
by
the
usag
e of
ele
ctro
nics
and
wha
t do
we
have
to w
orry
abo
ut?
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
10iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
mag
ic k
ey: e
lect
roni
cs
1)
It is
mu
ch f
aste
r th
an h
um
an in
tak
ing
dec
isio
ns
E
.g.
evol
utio
n of
pho
to c
amer
as
auto
mat
ic s
ettin
g of
foc
us,
timin
g, o
peni
ng…
Wha
t do
we
gain
fro
m th
e us
age
of e
lect
roni
cs?
toda
yye
ster
day
Fas
t res
pons
eF
ast s
et-u
pM
uch
easi
er to
dup
lica
te d
ata
Sto
rage
/man
agem
ent o
f lo
ts o
f da
ta m
ore
sim
ple…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
11iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
mag
ic k
ey: e
lect
roni
cs
2)
Allo
ws
to t
rig
ger
C
hoic
e of
rec
ord
whe
n a
spec
ific
scen
ario
hap
pens
W
hy?
We
can
not r
ecor
d al
l the
info
rmat
ion
W
e ne
ed to
cho
ose
whi
ch o
ne is
inte
rest
ing…
E
.g.:
bubb
le c
ham
ber
expe
rimen
ts: c
anno
t trig
ger
=
O
ne p
ictu
re e
ach
spill
m
illio
ns o
f pi
ctur
es o
f th
e ch
ambe
r
m
ayb
e fe
w h
undr
eds
with
an
even
t…
Wha
t do
we
gain
fro
m th
e us
age
of e
lect
roni
cs?
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
12iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
mag
ic k
ey: e
lect
roni
cs
T
hin
k ab
ou
t th
e L
HC
exp
erim
ents
:
even
with
sel
ectin
g th
e ev
ent t
o b
e st
ored
(=
trig
gerin
g), t
he
data
-str
eam
to th
e st
orag
e sy
stem
is o
f the
ord
er 1
00 M
b/s
D
iffe
ren
t fr
om
mo
nit
ori
ng
th
e te
mp
erat
ure
in a
ro
om
:
not s
ampl
ing
at a
fixe
d fr
eque
ncy,
but a
naly
zing
the
eve
nt in
rea
l tim
e an
d ta
king
a d
ecis
ion…
Tri
gger
ing…
89
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
13iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
mag
ic k
ey: e
lect
roni
cs
E
.g.
a m
ovi
e an
d a
pic
ture
:
The
vid
eo c
amer
a is
rec
ordi
ng e
very
thin
g:
yo
u w
ill n
ot lo
se e
vent
s, b
ut…
bi
g am
ount
of
data
to
be s
tore
d
on
ce y
ou lo
ok o
ut f
or t
he e
vent
, yo
u ne
ed t
o w
atch
all
the
mov
ie
T
he p
hoto
cam
era
is r
egis
terin
g on
ly s
cene
s th
at a
re “
trig
gere
d”
by t
he o
pera
tor:
less
dat
a to
be
stor
ed
m
uch
sim
pler
to
look
at
them
, bu
t…
yo
u lo
se a
ll w
hat
happ
ene
d in
bet
we
en o
ne t
rigge
r an
d an
othe
r
Tri
gger
ing…
B
e ca
refu
l: ac
quir
ed r
aw d
ata
are
a se
lect
ed su
bsec
tion
of w
hat t
he e
xper
imen
tal s
et-u
p co
uld
dete
ct!!
!
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
14iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
mag
ic k
ey: e
lect
roni
cs
W
ith
th
e u
sag
e o
f el
ectr
on
ics
we
gai
n t
wo
maj
or
asp
ects
:
1)
vel
ocity
in ta
king
dec
isio
ns
2)
trig
ger
capa
bilit
y
b
ut
be
care
ful:
acq
uir
ed d
ata
are
a se
lect
ed s
ub
sect
ion
!!!
A b
rief
sum
mar
y:
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
15iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
The
FP
GA
FP
GA
s ar
e w
idel
y us
ed in
mod
ern
set-
ups.
A g
ener
al o
verv
iew
of w
hat
an F
PG
A is
and
how
it
wor
ks w
ill b
e pr
esen
ted,
focu
sing
on
the
usag
e in
ph
ysic
s re
sear
ch…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
16iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Sig
nal p
roce
ssin
g: th
e F
PG
A *pic
ture
fro
m: h
ttp:
//w
ww
.ni.c
om/w
hite
-pap
er/6
983/
en/
Fie
ld P
rogr
amm
able
Gat
e A
rray
90
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
17iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Sig
nal p
roce
ssin
g: th
e F
PG
A
Sp
arta
n-6
Vir
tex-
7V
irte
x U
ltra
Sca
le
Logi
c C
ells
147,
443
1,95
4,56
04,
407,
480
Blo
ckR
AM
4.8M
b68
Mb
115M
b
DS
P S
lices
180
3,60
02,
880
DS
P P
erfo
rman
ce
(sym
met
ric F
IR)
140
GM
AC
s5,
335
GM
AC
s4,
268
GM
AC
s
Tra
nsce
iver
Cou
nt8
9610
4
Tra
nsce
iver
Spe
ed3.
2 G
b/s
28.0
5 G
b/s
32.7
5 G
b/s
Tota
l Tra
nsce
iver
B
andw
idth
(fu
ll du
plex
)50
Gb/
s2,
784
Gb/
s5,
101
Gb/
s
Mem
ory
Inte
rfac
e (D
DR
3)80
01,
866
2,40
0
I/O P
ins
576
1,20
01,
456
I/O
Vol
tage
1.2V
-3.
3V1.
2V -
3.3V
1.0
–3.
3V
*tab
le f
rom
: htt
p://
ww
w.x
ilin
x.co
m/p
rodu
cts/
sili
con-
devi
ces/
fpga
/
Som
e nu
mbe
rs…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
18iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Sig
nal p
roce
ssin
g: th
e F
PG
A
P
rog
ram
mab
le =
re-
usa
ge
of
the
sam
e h
ard
war
e fo
r m
ult
iple
ta
sks
P
aral
lelis
m =
in p
rin
cip
le a
ny
n-t
o-o
ne
log
ic
Mas
sive
per
form
ance
s on
sam
e al
gorit
hms
E
xact
det
erm
inat
ion
of
the
exec
uti
on
ord
er o
f th
e d
iffe
ren
t ta
sks
H
ardw
are
chai
n of
bin
ary
func
tions
Why
sho
uld
an F
PG
A b
e us
ed?
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
19iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple…
Eac
h de
tect
or u
sed
in e
xper
imen
tal s
et-u
ps c
onsi
sts
of
num
erou
s ch
anne
ls, d
etec
ting
sign
als
with
rat
e of
the
orde
r of
MH
z or
eve
n G
Hz;
a la
rge
amou
nt o
f dat
a ar
e pr
oduc
ed. T
o se
lect
whi
ch d
ata
are
inte
rest
ing
to b
e re
cord
ed, i
t is
impo
rtan
t to
proc
ess
the
data
as
soon
as
pos
sibl
e ne
ar th
e de
tect
or…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
20iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple
B
asel
ine
S
lop
e
S
tart
tim
e
C
har
ge
inte
gra
l
E
tc..
N
eeds
to b
e do
ne a
s fa
st a
s po
ssib
le
With
a fi
xed
dela
y tim
e
In p
aral
lel f
or m
any
chan
nels
Fea
ture
ext
ract
ion
from
a S
ampl
ing-
AD
C
91
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
21iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple
Am
plan
alog
sp
litt
er
AD
C
Dis
crT
DC
dete
ctor
# ch
# ch
# ch
# ch# ch
# ch
# ch
DA
Q &
an
alys
is
Fea
ture
ext
ract
ion
from
a S
ampl
ing-
AD
C
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
22iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple
Am
pl
Sam
plin
g A
DC
dete
ctor
# ch
# ch
Fea
ture
s ex
trac
tor
FP
GA
boa
rd
trig
DA
Q
Fea
ture
ext
ract
ion
from
a S
ampl
ing-
AD
C
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
23iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
An
exam
ple
Fea
ture
ext
ract
ion
from
a S
ampl
ing-
AD
C
L
ess
cab
ling
L
ess
elec
tro
nic
s m
od
ule
s
T
rig
ger
cap
abili
ty
B
ut…
Sam
plin
g of
the
ana
log
sign
al
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
24iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e ch
ip to
the
boar
d
Eac
h el
ectr
onic
chi
p is
gen
eral
ly m
ount
ed o
n bo
ards
, to
pro
vide
the
nece
ssar
y po
wer
and
buf
ferin
g of
the
sign
als;
eve
n if
the
perf
orm
ance
of t
he c
hip
itsel
f (F
PG
A o
r A
SIC
) is
ver
y hi
gh, o
ne o
f the
oth
er
com
pone
nts
can
limit
the
char
acte
ristic
of t
he b
oard
…
92
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
25iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e ch
ip to
the
boar
d
F
PG
A:
very
po
wer
ful
…o
k b
ut
wh
at a
bo
ut
the
elec
tro
nic
s ar
ou
nd
it?
??
N
ot p
ossi
ble
to c
onne
ct th
e si
gnal
s fr
om th
e de
tect
or d
irect
ly t
o th
e F
PG
A!
Y
our
tran
smis
sion
line
cou
ld “
spea
k a
diffe
rent
lang
uage
”
P
ower
and
filt
erin
g ar
e ne
ede
d
S
pace
con
stra
ins
B
UF
FE
RIN
G:
–ne
ver
trus
t the
tran
smis
sion
line
s: u
nkno
wn
sign
als
can
arriv
e an
d w
e do
n’t w
ant
to d
amag
e ou
r F
PG
A c
hips
!!!
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
26iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e ch
ip to
the
boar
d
N
ot
alw
ays
the
mo
st p
erfo
rmin
g b
oar
d is
th
e co
rrec
t ch
oic
e:
m
ore
perf
orm
ance
= m
ore
pow
er
m
ore
pow
er =
mor
e co
olin
g
mor
e co
olin
g =
mor
e m
ater
ial b
udge
t in
the
dete
ctor
…
Is it
fin
e w
ith “
my
phys
ics”
???
It
is i
mp
ort
ant
to lo
ok
at t
he
full
syst
em,
mo
re t
han
at
the
sin
gle
mo
du
le…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
27iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Fro
m th
e ch
ip to
the
boar
d
S
ign
al p
roce
ssin
g
FP
GA
s us
ed fo
r fa
st,
para
llel,
on-li
ne o
pera
tions
N
ot p
ossi
ble
to c
onne
ct a
nalo
g si
gnal
s di
rect
ly t
o th
e F
PG
A!
B
andw
idth
(in
/out
-put
buf
fers
, etc
…)
S
yste
m o
pti
miz
atio
n m
ore
th
an c
om
po
nen
t o
pti
miz
atio
n
Logi
stic
con
stra
ins
(spa
ce, c
oolin
g, p
ower
, et
c…)
A b
rief
sum
mar
y…
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
28iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Hig
h-S
peed
Dig
ital S
igna
l
Hig
h-S
peed
dig
ital s
igna
ls n
eed
to b
e ha
ndle
d as
an
alog
sig
nals
…
93
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
Le
ctur
e2
iCS
C20
14
24-2
5 F
ebru
ary
2014
, C
ER
N
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
29iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
-3-2
-10
12
3
-0.8
-0.6
-0.4
-0.20
0.2
0.4
0.6
0.8
-3-2
-10
12
3
-1
-0.50
0.51
-3-2
-10
12
3
-1
-0.50
0.51
Hig
h S
peed
dig
ital s
igna
ls
A
dig
ital
sig
nal
is i
n b
inar
y lo
gic
I
have
to ta
ke c
are
only
of
two
stat
es: “
1” a
nd “
0”
It is
less
affe
cted
by
nois
e th
an a
n an
alog
sig
nal
T
han
it is
muc
h si
mpl
er to
han
dle…
tru
e in
pri
nci
ple
, b
ut…
need
to b
e ha
ndle
d as
ana
logu
e on
es…
Far
bel
ow th
e ba
ndw
idth
lim
itC
lose
to th
e ba
ndw
idth
lim
itF
ar o
utsi
de th
e ba
ndw
idth
lim
it
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
30iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Rea
d-O
ut E
lect
roni
cs:
whe
re d
ata
com
e fr
om
lect
ure
2su
mm
ary
and
conc
lusi
ons
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
31iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Sum
mar
y an
d co
nclu
sion
s
T
o in
vest
igat
e th
e m
atte
r, w
e b
uild
det
ecto
rs
M
od
ern
det
ecto
rs a
re p
rovi
din
g a
nal
og
ele
ctri
cal s
ign
als
E
lect
ron
ics
is n
eed
ed t
o “
read
th
e in
form
atio
n”
fro
m t
he
det
ecto
rs
T
he
elec
tro
nic
s ch
ain
has
to
:
Pre
proc
ess
the
sign
als
w/o
dis
tort
ion
B
e re
liabl
e
Be
effic
ient
B
e fa
st
Be
calib
rate
d to
the
exp
erim
ent!!
!
Lec
ture
1
Rea
d-O
ut
Ele
ctro
nic
s: w
her
e d
ata
com
e fr
om
(le
ctu
re 2
)
32iC
SC
2014
, F
ran
cesc
o M
essi
, U
niv
ersi
ty o
f B
on
n
Sum
mar
y an
d co
nclu
sion
s
M
od
ern
ele
ctro
nic
s is
ver
y p
ow
erfu
l
Hig
h ba
ndw
idth
H
igh
prog
ram
mab
ility
B
ut…
O
ne
elem
ent
is e
no
ug
h t
o d
ecre
ase
the
per
form
ance
of
the
full
chai
n
Sys
tem
opt
imiz
atio
n m
ore
than
com
pone
nt o
ptim
izat
ion
H
igh
sp
eed
dig
ital
sig
nal
s n
eed
to
be
han
dle
d a
s an
alo
g o
nes
94
LECTURE 8 Introduction to machine learning and data mining
Tuesday 25 February
14:30-15:30
Description In this lecture, the basis of machine learning and data mining will be explained. Then, the most typical problems where machine learning is applied will be presented: classification, clustering, regression and anomaly detection. For those problems, some techniques that can be applied will be presented and briefly explained: decision trees, support machine vectors, k-NN, k-means and neural networks.
Juan Lopez Gonzalez
CERN
Audience After this lecture, the attendees are expected to understand the differences between machine learning and data mining, comprehend the general concepts of machine learning as well as some of the typical problems and its solution approaches.
Pre-requisite This lecture can be followed by anyone having experience with algorithms and computer science.
95
96
1
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
1iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Juan
Ló
pez
Go
nzá
lez
Un
iver
sit
y o
f O
vied
o
Inve
rted
CE
RN
Sch
oo
l of
Co
mp
uti
ng
, 24-
25 F
ebru
ary
2014
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
2iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
Gen
eral
ove
rvie
w
L
ectu
re1
M
achi
ne le
arni
ng
Intr
oduc
tion
D
efin
ition
P
robl
ems
T
echn
ique
s
L
ectu
re2
A
NN
S
OM
s
Def
initi
on
A
lgor
ithm
S
imul
atio
n
SO
M b
ased
mod
els
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
3iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
LEC
TU
RE
1
Intr
oduc
tion
to m
achi
ne le
arni
ng a
nd d
ata
min
ing
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
4iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
1.1.
Som
ede
finiti
ons
1.2.
Mac
hine
lear
nin
gvs
Dat
a m
inin
g1.
3. E
xam
ples
1.4.
Ess
ence
of m
achi
ne le
arni
ng1.
5. A
lear
nin
gpu
zzle
1. In
trod
uctio
n
97
2
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
5iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
1.1
Som
ede
finiti
ons
T
o le
arn
T
o us
e a
set o
f obs
erva
tions
to
unco
ver
an
unde
rlyin
g pr
oces
s
T
o m
emo
rize
T
o co
mm
it to
mem
ory
It
does
n’t
mea
n to
und
erst
and
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
6iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
1.2
Mac
hine
lear
ning
vs D
ata
min
ing
M
ach
ine
lear
nin
g(A
rthu
r S
amue
l)
S
tudy
, des
ign
and
deve
lopm
ent
of a
lgor
ithm
sth
atgi
veco
mpu
ters
capa
bilit
yto
lear
nw
ithou
tbei
ngex
plic
itly
prog
ram
med
.
D
ata
min
ing
E
xtra
ctkn
owle
dge
orun
know
npa
ttern
sfr
omda
ta.
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
7iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
1.3
Exa
mpl
es
C
red
itap
pro
val
G
ende
r, a
ge, s
alar
y, y
ears
in jo
b, c
urre
ntde
bt…
S
pam
filt
erin
g
Sub
ject
, Fro
m…
T
op
icsp
ott
ing
C
ateg
oriz
ear
ticle
s
W
eath
erp
red
icti
on
W
ind,
hum
idity
, te
mpe
ratu
re…
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
8iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
1.4
Ess
ence
of m
achi
ne le
arni
ng
A
pat
tern
exis
ts
W
eca
nnot
pin
itdo
wn
mat
hem
atic
ally
W
eha
veda
ta o
nit
98
3
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
9iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
1.5
A le
arni
ngpu
zzle
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
10iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
2.1.
Com
pon
ents
2.2.
Gen
eral
izat
ion
and
repr
ese
ntat
ion
2.3.
Typ
esof
lear
ning
2. D
efin
ition
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
11iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
2.1
Com
pone
nts
In
put
(cus
tom
erap
plic
atio
n)
O
uput
(apr
ove/
reje
ctcr
edit)
Id
eal f
unct
ion
(f: X
↦Y
)
D
ata:
(a1,
b 1,..
,n1)
, (a 2
,b2,
..,n 2
) …
(a N
,bN,..
,nN)
(hi
stor
ical
reco
rds)
R
esul
t:(y
1),
(y2)
… (
y N)
(loa
n pa
idor
notp
aid)
H
ypot
hesi
s(g
: X ↦
Y)
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
12iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
2.1
Com
pone
nts
Unk
now
nta
rget
func
tion
f: X
↦Y
Tra
inin
g ex
ampl
es(a
1,b 1
,..,n
1)…
(aN,b
N,..
,nN)
Lea
rnin
gal
go
rith
m
Fin
al h
ypo
the
sis
g ≈
fH
ypo
thes
isse
t
H
99
4
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
13iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
2.2
Gen
eral
izat
ion
and
repr
esen
tatio
n
G
ener
aliz
atio
n
The
algo
rithm
has
to b
uild
a ge
nera
l mod
el
O
bje
ctiv
e
Gen
eral
ize
from
exp
erie
nce
A
bilit
yto
per
form
accu
rate
lyfo
run
seen
exam
ples
R
epre
sen
tati
on
R
esul
tsde
pend
onin
put
In
put d
epen
dson
repr
esen
tatio
n
–P
re-p
roce
ssin
g?
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
14iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
2.3
Typ
esof
lear
ning
S
uper
vise
d
Inpu
t and
out
put
U
nsup
ervi
sed
O
nly
inpu
t
R
einf
orce
me
nt
Inpu
t, ou
tput
and
gra
de o
f out
put
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
15iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
3.1.
Reg
ress
ion
3.2.
Cla
ssifi
catio
n3.
3. C
lust
erin
g3.
4. A
ssoc
iatio
nru
les
3. P
robl
ems
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
16iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
3.1
Reg
ress
ion
S
tati
stic
al p
roce
ss f
or
esti
mat
ing
th
e re
lati
on
ship
s am
on
g v
aria
ble
s
Cou
ldbe
use
dfo
rpr
edic
tion
100
5
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
17iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
3.2
Cla
ssifi
catio
n
Id
enti
fy t
o w
hic
h o
f a
set
of
cate
go
ries
a n
ewo
bse
rvat
ion
b
elo
ng
s
Sup
ervi
sed
lear
ning
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
18iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
3.3
Clu
ster
ing
G
rou
pin
g a
set
of
ob
ject
s in
su
ch a
way
th
at o
bje
cts
in
the
sam
e g
rou
p a
re m
ore
sim
ilar
U
nsup
ervi
sed
lear
ning
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
19iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
3.4
Ass
ocia
tion
rule
D
isco
veri
ng
rel
atio
ns
bet
wee
n v
aria
ble
s in
la
rge
dat
abas
es
Bas
edon
‘str
ong
rule
s’
Ifor
der
mat
ters
-> S
eque
ntia
lpat
tern
min
ing
AB
C
10
01
21
01
31
10
41
00
50
10
freq
uent
item
set
latti
ce
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
20iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.1.
Dec
isio
ntr
ees
4.2.
SV
M4.
3. M
onte
Car
lo4.
4. K
-NN
4.5.
AN
N
4. T
echn
ique
s
101
6
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
21iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.1
Dec
isio
ntr
ees
U
ses
tree
-lik
e g
rap
h o
f d
ecis
ion
s an
d p
oss
ible
co
nse
qu
ence
s
Inte
rnal
node
: attr
ibut
e
Leaf
: res
ult
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
22iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.1
Dec
isio
ntr
ees
R
esul
tshu
man
rea
dabl
e
E
asily
com
bine
dw
ithot
her
tech
niqu
es
Pos
sibl
esc
enar
ios
can
be a
dded
E
xpen
sive
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
23iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.2
Sup
port
Vec
tor
Mac
hine
(S
VM
)
S
epar
ates
th
e g
rap
hic
al r
epre
sen
tati
on
of
the
inp
ut
po
ints
C
onst
ruct
sa
hype
rpla
new
hich
can
be u
sed
for
clas
sific
atio
n
Inpu
t spa
cetr
ansf
orm
atio
nhe
lps
N
on-h
uman
rea
dabl
ere
sults
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
24iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.3
Mon
te C
arlo
O
bta
in t
he
dis
trib
uti
on
of
an u
nkn
ow
n p
rob
abili
stic
en
tity
R
ando
m s
ampl
ing
to o
btai
n nu
mer
ical
res
ults
A
pp
licat
ion
s
Phy
sics
M
icro
elec
tron
ics
G
eost
atis
tics
C
ompu
tatio
nalb
iolo
gy
Com
pute
rgr
aphi
cs
Gam
es
…
102
7
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
25iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.4
K-N
eare
stne
ighb
ors
(K-N
N)
C
lass
ifie
s b
y g
etti
ng
th
e cl
ass
of
the
K c
lose
st t
rain
ing
ex
amp
les
in t
he
feat
ure
sp
ace
K=
1K
=5
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
26iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.4
K-N
eare
stne
ighb
ors
(K-N
N)
E
asy
to im
plem
ent
na
ive
vers
ion
H
igh
dim
ensi
onal
dat
a ne
eds
dim
ensi
onre
duct
ion
La
rge
data
sets
mak
eit
com
puta
tiona
lexp
ensi
ve
M
any
k-N
N a
lgor
ithm
str
y to
red
uce
the
num
ber
of
dist
ance
eval
uatio
nspe
rfor
med
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
27iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.5
Art
ifici
al n
eura
l net
wor
ks(A
NN
)
S
yste
ms
of
inte
rco
nn
ecte
dn
euro
ns
that
com
pu
te f
rom
inp
uts
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
28iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.5
Art
ifici
al n
eura
l net
wor
ks(A
NN
)
Hu
man
Art
ific
ial
Neu
ron
Pro
cess
ing
elem
ent
Den
drite
sC
ombi
ning
func
tion
Cel
lbod
yT
rans
fer
func
tion
Axo
nsE
lem
ent
outp
ut
Syn
apse
sW
eigh
ts
103
8
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
29iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.5
Art
ifici
al n
eura
l net
wor
ks(A
NN
)
Exa
mpl
e:
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
30iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.5
Art
ifici
al n
eura
l net
wor
ks(A
NN
)
P
erce
ptr
on
si
ngle
-laye
r ar
tific
ial n
etw
ork
with
one
neu
ron
ca
lcul
ates
the
linea
r co
mbi
natio
n of
its
inpu
ts a
nd p
asse
s it
thro
ugh
a th
resh
old
activ
atio
n fu
nctio
n
Eq
uiv
alen
t to
a li
nea
r d
iscr
imin
ant
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
31iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.5
Art
ifici
al n
eura
l net
wor
ks(A
NN
)
P
erce
ptr
on
Eq
uiv
alen
t to
a li
nea
r d
iscr
imin
ant
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
32iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
4.5
Art
ifici
al n
eura
l net
wor
ks(A
NN
)
L
earn
ing
Le
arn
the
wei
ghts
(and
thr
esho
ld)
S
ampl
esar
e pr
esen
ted
If
outp
ut is
inco
rrec
tadj
ustt
hew
eigh
tsan
d th
resh
old
tow
ards
desi
red
outp
ut
If
the
outp
ut is
corr
ect,
do n
othi
ng
104
9
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
Le
ctur
e1
iCS
C20
14
24-2
5 F
ebru
ary
2014
, CE
RN
Intr
od
uct
ion
to
mac
hin
e le
arn
ing
an
d d
ata
min
ing
33iC
SC
2014
, Ju
an L
óp
ezG
on
zále
z, U
niv
ersi
ty o
f O
vied
o
Q &
A
105
106
LECTURE 9 Self organizing maps A visualization technique with data dimension reduction
Tuesday 25 February
16:00-17:00
Description In this lecture, the general concepts of self-organizing maps and its properties will be explained. Starting from the classic neural network approach, a MLP (introduced in the previous lecture), the concept of SOM will be explained. Its structure, the learning process and the later classification of the inputs for not seen cases. The main features of the maps: dimensional reduction and the conservation of the topological properties of the inputs, will be highlighted. Also, a small example will be shown where the attendants will see an actual map arranging itself and the resultant order will be interpreted. Finally, some other SOM based models will be shown to point out different architectures and possibilities.
Juan Lopez Gonzalez
CERN
Audience After this lecture, the attendees are expected to understand how basic self-organizing maps are built, to understand and interpret the properties of the resultant maps. Also, they will learn its advantages and disadvantages. Finally, they will see more complex SOM structures that could help them creating more specific problem-oriented models.
Pre-requisite This lecture can be followed by anyone having a basic knowledge of Artificial Neural Networks and machine learning.
![[iCSC2014] supporter package](https://static.documents.pub/doc/80x56/568bf23f1a28ab893395fa4c/icsc2014-supporter-package.jpg)
