u-darmstadt.de Fax. +49 6151 16-3052
, Dept. of Computer Science51 166151,M.tu-darmstadt.de Fax. +49 6151 166152
ogy
03A-video.fm 1 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser Multimedia-Systems:
Video
Prof. Dr.-Ing. Ralf SteinmetzProf. Dr. Max MühlhäuserMM: TU Darmstadt - Darmstadt University of Technology,
Dept. of of Computer ScienceTK - Telecooperation, Tel.+49 6151 16-3709,Alexanderstr. 6, D-64283 Darmstadt, Germany, [email protected]
RS: TU Darmstadt - Darmstadt University of Technology,Dept. of Electrical Engineering and Information Technology
KOM - Industrial Process and System Communications, Tel.+49 61Merckstr. 25, D-64283 Darmstadt, Germany, Ralf.Steinmetz@KOGMD -German National Research Center for Information Technolhttc - Hessian Telemedia Technology Competence-Center e.V
User Interfaces
.Synchro-nization
GroupCommuni-
cations
Programming
ms Communications
ice Networks
ession
Video Audio
03A-video.fm 2 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Scope
Usa
ge Applications
Learning & Teaching Design
Ser
vice
s ContentProcess-
ing
Docu-ments
Security ..S
yste
ms Databases
Media-Server Operating Syste
Opt. Memories Quality of Serv
Bas
ics Computer
Archi-tectures
Compr
Image &Graphics
Animation
levision
g
03A-video.fm 3 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Contents
1. Intention
2. Human Visual Perception
3. Video Generation / Capturing
4. Video Coding and Transmission
5. Video Presentation
6. Conventional (Analog) Video Broadcast / Te
7. Digital Television / Digital Video Broadcastin
its characteristicsay and tomorrow
uman physology
ynthesis (computer-gen.)
03A-video.fm 4 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
1. Intention
to provide• basic understanding of the media „video“ and• to describe (in general) video standards of tod
not to provide• lecture on TV or digital TV
look at “video” as the process of:• generation -> transmission -> perception• with
• perception• which is influenced by output device and h
• generation• either capturing (processing, storage) or s
• transmission• in the analog and digital world
uous?)
al perception
still be identified as separateion
03A-video.fm 5 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
2. Human Visual Perception
Human eye:• build up of
• cones to perceive color• rods to perceive brightness
• minimal distance between two uvula (cones)• 0.004 mm
• perceive single images and sequences (contin
Specification of video systems determined by:• Characteristics and limitations of human visu
• spatial resolution• brightness• black/white vs. color
• Human information processing• interpolation...:• "low" frequency: pictures and events that can• "high" frequency: impression of coherent mot
rightness
:
an cones):
undings
03A-video.fm 6 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Visual Perception: Resolution and B
Spatial resolution (of single points) depends on• Image size• Viewing distance
Perception of brightness:• Higher than perception of color (more rods th
• Especially high perception of bright edges• Perception decreases with brightness of surro
• Different perception of the primary colors• Relative brightness:
green : red : blue = 59% : 30% : 11%
Example:• 2 lines with distance of 1 mm are
recognized as two lines if:• Viewing distance < 3m or• Viewing angle >10°
olor spectrum
ntical visual effect!!ensitivity of eye
03A-video.fm 7 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Visual Perception: Colors
Each color defined by mix of primary colors:• Red• Green• Blue
E.g. white is represented by:• red : green : blue = 100% : 100% : 100%
Human perception: used to continuous mix in c• about 250nm --- 780 nm wavelengths• ought to regenerate this mix?• NO: mix of 3 monochromatic waves yields ide• use R / G / B (700 / 546 / 436 nm) due to high s
Color Cube:
black
green
red
blue
white
grey
tion
quency high enoughence
n too low (< 50 Hz)
ma: 3 x 16 Hz), interlacing or
*25Hz) and non-interlaced!!
t
rate
03A-video.fm 8 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Visual Perception: Temporal Resolu• effects caused by inertia of human eye• for a sequence of images (= „frames“):
• no identification of single frames if refresh fre• perception of 16 frames/s as continuous sequ
• special effect: Flicker• perceived if frame rate or refresh rate of scree• especially in large bright areas• maybe reduced by addional interruptions (cine
additional buffering (100 Hz TV sets)
• Higher refresh rate requires:• Higher scanning frequency• Higher bandwidth
• THUS the difference between interlaced (TV: 2
refresh rate 25 Hz50 Hz
1 / fv = 1/50 s = 20 ms
1 / fv = 1/25 s = 40 ms
Bp
OpticalStimulus
fv = refresh
tical lines
03A-video.fm 9 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Visual Perception: „Kell Effect“
Raster interference between:• Raster of scanning system and• Raster of scanned image
Correct reproduction of line if:• Line at scanning line
Incorrect reproduction of line if:• Line between scanning line
i.e. Kell-factor:• Ratio: number of active lines to number of ver• K = 0.64• Europe: 625 x 0.64 = 400 active lines• USA and Japan: 525 x 0.64 = active 336 lines
onal TV)
03A-video.fm 10 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Visual Perception: Further aspects
perception is also influenced by:• viewing distance• display ratio (width / height - 4 /3 for conventi• number of details still visible• spatial (3D) impression• intensity (luminance)• dynamics of changes• ...
Why all the hype about 16:9 ???• represents ratio of human vision field• 4:3 yields peep-hole effect• 16:9 fosters "immersive" sensation
/ transmission / display of
ues by a certain value /es“
systems,
lly“ - think of your computers could even use TVs as
single image
03A-video.fm 11 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
3. Video Generation / Capturing• generation / capturing / storage / manipulation
single images based on:• grey-scale or color values of its pixels
• basic knowledge (pixel, representing pixel valnumber of bytes ...) known from lecture „Imag
• mention:• due to the characteristics of todays television
we will also deal with analog methods• video signal may also be generated „analytica
display as a TV set / early personal computermonitors
pixel
scanning
-> results in a sequence of values for every
ans of a scanning process
ces - CCD)
03A-video.fm 12 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Capturing: Video Cameras• Intention:
• Transformation of a two-dimensional pictureinto a one-dimensional electrical signal by me(e.g. line by line)
• Principle of operation:• Plate of photosensitive material• Evolving of a charge in the material
depending on amount of light at each spot• Charge read out:
• Emitting an electron beam onto the plate• Collecting generated signals
• Alternative: silicon chip (Charge coupled devi
color cube
green (Gc), and blue (Bc)
ry colorslor depth
(Rc,Gc,Bc)
03A-video.fm 13 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
4. Video Coding and Transmission
Basic question:• how to represent video as continous signal• how to transmit it via a single „channel“
RGB color coding:• Color code = coordinates of a point within the
• Three independent components for red (Rc),• I.e. code is a tripel (Rc,Gc,Bc),
Rc, Gc, Bc being the contributions of the prima• Number of bits per component determines co
green
red
bluez
y
x
t. UV axes
Y
Black
Blue
Magenta
White
ellow
Cyan
RedY
Black
Blue
Magenta
White
ellow
Cyan
Red
Black
Blue
Magenta
White
ellow
Cyan
Red
03A-video.fm 14 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Alternative Representations
above: RGB --> representation in YUV "space"
YIQ (US): IQ axes just rotated by 30 degrees wr
Color Cube:
black
green
red
blue
white
grey
GreenGreenGreen
V
U
Red
Cyan
Blue
Magenta
Green
Yellow
Cut-out UV plane
V
U
Red
Cyan
Blue
Magenta
Green
Yellow
Cut-out UV plane
V
U
Red
Cyan
Blue
Magenta
Green
Yellow
Cut-out UV plane
nce Principle
omponents:
f pixel
-0.29G + 0.44B)R -0.52G -0.10B)
)
03A-video.fm 15 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Color Coding: Luminance/Chromina
Code consists of luminance and chrominance c• Luminance component: brightness of pixel• Chrominance difference components: color o
• Hue: which color• Saturation: depth of color
Examples:• YUV coding:
• Y = 0.30R + 0.59G + 0.11B (luminance)• U = (B - Y) x 0.493(chrominance 1) (= -0.15R• V = (R-Y) x 0.877(chrominance 2) (= +0.62
• YIQ coding:• Y = 0.30R + 0.59G + 0.11B(luminance)• I = 0.60R - 0.28G - 0.32B(chrominance 1• Q = 0.21R - 052G + 0.31B(chrominance 2)
nce (cont.)
nance possible:
interlaced (half-)frames
V1
V2
+
V+
03A-video.fm 16 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Color Coding: Luminance/Chromina
Different resolutions for luminance and chromi• Luminance Y:high resolution• Chrominance U, V: lower resolution
Examples:• 4:2:2: double resolution for luminance
• 4:1:1: quadrupel resolution for luminance
• 4:1:0: UV like in 4:1:1, but only for one of two
Y11
Y22Y21
Y12 U1
U2
+Codingof fourpixels:
Y11
Y22Y21
Y12
U+Codingof fourpixels:
t
03A-video.fm 17 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Composite Signal
Composite signal:• Image data• Sampling data• Synchronization data
active lineline 64µs
10%
100%am
plitu
deof
black
grey
white
com
posi
tesi
gnal
audio carrier
6.75
+- 50 kHz
f in MHz
03A-video.fm 18 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Composite Signal
Signal modulation:• Amplitude modulation
Spectrum of composite signal:
image carrier
1 2 3 4 5 60
6.75 MHz
1.25
over one channel
ite
t
03A-video.fm 19 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Composite Coding
One signal:• Transmission of luminance and chrominance• „Composite Color Signal“
black
red
blue
yellow whactive line
line 64µs
color carrier
10%
100%luminance
signal
sion of a cathodeterial caused by electrons
tional to image brightnessss and color information
video memory)
03A-video.fm 20 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
5. Video Presentation
Cathode Ray Tube (CRT):• Transformation of one-dimensional electrical
into two-dimensional visual image
• Principle of operation:• Transformation of signal into electron emis• Light emission in a layer of fluorescent ma
• Variants:• Black-and-white: Signal amplitude propor• Color: Signal includes brightne
(luminance and chrominance)
nowadays: LCD or TTF Displays:• with analog interface (for compatibility)• with digital interface (direct transmission from
rds
coloursx 16 colours
colours
ced Graphic Port - AGP)t supporttangle fill)
03A-video.fm 21 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Video Presentation: Adapter Standa
history:• Hercules Monochrome Adapter• Color Graphics Adapter (CGA) - 320x200 x 4• Enhanced Graphics Adapter (EGA) - 640x350
actual:• Video Graphics Array (VGA) - 640x480 x 256• Super Video Graphics Array (SVGA)
actual developments:• dedicated busses for fast data transfer (Advan• dedicated video processors (accelarators) tha
• manipulation of graphic primitives (e.g. rec• visibility algorithms (e.g. Z-buffer)• or texturing algorithms• hardware support for MPEG presentation• ...
adcast / Television
)
oding modul-ation
YIQ AM
YUVeq. trans-
ission)FM
YUVphase sig-
nal)AM
03A-video.fm 22 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
6. Conventional (Analog) Video Bro
Transmission Standards:• NTSC (National Television Systems Comittee• SECAM (Sequentiel Colour avec Memoire)• PAL (Phase Alternating Line)
standard lines pixels/line
frames/sec c
NTSC 525 700 30
Secam 625 864 25(s
m
PAL 625 864 25(2-
mponents (HD-MAC):
ntatives, television, research
E):
Hztandard
.94
03A-video.fm 23 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
HDTV Standards
European High Definition Multiple Analogue Co• Defined in Eureka Project EU95• Cooperation of 35 European industry represe
centers• ‘Some’ compatibility to existing standards
Japanese Multiple Sub-Nyquist Encoding (MUS• Not open to TV standards• Vertical resolution: 1125 lines, Frame rate: 60• 1992: 1 hour/day broadcasting using MUSE s
USA:• Goal: compatibility to NTSC• Vertical resolution: 1050 lines, Frame rate: 59
roadcasting
tandard
ite signal:
nal:
of the color carrierrrier
03A-video.fm 24 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
7. Digital Television / Digital Video B
History• 1982: CCIR international digital television s• ....: HDTV• 1995: DVB Digital Video Broadcasting
Starting point:• Combined Digitalization of video signals
Combined Digitalization of composite black/wh• Video bandwidth: 2 * 5 MHz * 8 bit = 80 Mbit/s
• 5 MHz = bandwidth of B/W TV signal
Combined Digitalization of composite color sig• Lower interference with color carrier• Requires even-numbered multiple frequency• Sampling frequency: 4 * frequency of color ca• Data rate: 4 * 4.43 MHz * 8 bit = 141.8 Mbit/s
• 4.43 MHz = frequency of color carrier
ent Coding
ance dataard
componentscomponents
le components:
z
216 Mbit/s
Pal & Ntscs/s
.5 was chosen
th PAL/NTSC
03A-video.fm 25 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Combined digitalization vs. Compon• Drawbacks of combinized digitalization:
• Interference between luminance and chromin• Digitalization technique depends on TV stand• No adaptation of:
• Sampling frequency to bandwidth of single• Data reduction to characteristics of single
• New approach: Separate digitalization of sing• Multiplexing of digitized signals
• Sampling frequency:• Luminance: 13.5 MHz, Chrominance: 6.75 MH
• 8 bit uniform quantization:• Data rate: (13.5 MHz + 2 * 6.75 MHz) * 8 bit =
• sampling frequency: 13.5 MHz for both– PAL: 625*50*½ lines/s, NTSC: 525*59.94*½ line– common multiples: 2.25 MHz, ... 13.5 MHz, ...; 13
• resolution:– PAL: 864 Samples/line, NTSC: 858 Samples/line
864*625*25 = 858*525*29,97 = 13.5 Mio.– visible lines 576 bzw. 486– Sichtbarer part of line: studio qual 720 pixel, bo– ! resolution 486*720 oder 576*720
)els)
bit/s, 34.368 Mbit/s , …)
rd 2 substandard 3
Hz
)
9 MHz(2/3 of
standard)
z
)
2.25 MHz(1/3 of
standard)
it/s 108 *106 bit/s
03A-video.fm 26 15.March.01
Scope
Contents
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.t
k.in
form
atik
.tu
-da
rmst
adt
.de
©R
.S
tein
me
tz,
M.
Mü
hlh
äu
ser
Digital Television: Features• Vertical resolution:
• 625 lines/frame• Horizontal resolution:
• Luminance: 864 pixels/line (visible: 720 pixels• Chrominance: 432 pixels/line (visible: 360 pix
• Frame rate: 25 frames/s
• High data rate• Not compatible to PCM hierarchy (139.264 M
Hence: definition of substandards• Lower data rates• Lower sampling frequencies
substandard 1 substanda
samplingfrequency
luminance
11.25 MHz(5/6 of
standard)
10.125 M(3/4 of
standard
chrominance
5.625 MHz(5/6 of
standard)
3.375 MH(1/2 of
standard
data rate 180*106 bit/s 135 *106 b