SMPTE ST 2110 SVIP

Timing and Video TransportPaul Briscoe

Televisionary Consulting / Evertz Microsystems

Toronto, Canada

Setting the Baseline

• We are talking about:• Private networks

• Heavily loaded switch fabrics

• IP multicast transport

• High sustained bandwidth

• No mix of other traffic

• Lossless (for UDP)

• Low latency

• Deterministic stream switching performance

• Ability to support PTP

Elemental Pieces of the Puzzle

• Timing and Synchronization• Necessary for live systems

• Existing methods need to be accommodated

• New technologies need to be supported• Networks, realtime packet transport, independent stream alignment.

• Video Signal Transport• Just the pixels

• RTP packets over IP Multicast

Synchronization Today

Master 1

`

GNSS

BlackBurst

DARS

Timecode

Master 2

Auto

chang

eover

``

``

`

`

`

`

`

`

BlackBurst

DARS

Timecode

Multiple Distribution TreesCentralized Equipment

Facility Equipment

Slave

Masters provide redundancy

Autochangeover monitors signal health

Without external reference, they lock together

With external reference, they lock to that

Legacy Synchronization Methods

• Fully meet existing needs

• Standard / Format specific

• Requires dedicated distribution infrastructure(s)• CAPEX, OPEX, SPOF

• Maybe more than one signal?• Black burst, time code, DARS

• Analog signals = Old school

• It’s time to move on. To networks.

Modern Synchronization Requirements

- meet legacy performance requirements and expectations

- one distribution infrastructure, not many

- one method to carry all references, not many

- a means of providing redundancy in the distribution (and masters)

- something that would directly support new / future standards

- something that can be externally / globally referenced

- can coexist with essence without performance impact

- as close to plug and play as today

- something that can run on a network

Network Foundation with PTP

• IEEE1588 Precision Time Protocol (PTP)• A mature IEEE Standard• Delivers wide span precision time over an IP network• Provides for redundancy in masters and network• Can be externally / globally referenced• Offers resilience to traffic-induced perturbations

• Broadly used elsewhere• Power grid, cell networks, robotics, CERN LHC• Chosen by AES for AES67

• Probably a safe bet for our use – but how?

How PTP Works

Master maintains a live running timecount

32 bits of seconds (~136 years)

32 bits of nanoseconds

PTP protocol transfers this time to all slave devices

“Sync Messages” deliver the precise time

“Delay Request / Response Messages” measure path delay

Messages are transacted with slaves periodically

The PTP Time Counter

Master maintains a live running timecount

32 bits of seconds (~136 years)

32 bits of nanoseconds

Counter value was zero at the “PTP Epoch” – 1970.01.01.0000z

PTP protocol transfers this time to all slave devices

“Sync Messages” deliver the precise time

“Delay Request / Response Messages” measure path delay

Messages are transacted with slaves periodically

The PTP Time Counter

1 S

econ

d

1 M

inut

e

1 H

our

1 D

ay

1 M

onth

1 Y

ear

1 M

illis

econ

d

1 M

icro

seco

nd

1 N

anos

econ

d

SDI Video

12M Timecode

Calendar

NTP

AES Audio

Composite Video

IEEE1588

GPS Native

Nanoseconds – 32 bits (1 ns)Whole Seconds - 32 bits (~136 years)

Time Counter

LSB

1 Hz

(PPS)

The PTP Time Counter

This is your life

Putting PTP to Work for SMPTE

SMPTE / EBU “Task Force on Timing and Synchronization” – 2008

Resulted in a report and “Request for Standardization”

SMPTE Standardization activity “ST 2059” – 2010

Published Q1 / 2015

IP wasn’t clearly in the picture when we did this, but we knew it was on the right track

ST 2059 Glues Our World to PTP

Two part document suite:

• Part 1 specifies generation of signals from PTP• Based on Epoch Alignment (just like PTP)

• “SMPTE Epoch” defined to be the same as PTP epoch

All signals of interest (SMPTE + AES3) have an “Alignment Point”

Contains data tables and formulae to generate signals from PTP time

• Part 2 is the “SMPTE Profile” for PTP

Essential PTP operating parameters

Application-specific metadata

ST2059PTP SMPTE

PTP + ST 2059

The small ‘t’ is the input PTP time count value

Legacy Signal Support

• Using PTP and ST2059, we can virtualize all existing signals• Reference signals *or* essence signals

• Slaves locked to a PTP master will all generate the same phases

• Independent PTP masters locked to GNSS will produce the same PTP

• Future unknown formats can be supported• Define their periodicities

• Define their alignment to SMPTE Epoch

Rather than continuously update 2059, these will be captured in future format Standards documents

PTP Synchronization

GNSS

PTP

Network Fabric (cloud of switches)Centralized Equipment

Facility Equipment

PTP Slave

PTP Slave1

Legacy

Master 1 BlackBurst

DARS

TimecodePTP Slave2

Legacy

Master 2

Auto

chang

eover

PTP

Grandmaster

1

Grandmaster 2

Legacy Master contains PTP slave functionality

Legacy signals generated same as any slave

Distributed via tree structure as before

Hybrid Synchronization in a PTP World

GNSS

PTP

Network Fabric (cloud of switches)Centralized Equipment

Facility Equipment

PTP Slave

PTP Slave1

Legacy

Master 1 BlackBurst

DARS

TimecodePTP Slave2

Legacy

Master 2

Auto

chang

eover

PTP

Grandmaster

1

Grandmaster 2

``

`

``

`

`

`

`

`

`

BlackBurst

DARS

Timecode

Facility Equipment

Legacy

Slave

Studio Video over IP Transport

• Can improve on how we do it today• Move from hard wiring / switchboard routing to IP network (obviously)

• Transport the essence streams independently

• For video, only transport the pixels of the active picture area• No audio, no ANC, no TRSs, no whitespace

• Format agnostic

• Transport independence – bulk bandwidth

• Utilize existing IP technologies where possible – invent only what we need

A bit of History

• Video Services Forum (VSF) drafted Technical Recommendation TR-03• Specifies transport of pixels over IP using RTP.

• TR-03 submitted to SMPTE for Standardization

• A subsequent corrigendum was submitted with additional color

• SMPTE formed 32NF-60 WG and DG SVIP• Producing the ST2110 suite of standards

• TR-03 is now a deprecated document – do not use!

A clue that we’re on the right path….

AES67 Audio

IP Media Transport and RTP

• IETF RFC 3550 Real-Time Protocol is a proven method for conveying time-critical data over UDP packets

• Has flexible built-in timestamp synchronization scheme

• Unidirectional, no ACKs or retransmission

• Content agnostic

• Sequence numbering

• Marker bit (handy for identifying vertical)

• Can work for video, audio, ANC…• But how to encapsulate video?

X Gb Fiber, Copper

IEEE 802.3 Ethernet

RFC 791 IP Internet Protocol

RFC 3550 RTP Realtime Protocol

RFC 768 UDP User Datagram Protocol

ST 2110-20 Uncompressed Active Video

• Why reinvent the wheel? Let’s use an existing standard if we canIETF RFC4175 “Payload Format for Uncompressed Video”

Drafted 2002 – 2005, published 2005 – not widely adopted• Provides all of the necessary elements we require• Raster-agnostic (up to 32K x 32K pixels)• Sample structure / bit depth agnostic • Frame rate agnostic

• Future-proof (see above)• Contains more than we need (e.g. 4:1:1)

• Rather than reference it, we decided to only copy the parts of interest• Removes concern about an external standard changing outside of our control• Simplifies our standard document

Features of 2110-20

• Arbitrary raster sizes• Can accommodate all of today’s formats

• Flexible Sample Structure / Bit Depth capabilities

• Data organization• Concept of “pgroups” central to byte (word, etc.) -level data handling

• Descriptive using Session Description Protocol• Registration and Discovery (NMOS IS-04)

• PTP-friendly synchronization mechanism

Timestamps

• Used for stream synchronization

• Derived from PTP by the sender

• Receiver can align related streams with extreme accuracy• Lip (and ANC) sync, audio channels in a sound field

• Receiver can use PTP locally to realign signals to reference

• RTP Timestamps represent the ‘capture time’ of the essence• Subject to much debate! That aside, the 2059 Alignment Point is used• All packets of a given video frame share the same timestamp

• Timestamps are essence-specific• 90 KHz for video, 48 KHz for audio

Timestamps in Action

• Sender uses PTP time to calculate RTP timestamp values

• Alignment point of source video is used to ‘grab’ the time

• Pixel array is packetized and transmitted with that timestamp value

• Associated essence (e.g. audio) is timestamped similarly

• Receiver buffers and re-aligns streams to match timestamps• Can be as many streams as desired

• If PTP-derived, all streams on the network will have same timing

• In the absence of PTP, the sender can use an internal timestamp• Receiver can still realign that sender’s streams (only)

Byte Handling - pgroups

• Mapping anything but 8 (16, etc.) bit entities to bytes is painful

• Pixel Groups (pgroups) is a way to simplify packing and parsing

• Specifies that:• No byte shall be split

• No pixel shall be split

• Ensures that there are an integer number of pixels in every packet

• Not constrained, any arrangement that meets the requirements is OK• Means all receivers need to be able to parse any valid pgroup

Byte Handling - pgroups

• Mapping anything but 8 (16, etc.) bit entities to bytes is painful

• Pixel Groups (pgroups) is a way to simplify packing and parsing

• Specifies that:• No byte shall be split up

• No pixel shall be split up

• Ensures that there are an integer number of pixels in every packet

• Not constrained, any arrangement that meets the requirements is OK• Means all receivers need to be able to parse any valid pgroup

Data Organization – Packing Modes

Two packing modes have been defined:

• General Packing Mode (GPM) follows the open packing of RFC4175

• Block Packing mode is a specifically constrained organization• Based on 180-byte blocks

• Supports large number of formats

• Can result in identically-sized packets on network regardless of format• Good for network engineering / troubleshooting

Block Packing Mode Example

Essence-Descriptive Metadata

• Carried out of band using Session Description Protocol (RFC4566)• Sampling Structure

• Bit Depth

• Raster Height and Width (in pixels)

• Frame Rate

• Colorimetry

• Transfer Characteristic

• Progressive / Interlace / Segmented (PsF)

• Packing Mode

• Pixel Aspect Ratio

Transmitter Behaviour – ST 2110-21

• Describes required transmitter performance for streams• Packet pacing• Bursts• Gaps

• Considers expected network behaviour and link / fabric loading

• Need to accommodate SDI-like sources (with V and H blanking gaps)

• Need to accommodate full-frame-time rasters (no gaps)

• Need to accommodate software senders (sloppy)

• Result (at this moment) is a wide and narrow transmitter profile

Putting it all Together

• PTP is instantiated as a network-wide resource

• IP senders use PTP to generate their media clocks and RTP timestamps

• IP receivers can (optionally) use it to realign to reference

• PTP slave sync generators can drive legacy equipment

• When gateway’d to and from IP, legacy sources and destinations will be synchronous and time-able like today

• System latency is greater in IP – there is learning to be done!

Interop Testing

• A number of interop test sessions have been held• Fox NE&O Lab, Woodlands (Houston), TX

• Continuously growing number of participants

• Conducted in concert with PTP interop testing (separate and together)

• To date – remarkable results!• At NAB we had over 40 vendors interoperating in the IP Showcase demo

• Included AES67 audio-only manufacturers

Clearly we are getting very close to this stuff being ready for prime time

Summary

• PTP + ST 2059 can provide virtualized reference (and essence) signal generation across an IP fabric

• PTP is native to RTP transport

• PTP is the system-level timebase of ST 2110• Drives sender timing• Drives receiver realignment to house.

• ST2110-20 supports all Studio Video formats used today

• Both support unknown future formats because they are parameterized.

SMPTE ST 2110 SVIPTiming and Video

Paul Briscoe

Televisionary Consulting / Evertz Microsystems

Toronto, Canada

televisionary@teksavvy.com