© 2011 Altera Corporation—Internal
Implementing Video Systems Using the Altera Video Framework
© 2011 Altera Corporation—Internal2
AGENDA
What is the Video Framework?- Overview
VIP Core Details
Boards/Reference Designs
© 2011 Altera Corporation—Internal3
Altera Video Framework Building block IP cores to speed up
development (VIP suite) Open, low-overhead interface standard to
mix-and-match custom or off-the shelf IP blocks
Format conversion reference designs that showcase silicon capability, provide a starting point for design
System Level Tools and design methodology
Variety of development boards/kits for rapid design prototyping
© 2011 Altera Corporation—Internal4
Building block IP to speed up development- Interfaces (SDI, DVI, etc), High Performance Memory Controllers, Nios- Video IP suite: library of interoperable Video IP blocks using common standards
and protocols Open interface standard to mix-and-match custom or off-
the shelf IP- Avalon Memory-Mapped and Avalon-Streaming Video protocol- Open Source HDL Template to facilitate integration of custom IP into framework
The Video IP framework – The IP
Custom IP
Input
interface
Clocked Video Input
Output
interface
Video in Video out
Clocked Video Input
Altera VIP 1
Altera VIP 2
Avalon-ST Video (over Avalon-ST) Avalon-MM
Nios Memory controller
© 2011 Altera Corporation—Internal5
The Video IP framework – Design Tools
Pin mapping,Interface IP,
Top-level HDL
Quartus II
Software control andconfigurationDebug (Nios II & SignalTap)
Nios II IDE
SOPC Builder
Nios II IDE / Quartus II
Development boards/kits for rapid design prototyping
Stratix II GX, Cyclone IIIStratix IV GX PCIe, Arria II GX
Abstract design captureSwitch fabric generation
IP Catalogue tool
Software library to control the cores of the VIP Suite and for interrupt servicing
1080p reference designs: out-of-the-box ASSP-level capabilities
and/or great starting pointsV-series, M-series, partner designs
© 2011 Altera Corporation—Internal6
AGENDA
What is the Video Framework?- Overview
VIP Core Details
Boards/Reference Designs
© 2011 Altera Corporation - Public
7
Video Image Processing (VIP) Suite
BT 656
Avalon ST video
Scaler
Deinterlacer
Chromaresampler
Alphablending mixer
Color planesequencer
Frame buffer
Image clipper
Gammacorrection
Test patterngenerator
2D FIR filter
2D median filter
Color spaceconverter
Avalon ST video
BT 656
Half a dozen hardware verified video processingreference designs using these functions
Available TODAY!
Switch
Control Sync
Frame ReaderScaler II
v11.0Deinterlacer II
v10.1
© 2011 Altera Corporation - Public
New Cores in VIP Suite (2011)
Scaler II- Reduces area and improves performance
Compared to first-generation scaler- Supports 4:2:2 chroma data sampling rate
Further reduces area used
Deinterlacer II- Supports 3:2 cadence detection- Supports low-angle edge detection- Delivers lower latency
8
© 2011 Altera Corporation—Internal9
VIP Function Details Chroma Resampler This function allows you to
change between 4:4:4, 4:2:2 and 4:2:0 sampling rates
For 4:2:2 4:4:4, the filtered algorithm uses a 4-tap filter with fixed Lanczos-2 coefficients
For 4:4:4 4:2:2, the filtered algorithm uses a 7-tap filter with fixed Lanczos-2 coefficients
© 2011 Altera Corporation—Internal10
VIP Function Details Color Space Converter This function provides a
flexible and efficient means to convert image data from one color space to another
You can select from a list of common color space conversion
Or choose a custom color space
© 2011 Altera Corporation—Internal11
VIP Function Details Deinterlacer / Deinterlacer II (motion adaptive)
Select the input image height and width Choose the deinterlacing algorithm –
bob, weave, motion adaptive
The weave and motion adaptive algorithms require external frame buffering
© 2011 Altera Corporation—Internal12
Understanding Algorithmic Constraints Deinterlacer (motion adaptive) Motion adaptive deinterlacer requires five
(master) accesses to the DDR memory- 1 field write- 2 field reads- 1 motion vector write- 1 motion vector read
Calculating DDR memory bandwidth- Input format: 1080i, 60 fields/sec, 10-bit color
1920 × 1080 × 30bits × 60/2 = 1.866Gbit/s- Output format: 1080p, 60 frames/sec, 10-bit color
1920 × 1080 × 30bits × 60 = 3.732Gbit/s- Motion format: Only use 10bits for the motion values
1920 × 1080 × 10bits × 60/2 = 0.622Gbit/s - Memory access:
1 × write at input rate: 1.866Gbit/s 1 × write at motion rate: 0.622Gbit/s 1 × read at motion rate: 0.622Gbit/s 2 × read at output rate: 7.464Gbit/s
Total: 10.574Gbit/s
Deinterlacer (MA)
DDR2
4:4:4 Mode, Motion Bleed is ON
4:4:4 Mode
© 2011 Altera Corporation—Internal13
Understanding Algorithmic Constraints Deinterlacer (motion adaptive) Motion adaptive deinterlacer requires five
(master) accesses to the DDR memory- 1 field write- 2 field reads- 1 motion vector write- 1 motion vector read
Calculating DDR memory bandwidth- Input format: 1080i, 60 fields/sec, 10-bit color
1920 × 1080 × 20bits × 60/2 = 1.24Gbit/s- Output format: 1080p, 60 frames/sec, 10-bit color
1920 × 1080 × 20bits × 60 = 2.48Gbit/s- Motion format: Only use 10bits for the motion values
1920 × 1080 × 10bits × 60/2 = 0.622Gbit/s - Memory access:
1 × write at input rate: 1.24Gbit/s 1 × write at motion rate: 0.622Gbit/s 1 × read at motion rate: 0.622Gbit/s 2 × read at output rate: 4.96Gbit/s
Total: 7.44Gbit/s
Deinterlacer (MA)
DDR2
4:2:2 Mode, Motion Bleed is ON
4:2:2 Mode, 30% reduction in DDR bandwidth
© 2011 Altera Corporation—Internal14
VIP Function Details Scaler
Choose the scaling algorithm – number of taps/phases
Select the input and output image sizes
Choose from a selection of pre-built filter functions or choose a custom function
Requires progressive 4:4:4 input
© 2011 Altera Corporation - Public
15
VIP Function Details Scaler II – Utilizes Fewer Resources
Reduces area and improves performance- Compared to first-generation scaler
Supports 4:2:2 chroma data sampling rate- Further reduces area used
Algorithm supports - Linear and polyphase with
run-time coefficient load
New core in HDL- More efficient in using resources
© 2011 Altera Corporation—Internal16
VIP Function Details Alpha Blending Mixer
Select the number of layers to be mixed
The level of blending is controlled by the number of alpha bits (up to 8)
Select the size of the image
© 2011 Altera Corporation—Internal17
VIP Function Details Generate a color bar test pattern
© 2011 Altera Corporation—Internal18
BT656 Avalon ST Video The Clocked Video Input
MegaCore function converts from clocked video formats (such as BT656 and DVI) to Avalon-ST Video
It strips the incoming clocked video of horizontal and vertical blanking, leaving only active picture data
No conversion is done to the active picture data- The color plane information
remains the same as in the clocked video format.
© 2011 Altera Corporation—Internal19
BT656 Avalon ST Video The Clocked Video Output
MegaCore function converts from Avalon-ST Video to clocked video formats (such as BT656 and DVI)
It formats Avalon-ST Video into clocked video by inserting horizontal and vertical blanking and generating horizontal and vertical sync information
No conversion is done to the active picture data- The color plane information
remains the same as in the Avalon-ST Video format.
© 2011 Altera Corporation—Internal20
Frame Buffer
© 2011 Altera Corporation—Internal21
Run-time Control of the Video Chain
Build video signal chains that can be updated on-the-fly without changing the HDL or bit-stream- Using the streaming interface protocol: Avalon ST Video- Using an embedded processor
© 2011 Altera Corporation—Internal22
Run-time Control of the Video Chain Avalon-ST Video protocol is
a packet-oriented way to send video and control data- First packet contains a frame of
video data - Second packet contains control
information that applies to the subsequent video packet/frame
Video function 1 Video function 1
DATAVALID
SOPEOP
READYAvalon ST
Examples of Avalon-ST Video Control Packets
© 2011 Altera Corporation—Internal23
Run-time Control of the Video Chain
Most video functions permit run-time control of some aspects of their behavior, using a common type of Avalon-MM slave interface
Each slave interface provides access to a set of control registers which must be set by an embedded processor/logic
Generally, a function can be updated on a frame boundary
Processor
Avalon MMMaster port
Video function 1
Avalon MM control plane
© 2011 Altera Corporation—Internal24
NIOS Control: Cyclone III Example DesignInitialize daughter card Initialize daughter card
Start the IP cores
Control the mixer and Scaler parameters at run-
time
© 2011 Altera Corporation—Internal25
AGENDA
What is the Video Framework?- Overview
VIP Core Details
Boards/Reference Designs
© 2011 Altera Corporation—Internal26
Video Development KitsCyclone III FPGA, $1,895
Stratix IV FPGA, $4,995
Cyclone III FPGA, $2,995
Arria II GX FPGA, $2,995$495
$795
Cyclone IV FPGA, $1,295
© 2011 Altera Corporation—Internal
New Demo in 2011 (VEEK) Demo # 1 – Simple Video Format Conversion
Demo # 2 – Camera Input Scaling
27
Input Processing Output
composite
de-interlaced video /w txt overlay (NTSC to 800x600 with real-time
resizing, zoom and panning) VEEK display VGA out
Download Design Files on Altera Video Wikiwww.alterawiki.com/wiki/Videoframework
Input Processing Output
Camera on VEEK
Camera Input/w txt overlay (Camera input scaling with real-time
resizing, zoom and panning) VEEK display VGA out
© 2011 Altera Corporation—Internal
New Demo in 2011 (VEEK)
Demo # 3 – Picture in Picture (PiP)
Demo # 4 - Multi-view
28
Input Processing Output composite de-interlaced video /w txt overlay VEEK display
camera on VEEK PIP - (camera source) VGA out
Input Processing Output composite CH 1 / 3: de-interlaced video DVI (HSMC) @ 800x600
camera (GPIO) CH 2 : camera videoDVI (HSMC) CH 4: DVI video (800x600 down scaling)
Watch Demo of VIP Suite
© 2011 Altera Corporation—Internal
New Low Cost UDX Reference Design
C4GX150 board- Develop in Quartus II software v11.0 using Qsys- Available at end of July
1 channel UDX design- SDI / DVI input switch- SDI / DVI output- Resolution supported
1920 x 1080P 50/59.94/60Hz1280 x 720P 50/59.94/60Hz
- Text overlay and testing pattern
29
© 2011 Altera Corporation—Internal
Video Reference DesignsDesign Device Features I/Os
Video and image processing (VIP) demo
Cyclone III, Cyclone IV
National Television System Committee (NTSC)-to-480p scaling, 720p camera input
scaling, mixing for PiP, multiview , 1-channel UDX design
Analog, digital video interface (DVI)
Cyclone® III FPGA design Cyclone III NTSC-to-720p scaling, MA deinterlacer,
scaler, mixing for PiP Analog, DVI
UDX2 Arria® II GX Same as UDX4 Serial digital interface (SDI), DVI
UDX3/4 Stratix® IV GX
V1, on-screen display (OSD), switch, active format description (AFD),
mixing, multiport front-end (MPFE), 1080p, 720p, 1080i, ntsc/pal, 4:2:2 scaler, MA
deinterlacer with low-angle edge
SDI, DVI, high-definition multimedia interface
(HDMI)
4K scaling Stratix IV GX 1080p 4K scaling, component design methodology
SDI4x SDI for output
Broad Portfolio of Reference Designs to Help You Get Started
30
© 2011 Altera Corporation - Public
31
Summary
Altera video framework enables rapid development- Mix and match existing IP – Leverage Altera’s open interface
standard - Automatically integrate embedded processors and arbitration
logic- Leverage building block IP provided by Altera- Use existing reference designs as starting points
And rapid prototyping- Implement design using the appropriate development boards- Test the design with actual video signals
© 2011 Altera Corporation—Internal32
Next Step Coming Video Framework Workshop, please contact Altera Sales orDistributors for more
information
Provide an introduction to Video and Image Processing suite including the cores, reference designs, roadmap, video framework, etc.
For information on:- DDR-based Memory controllers
http://www.altera.com/education/training/courses/IMEM210 http://www.altera.com/technology/memory/mem-index.jsp
- SOPC Builder http://www.altera.com/education/training/courses/OEMB1115 http://www.altera.com/literature/lit-sop.jsp
- System Console http://www.altera.com/education/training/courses/OEMB1117 http://www.altera.com/literature/ug/ug_system_console.pdf
- NIOS-II Flow http://www.altera.com/education/training/courses/IEMB115 http://www.altera.com/literature/lit-nio2.jsp
- Video and Image Processing Suite http://www.altera.com/products/ip/dsp/image_video_processing/m-alt-vipsuite.html http://www.altera.com/end-markets/broadcast/1080p/bro-1080p.html http://www.altera.com/support/refdesigns/sys-sol/broadcast/ref-format-conversion.html
- On-chip Debugging, including SignalTap-II http://www.altera.com/education/training/courses/ODSW1164 http://www.altera.com/support/software/debugging/sof-qts-debugging.html
© 2011 Altera Corporation—Internal
Backup
33