CS 150 - Spring 2007 – Lec. #11: Course Project - 1
Videoconferencing Project
Project Concept and Background Checkpoint Structure Bells and Whistles
CS 150 - Spring 2007 – Lec. #11: Course Project - 2
Objectives
Broad “brush” overview of the project
Details will be covered in the lab lectures, starting next week
NOTE: anything discussed in the lab lectures and project checkpoint write-ups supercedes what I describe here! Neil and Allen have a working implementation of the
project They know the project better than I do! Listen to them!
CS 150 - Spring 2007 – Lec. #11: Course Project - 3
Course Project: Videoconferencing System
Not quite this… but: Video camera capture CRT video display Serial compressed video
2-way transmission between two stations
Wireless communications (no audio this semester) Implemented in a
Xilinx FPGA on theCalinx boards in the lab
Groups of two -- your Lab #4/#5 partner Commit to a TA now for grading purposes
CS 150 - Spring 2007 – Lec. #11: Course Project - 4
Calinx EECS 150 Lab/Project Protoboard
Flash Card & Micro-drive Port
Video Encoder & Decoder
AC ’97 Codec & Power Amp
Video & Audio Ports Four 100 Mb Ethernet Ports
8 Meg x 32SDRAM
Quad Ethernet Transceiver
XilinxVirtex 2000ESeven Segment
LED Displays
Prototype Area
CS 150 - Spring 2007 – Lec. #11: Course Project - 5
Complete Videoconferencing System
Display
Video Encoder
Video Encoder(Checkpoint #1)
Video Decoder
Camera
Videostream
VideoDecoder
Checkpoint #2
Checkpoint #4
SDRAM(Checkpoint #0)
Multiport SDRAMMemory System
MultiportArbitration
Wireless Transceiver(Checkpoint #3)
CS 150 - Spring 2007 – Lec. #11: Course Project - 6
Checkpoint #0/#1/#2: SDRAM Interface
Memory protocols Bus arbitration Address phase Data phase
DRAM is large, but few address lines and slow Row & col address Wait states
Synchronous DRAM provides fast synchronous access current block Little like a cache in the DRAM Fast burst of data
Arbitration for shared resource
CS 150 - Spring 2007 – Lec. #11: Course Project - 7
Checkpoint #1: Video Encoding Pixel Array:
Digital image represented by matrix of values, where each is a function of the information surrounding it in the image; single element in image matrix: picture element or pixel (includes info for all color components)
Array size varies for different apps and costs: some common sizes shown
Frames: Illusion of motion created
by successively flashing still pictures called frames
High-Definition Television (HDTV), 2 Mpx
Workstation, 1 Mpx
PC/Mac, 1‡2 Mpx
Video, 300 Kpx
SIF,82 Kpx
High-Definition Television (HDTV), 1 Mpx
CS 150 - Spring 2007 – Lec. #11: Course Project - 8
Checkpoint #1: Video Encoding
Video details fairly complex and involve many choices: NTSC vs. PAL, HDTV, … Interleaved even-odd frames (TV) vs. progress scan (computer
and digital displays) Frame size, frame rate Pixel encodings: RGB, YUV/YCB (Luminance, Chrominance --
brightness plus color difference signals) Subsampling to reduce data demands (compression trick) Inputs: ITU-R BT.601 Format (Digital Broadcast NTSC) Outputs: Component video, S-video to drive LCDs in lab Fortunately, Calinx board has a chip on-board that deals with
much of the grungy details …
CS 150 - Spring 2007 – Lec. #11: Course Project - 9
ITU-R BT.656 Details Interfacing details for ITU-601
Pixels per line 858Lines per frame 525Frames/sec 29.97Pixels/sec 13.5 MViewable pixels/line 720Viewable lines/frame 487
With 4:2:2 chroma sub-sampling, send 2 words/pixel (Cr/Y/Cb/Y)
Words/sec = 27MEncoder runs off a 27MHz clock
Control info (horizontal & vertical synch) is multiplexed on data lines
Encoder data stream show to right
See video tutorial documents on course documentation web page!
718 719 720 721 0 1 2
359 360 0 1
359 360 0 1
736732( )
368366( )
368366( )
857863)(
Y 718
Y 719
C 360
B Y 720
C 360
RY 721
C 359
BC 359
RY 736(732)
C 368(366)
BC 368(366)
RY 855(861)
C 428(431)
BY 856(862)
Y 857(863)
C 0B
Y 0
C 0R
Y 1
C 428(431)
R
C 0B
Y 0
Y 1
C 0R
C 359
BY 718
Y 719
C 359
R
Last sampleof digital active line
Sample datafor O instant
First sampleof digital active lineH
Luminancedata, Y
Chrominancedata, CR
Chrominancedata, CB
Replaced bytiming reference
signal
Replaced bydigital blanking data
Replaced bytiming reference
signal
End ofactive video
Start ofactive video
Timing reference signals
Note 1 – Sample identification numbers in parentheses are for 625-line systems where these differ from those for 525-line systems. (See also Recommendation ITU-R BT.803.)
FIGURE 1
Composition of interface data stream
D01
CS 150 - Spring 2007 – Lec. #11: Course Project - 10
Checkpoint #1: Video Encoder
Display driver processes pixels within frame buffer Drive ADV7194 video encoder device to output correct NTSC video Gain lots of experience reading data sheets Dictates the 27 MHz operation rate
Used throughout graphics subsystem
CS 150 - Spring 2007 – Lec. #11: Course Project - 11
Calinx On-Board Video Encoder Analog Devices ADV7194: ITU 601/656 in, Composite Video
Out
Supports: Multiple input formats and outputs
Operational modes, slave/master
Used in default mode: ITU-601 as slave
s-video output
Digital input side connected to Virtex pins
Analog output side wired to on board connectors or headers
I2C interface for initialization: Wired to Virtex
CS 150 - Spring 2007 – Lec. #11: Course Project - 12
SDRAM READ Burst Timing
CS 150 - Spring 2007 – Lec. #11: Course Project - 13
Checkpoint #2: Video Decode
Pretty much the reverse of the encoding process of Checkpoint #1
We will provide the base Verilog for video decode
You will need to integrate video decode with your SDRAM arbitrated write port
Integrate with your Checkpoint #1
CS 150 - Spring 2007 – Lec. #11: Course Project - 14
Checkpoint #3: Wireless Transceiver
This will involve interfacing to the wireless transceiver chip on the Calinx2 board
Neil working on a clear description of how this works
CS 150 - Spring 2007 – Lec. #11: Course Project - 15
Checkpoint Build Up to Complete Project
Week #7: Lab #6/Checkpoint #0 -- Basic SDRAM Subsystem
Week #8: Checkpoint #1 -- SDRAM to Video Display (Encoder)
Week #9: Checkpoint #2 -- Local Video System Video Capture (Decoder) to SDRAM to Video Display (Encoder) Video Decoder Verilog will be provided to you
Week #10/11 : Checkpoint #3 -- Wireless Transceiver Midterm #2 scheduled for Week #10 Spring break between Week #10 and #11
Week #12/13: Checkpoint #4 -- Putting it altogether Video Capture to SDRAM to Wireless Transceiver to SDRAM to
Video Display
Week #14: Final Report
CS 150 - Spring 2007 – Lec. #11: Course Project - 16
Possible Bells and Whistles
Still thinking about this but here are some ideas: Performance tuning: larger remote display, higher refresh rate Sending more data per unit time via
compression/decompression through the wireless transceiver Your good idea here NOTE: We don’t necessary know how to implement these
ourselves! (these haven’t been implemented in the TA solution, for example)
NOTE: There will be a bonus for an early demo of the complete project at the end of Week #12 (one week early)
NOTE: Extra credit will be limited to 20% extra points and no extra credit unless the standard functionality works