+ All Categories
Home > Documents > CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report...

CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report...

Date post: 06-Jul-2020
Category:
Upload: others
View: 2 times
Download: 0 times
Share this document with a friend
22
CS150 Spring 2007 Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM Controller and Arbiter 4 1.1 Initialize and Load Mode Register ................................. 5 1.2 Read Request ............................................. 5 1.3 Write Request ............................................. 5 1.4 SDRAM Arbiter ........................................... 5 2 Video Capture and Display 8 2.1 Video Encoder ............................................ 8 2.1.1 Counter ............................................ 8 2.1.2 Counter Controlled State Machine ............................. 9 2.1.3 Data Generator ........................................ 10 2.1.4 Data Clipper ......................................... 10 2.1.5 Data Feeder .......................................... 11 2.2 Video Ports .............................................. 11 2.2.1 VD Processor ......................................... 11 2.2.2 VE Processor ......................................... 11 2.2.3 Compressor .......................................... 11 3 Wireless Communications 12 3.1 Transceiver .............................................. 13 3.1.1 Transceiver State Machine .................................. 13 3.1.2 SPI Controller ........................................ 15 3.2 Communications Ports ........................................ 15 3.2.1 Recieving Processor (RR Processor) ............................ 15 3.2.2 Sending Processor (SS Processor) .............................. 16 3.3 Communications Protocol ...................................... 18 3.3.1 Master ............................................. 19 3.3.2 Slave .............................................. 20 4 User Interface 20 4.1 Text Module ............................................. 20 1
Transcript
Page 1: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

CS150 Spring 2007

Wireless Video Conferencing

Final Project Report

Eudean Sun

Nan Yu

April 29, 2007

Contents

I Overview 3

II System Description 3

1 SDRAM Controller and Arbiter 41.1 Initialize and Load Mode Register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Read Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 Write Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4 SDRAM Arbiter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Video Capture and Display 82.1 Video Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.1 Counter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.1.2 Counter Controlled State Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.1.3 Data Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.4 Data Clipper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.5 Data Feeder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 Video Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.1 VD Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.2 VE Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.3 Compressor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Wireless Communications 123.1 Transceiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1.1 Transceiver State Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1.2 SPI Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 Communications Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2.1 Recieving Processor (RR Processor) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2.2 Sending Processor (SS Processor) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.3 Communications Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3.1 Master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3.2 Slave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4 User Interface 204.1 Text Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1

Page 2: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

CONTENTS 2

III Conclusion 21

5 Design Metrics 21

6 References 21

7 Conclusion 21

8 Suggestions 22

Page 3: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3

Abstract

The CS150 project for the Spring 2007 semester consisted of designing and implementing a wirelessvideo conferencing system in Verilog. The final functionality consists of two boards communicatingthrough wireless and displaying both their own local video as well as the remote video from the othernode on a monitor. The design of this system necessitated some compromises: first, there is no audiofunctionality; second, the video that is sent over wireless is low resolution and grayscale; third, theremote video has a low frame rate (about one frame per second). The overall system comprises fiveports to read/write local video and send/receive remote video. An arbiter module controls which ofthese ports has control at any time. By properly designing each of the read/write and send/receive portsand properly controlling these ports through the arbiter module, we have achieved a working wirelesstwo-way video conferencing system.

Part I

Overview

Our project is a two-way wireless video conferencing system. Each of the nodes in the system has a monitor,camera, and wireless interface. Both of the nodes display the following on their screens: in the main partof the screen, local video in full color; in the upper left hand corner, local video that is compressed; inthe upper right hand corner, remote video that is compressed. Since each node can see both video fromitself and video from the remote node via wireless, we can describe this system as a two-way wireless videoconferencing system. An important distinction is that this system lacks audio communications, presumablydue to the limited time available to complete the design as well as the limited bandwidth available in thewireless channel.

Figure 1 shows a high-level diagram of the system we’ve designed and how two of these systems commu-nicate via a wireless channel. The system is comprised of three primary building blocks: an arbiter, the portscontrolled by the arbiter, and a communications system. Since the ports must communicate with SDRAM,performing either reads or writes, only one port can have control at any given time. The arbiter decideswhich port has access to SDRAM, ensuring that no two ports try to read/write to SDRAM at the sametime, but also ensuring that none of the ports experience starvation.

We’ve designed our system to include five of these read/write ports: two to read/write full local video,two to send/receive compressed remote video, and one to write compressed local video. To explain thefunctionality of these ports, let’s consider a typical conference between two nodes A and B. Both A andB are continuously writing and reading local video and displaying it on the monitor. There is an exchangebetween A and B with remote video: A writes a compressed frame, then sends it to B. Upon receiving thisframe, B displays it locally, then writes its own compressed frame and sends that to A, who receives it anddisplays it locally. This exchange continues until communications is lost either by resetting the Xilinx chipor as a result of packet loss.

The communications module is the enabler of the sending and receiving mentioned previously. The portscontrolled by the arbiter interface with the communications module to allow compressed frames to be written,sent, and received at the appropriate times to enable proper video conferencing functionality. In addition, thecommunications module is in charge of an initial handshake between the two nodes A and B. This handshakeallows A and B to identify each other as a pair that is communicating. The communications module alsocontrols which of the nodes is acting as the “Master” or the “Slave”. This distinction is important for theinitial handshake as well as for error conditions due to packet loss and will be described in detail later inthis report.

Part II

System Description

In this section we describe each of the parts of our system in detail. These correspond more or less to thecheckpoints designated in this project.

Page 4: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

1 SDRAM CONTROLLER AND ARBITER 4

SDRAM Arbiter

Video Encoder Processor Video Decoder Processor Compressor Sender Receiver

SDRAM Controller

Communications

ControllerTranceiver

SDRAM Arbiter

Video Encoder Processor Video Decoder Processor Compressor Sender Receiver

SDRAM Controller

Communications

ControllerTranceiver

Wir

ele

ss C

om

munic

ation

System A

System B

Figure 1: System overview

1 SDRAM Controller and Arbiter

The SDRAM controller is our interface to the Micron Technologies MT48LC16M16A2TG SDRAM module.Since our other sub-systems interface heavily with SDRAM, we needed some type of interface that couldhandle reading and writing to SDRAM so that the timing details of these operations could be abstracted

Page 5: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

1 SDRAM CONTROLLER AND ARBITER 5

out of the rest of the video conferencing system.In designing the SDRAM controller, we had to consider how much of the Micron Technologies datasheet

we needed to implement. The datasheet provides timing information for many special cases of writingand reading. We also had to consider whether we needed to refresh memory periodically to ensure datawouldn’t decay. We decided that we only needed to implement basic reading and writing in bursts intoSDRAM, without consideration for being interrupted in the middle of a read/write operation (this wasensured by the arbiter described later). We also concluded that we did not need any special logic to handlerefreshing memory, since we constantly read all of the data we need from memory and reading from memoryautomatically refreshes the data.

Figure 2 shows the state machine we designed for the SDRAM controller. This section describes how wedesigned the state machine using the SDRAM datasheet.

1.1 Initialize and Load Mode Register

Our SDRAM control state machine follows the specifications in the timing diagrams provided in the MicronMT48LC16M16A2 datasheet. Figure 3 shows the timing diagram that corresponds to the “Initialize andLoad Mode Register” group of states labeled in our state machine. We follow the same progression of states,first allowing for 100µs of time for initialization while issuing command NOP, then running PRECHARGE,two AUTO REFRESH commands (spaced with NOPs to meet timing constraints), then finally performingthe LOAD MODE REGISTER command. The state machine diagram also indicates (in the upper right)the parameters we are using when asserting LOAD MODE REGISTER.

You can see in this portion a state which we call the “Ready” state. After any read or write request hascompleted, we return to this state and await the next read or write request.

1.2 Read Request

Upon receiving a read request from the “Ready” state, we transition into the portion of our state machinethat handles SDRAM reads. The first command we must assert is ACTIVE, so we go to the “Active Read”state which asserts that command. Since the black box we are given is guaranteed to read and write inbursts of 8, we don’t have to worry about any special cases provided in the datasheet such as interruptedreads and writes. Thus, the states involved in read operation simply have to assert the READ command,then wait 1 clock cycle (asserting NOP) for the CAS latency, then assert 8 NOP commands to grab the 8segments of data in the read burst. On the last NOP, we have to set Done high in our module, at which timewe transition into a “Done” state, from which we can proceed to “Ready”, “Read”, or “Write”, dependingon the subsequent input.

Figure 4 shows the timing diagram corresponding to the portion of the state machine that is labeled“Read”. Note that the NOP that follows the READ assertion at edge T1 of the clock is omitted becauseit is unnecessary to meet timing constraints. We also tie DQML and DQMH to zero, since we never needto mask the read data. Since we’re using the Auto Precharge feature of the SDRAM, we’re referencing theappropriate timing diagram and do not need to assert PRECHARGE when reading or writing.

1.3 Write Request

Upon receiving a write request from the “Ready” state, we transition to the portion of our state machinethat handles SDRAM writes. This is very similar in nature to the portion that handles reads, except wemust pass the Mask input into RAM DQML and RAM DQMH. The rest follows the same logical flow asthe handling of the Read request (but instead of there being an initial CAS latency, we must wait an extra2 cycles at the end to meet the timing specification). The timing diagram we refer to here is the “Write —DQM Operation” diagram.

1.4 SDRAM Arbiter

The SDRAM arbiter is the interface through which all other modules access SDRAM. This piece is essentialbecause only one read/write operation can occur at any given time. The SDRAM arbiter has to ensure that

Page 6: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

1 SDRAM CONTROLLER AND ARBITER 6

InitializeNOP

PrechargePRECHARGE

Idle 0AUTO

REFRESH

Clock

Idle 1NOP

Clock

Idle 2AUTO

REFRESH

Clock

Idle 3NOP

Clock Clock

Program

LOAD MODE

REGISTER

*1

Clock && Count < 2700 Count = Count + 1

*1. Burst Length: M2, M1, M0 = 0, 1, 1

Burst Type: M3 = 0

CAS Latency: M6, M5, M4 = 0, 1, 0

Op Mode: M8, M7 = 0, 0

Burst Mode: M9 = 0

Reserved: M12, M11, M10 = 0, 0, 0

*2. DQM short for DQML, DQMH

*3. A[12:0} = Input Address

B[1:0] = Input Bank

Clock

Ready

NOP

Ready = 1

Done = 0

Power-up

Count = 0

Active Read

ACTIVE

Done = 0

*3

Active Write

ACTIVE

Done = 0

*3

Clock && WriteRequest

Clock && ReadRequest

Clock && Count >= 2700

Initialize and Load Mode Register

Read

READ

DQM = 0

A10 = 1

Read Burst

NOP

Clock

Clock

Count = 0

Clock && Count < 7

Count = Count + 1

Write

WRITE

DQM = Mask

A10 = 1

Write Burst

NOP

Clock

Clock

Count = 0

Clock && Count < 5

Count = Count + 1

Read Write

Read Done

NOP

Done = 1

Clock && Count >= 5Clock && Count >= 7

Clock && ReadRequest

Clock && WriteRequest

Write Done

NOP

Done = 1

Clock && WriteRequest

Clock &&

!(ReadRequest || WriteRequest)Clock &&

!(ReadRequest || WriteRequest)

Clock && ReadRequest

Figure 2: SDRAM controller state machine

only one such operation occurs at any given time, and that the operation completes before moving on toany other operation. Further, the SDRAM arbiter is also central in ensuring that all of the ports are servedsufficiently so that starvation never occurs—this could result in poor performance or overflow/underflow ofFIFOs, resulting in lost data.

Figure 6 shows part of the arbiter state machine. This diagram shows only two of the five ports we used,

Page 7: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

1 SDRAM CONTROLLER AND ARBITER 7

Figure 3: Initialize and Load Mode Register

Figure 4: Read with Auto Precharge

Figure 5: Write — DQM Operation

but the extension to five ports from this diagram is trivial (and showing all five ports would be cumbersome).The two ports shown are the local full video read/write ports, which are the VE Processor and VD Processor.Note that when servicing the VE Processor, the SDRAM arbiter goes through a sequence of states allowingthe VE Processor to read data from SDRAM, while when sevicing the VD Processor, the arbiter goes througha sequence allowing the VD Processor to write data to SDRAM. Since all of the ports involve either readingor writing, they can all be serviced in a similar fashion by the arbiter (which is why extending these twoports to our five ports in this state machine is so simple). The ports that follow the read model are VEProcessor and the sender. The ports that follow the write model are the VD Processor, compressor, and thereceiver. Figure 7 shows how the ports connect to the arbiter.

When servicing a port that reads from SDRAM, the arbiter can rely on the DataValid and Done signalsfrom SDRAM to determine when it has finished servicing the port. This is because we’ve designed our portsto only read or write a single burst of data at a time. When servicing a write port, however, the arbitermust keep track of timing itself, so we utilize a counter to determine when the write has finished. This isa suitable method since, having designed the SDRAM controller as well, we know exactly how many clockcycles it requires to finish writing a burst.

Page 8: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

2 VIDEO CAPTURE AND DISPLAY 8

Figure 6: SDRAM arbiter state machine (two ports only)

2 Video Capture and Display

The video capture and display is encapsulated by two modules: the video encoder and the video decoder.The video decoder was provided to us, so here we will only discuss the video encoder. The video encoderand decoder conform to the ITU 601 specification for video data.

2.1 Video Encoder

Figure 8 shows the block diagram that implements the functionality of the video encoder. Each componentwe’ve designed will be discussed individually.

2.1.1 Counter

The counter we’ve implemented simply generates the HCount and VCount signals. It iterates HCount from 0to 429 (half the number of pixels per line, since we deal with pixel pairs) and it iterates VCount from 0 to524. The signals are only changed once every four clock cycles, so we keep track of the number of clock cyclesthat have passed with a modulo 4 counter value called WCount. The basics of our logic is that if WCount is3, then we increment HCount modulo 430. If HCount wraps to zero, then we increment VCount modulo 525.

Page 9: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

2 VIDEO CAPTURE AND DISPLAY 9

Figure 7: Arbiter-port interface

2.1.2 Counter Controlled State Machine

The second attached figure shows the state transition diagram for our counter controlled state machine.This state machine produces 4 bits of output encoding the state we’re in, which is passed into our datagenerator. The first two bits represent our horizontal state (either EAV, SAV, Horizontal Blanking, orHorizontal Active), while the second two bits represent our vertical state (either Vertical Blanking Odd,Vertical Blanking Even, Vertical Odd Active, or Vertical Even Active). Since each of these states has aunique output, we use these signals to select which output our data generator should pass to the dataregister.

Additionally, this state machine generates requests to the Test ROM as to what data we should request(which will then be delivered to the data generator). When in the states representing Active Even operation,it requests lines 1, 3, 5, etc. from the ROM, while when in the states representing Active Odd operation, itrequests lines 0, 2, 5, etc. from the ROM (these values are computed from HCount and VCount). The finaloutput bit produced by the state machine is an enable bit for the register in the data feeder section. It setsthis bit high once very four clock cycles, when the data needs to be buffered into the register.

It uses the I2CDone signal to determine when it should reset the counters in the Count module (themodule that produces HCount and VCount) so that it cycles through the states in the correct order once theADV7194 chip is ready for data.

This state machine can be implemented using combinational logic acting on the HCount and VCount

inputs. We simply test HCount to see whether it is 0 (EAV), between 1 and 67 (Horizontal Blanking), 68(SAV), or between 69 and 428 (Horizontal Active), then assign a unique number between 0 and 3 to theHState (Horizontal State) output.

For the vertical state, we test whether VCount is between 0 and 2 or between 265 and 271 (VerticalBlanking Even), between 3 and 8 or between 263 and 264 (Vertical Blanking Odd), between 9 and 262(Vertical Active Odd), or between 272 and 524 (Vertical Active Even), then assign a unique number between0 and 3 to VState.

Page 10: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

2 VIDEO CAPTURE AND DISPLAY 10

Figure 8: Video encoder block diagram

Table 1 summarizes this information (square brackets denote inclusive intervals).

HCount 0 [1,67] 68 [69,428]VCount [0,2] or [265,271] [3,8] or [263,264] [9,262] [272,524]HState EAV Horizontal Blanking SAV Horizontal ActiveVState Vertical Blanking Even Vertical Blanking Odd Vertical Active Odd Vertical Active Even

Table 1: Logic for the State Machine

2.1.3 Data Generator

The data generator is a module that generates data for us and then allows us to pick which data to outputbased on the state information passed in by the state machine module (HState and VState). Based onHState and VState, we can use Table 2 to decide what information we need to output.

VState\HState 0 1 2 30 EAV (H,F,V) = (1,1,1) Blank SAV (H,F,V) = (0,1,1) Blank1 EAV (H,F,V) = (1,0,1) Blank SAV (H,F,V) = (0,0,1) Blank2 EAV (H,F,V) = (1,0,0) Blank SAV (H,F,V) = (0,0,0) ROM Data3 EAV (H,F,V) = (1,1,0) Blank SAV (H,F,V) = (0,1,0) ROM Data

Table 2: Logic for the Data Generator

2.1.4 Data Clipper

The data clipper ensures that the values we send to the ADV7194 are between 0x10 and 0xF0. First, wehave to split the 32-bit input data into four sets of 8-bit data. We clip each of these in parallel so that

Page 11: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

2 VIDEO CAPTURE AND DISPLAY 11

they all are between 0x10 and 0xF0, then re-combine them to form a 32-bit output for which each 8 bits isclipped. We actually create a separate module to clip 8 bits of data, then use that in the creation of themodule that clips all 32 bits of data.

2.1.5 Data Feeder

Data exits the data generator in 32-bit wide blocks. Since ROM data is only available for one clock cycle, wemust store all 32 bits and then time-multiplex it in accordance with the established protocol. Implementation-wise, this means the data generator feeds into a data register whose enable bit is high once every four clockcycles (as controlled by our state machine). Every four clock ticks, a new set of 32 bits is loaded. The dataexits the register along a four-way splitter, and these four 8-bit wide packets are sequentially fed to the IOReg via a multiplexer controlled by WCount. In addition to selecting signals, the multiplexer also pads the8-bit packets with two zeroes on the lower order bits, making them valid 10-bit NTSC packets.

2.2 Video Ports

In addition to the video encoder and video decoder, we also need a way to interface the encoder and decoderwith SDRAM. This was implemented in the form of ports controlled by the SDRAM arbiter. Here we’lldescribe the ports interfacing with the video encoder and decoder, namely the full local video read/writeports and the compressed local video write port.

2.2.1 VD Processor

The VD processor is the port responsible for writing data from the camera into SDRAM. Its functionalityis extremely simple: it consists of an address counter (to keep track of where to write the data), a FIFO (tobuffer the data), and just a few control signals so it can interface with the arbiter. The block diagram isshown in Figure 9.

When the FIFO’s DataCount signal indicates that the FIFO has at least one burst of data in it, thecomparator asserts the VD Ready signal, which tells the arbiter it is ready to be serviced. This ensures thatthe FIFO is kept as empty as possible at all times, which helps prevent possible overflow as data from thecamera needs to be constantly written.

2.2.2 VE Processor

The VE processor is the port responsible for reading data from SDRAM and sending it to the video encoder,which subsequently displays it on the monitor. Its functionality mirrors that of the VD processor in that ithas a FIFO to buffer data, a comparator to signal when it is ready based on how much data is in the FIFO,and an address counter to indicate where it should read data from in SDRAM. The block diagram for VEprocessor is shown in Figure 10.

When the FIFO’s DataCount signal indicates that the FIFO is missing at least one burst of data, thecomparator asserts VE Ready. This ensures that the VE processor FIFO is kept constantly full, which helpsprevent underflow as data needs to be sent to the monitor constantly. One important difference between theVE processor and VD processor arises in their address counters. The VD processor simply writes a frameof data into Bank 0 of the SDRAM as a rectangle corresponding to pixel locations on the screen. The VEprocessor does this for the most part, except when reading the upper left and upper right corners of thescreen, it reads from Bank 1. The reason for this difference is that the compressed data that is written bythe other ports (namely the compressor and the receiver) is written to Bank 1 in those very locations. Thisessentially allows the compressed data to be overlaid upon the uncompressed local video.

2.2.3 Compressor

The compressor module serves two main purposes. The first is to sample the data in a way that is convenientfor the communications module to send. The second is to write a representation of this data to the locationthat the video display portion of the graphics module expects it to be. Compressor operates in parallel withVDProcessor. Its basic components are a 32-bit wide, 4-word long shift register with Q ports coming out

Page 12: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 12

Figure 9: VD processor block diagram

of each stage, an asynchronous FIFO (just like the one in VDProcessor), some counters, and a small statemachine. Figure 11 shows the block diagram structure of the compressor module as well as the simple statemachine used to control the module.

Compressor operates as follows: the shift register’s input port is connected to the video in from decoderand the enable line is connected to the valid signal from decoder. It is synchronized with the camera clock.The four outputs of the shift register are fed through an averager, the output of which provides us with afour point moving average of the video data (QAverage). QAverage is then fed into the DataIn port of theasynchronous FIFO. The write enable input of the FIFO is high in discreet active sessions. An externalrequest signal indicates that we should begin a session. At the start of the next video frame (first odd linepixel of the screen), the session begins. WriteEnable is timed to be high on every fourth decoder valid pulsefor the length of a line (360 pulses) for every other odd line (every other line from 0 to 253). Then the sessionis done and compressor will not be active again until the next request signal.

Interfacing with the compressor module is identical to interfacing with VDProcessor. When the FIFOin compressor has at least one burst of data, it asserts Ready and the request is handled in the exact sameway by SDRAMArbiter. Compressor contains CompressedAddressCounter, which behaves in the same wasas AddressCounter, but with an address domain corresponding to the box in the top left corner. With justa bit of pruning, the data stored in bank 1 is ready for packing and transmitting.

3 Wireless Communications

The wireless communications was handled by a transceiver module we created to interface with the serialinterface to the wireless chip. We also utilized two more ports controlled by the arbiter: one to handlereading compressed video data from SDRAM to send wirelessly; the other to handle writing compressedvideo data to SDRAM when it is received by the transceiver. Finally, we had to write a communications

Page 13: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 13

Figure 10: VE processor block diagram

module to handle handshaking and controlling the overall behavior of the system.

3.1 Transceiver

Our transceiver module is split between the main transceiver state machine and what we call the SPIcontroller, whose job is to interface directly with the SPI. The reason we split the module in this fashion isto hide the process of communicating with the SPI from the transceiver to simplify their interaction. Themain transceiver state machine is able to send commands into the SPI controller which the SPI controller willcommunicate to the SPI module. Once the command has been executed, the SPI controller has an outputto the transceiver state machine to indicate that the command has been executed, allowing the transceiverstate machine to continue to its next state. This relationship is detailed in Figure 12, a block diagram forthe transceiver.

3.1.1 Transceiver State Machine

The transceiver state machine consists of three primary sections: initialize, receive, and transmit. Each ofthese sections follows the specification outlined in the documentation for SPI interface. However, we’ve madea couple of notable changes to the design aside from just attaching these state machines together. Figure 13shows the initialization and receiving portion of the state machine (note that each command state is actuallyimplemented as two states—one to assert the command, the other to wait for the CommandDone signalfrom the SPI controller). Figure 14 shows the transmission portion of the state machine.

In addition to the shown state machine, we also have special transitions to handle overflow. If weexperience overflow in particular states, we jump to a set of states that will flush the FIFOs (not shown in

Page 14: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 14

Figure 11: Compressor block diagram and state machine

the state machine). In order to accomodate this jumping around, we also have the ability to jump back towhere we came from by storing where to return to in an extra register. This ensures that if we overflowduring a transmission and have to leave the transmit states, that we don’t just forget to send that packet.We instead will jump back to the transmit states after flushing the FIFO and complete that transmission.

Page 15: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 15

Figure 12: Transceiver block diagram

3.1.2 SPI Controller

The SPI controller is simply our abstracted interface to the SPI. It receives commands from the transceiverstate machine, then issues those commands with appropriate timing to the SPI. It takes in a command, acommand type, and any necessary data it needs in order to execute the command. Its state machine isomitted since it simply conforms to the timing specification of the SPI. The timing diagram demonstratingthis specification is shown in Figure 15.

3.2 Communications Ports

3.2.1 Recieving Processor (RR Processor)

Figure 16 shows the block diagram structure of the RRProcessor. RRProcessor operates in much the samemanner that VDProcessor does. The main difference it that there is only one clock, so it is not necessary tohave an asynchronous FIFO. RRProcessor waits for the Communications module to signal that a new datapacket has arrived. It then iterates through the data portion serially, five bits at a time and constructs anNTFC pixel pair by treating the five bits as the top of both eight-bit luminance values. The luminancesare padded with zeroes and the chrominance are arbitrarily set to 8’h80 to create a grayscale image. Eachreconstructed pixel pair is placed in order into a FIFO queue. In parallel to this FIFO is another one whichstores the addresses of the video data. The address is decoded by analyzing the header and line informationfrom the received packet. It is important to note that every packet contains two bursts worth of videodata so two consecutive burst locations must be constructed for each packet. The SDRAM Arbiter treatsRRProcessor just as it treats VCProcessor, which asserts the Ready signal when the data FIFO has reached25% capacity. The CountEnable signal that incremented VDProcessor’s address counter now pops the top

Page 16: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 16

Figure 13: Transceiver state machine (initialization and receiving)

address off of the address FIFO. The only modification needed to handle RRProcessor correctly is that nowCountEnable must be asserted before writing data to SDRAM in contrast with after, as was the case withVDProcessor.

3.2.2 Sending Processor (SS Processor)

The purpose of the SSProcessor module is to package compressed data into a form that is appropriate forsending over wireless and to present it to the communications module. SSProcessor operates in discreetsessions. The beginning of a session is triggered by a signal from the Communications module. WhenCommunications tells SSProcessor that a new packet of data is needed, SSProcessor raises its request flagto the SDRAM Arbiter. The Arbiter treats this request in the exact same way that it treats a requestfrom VEProcessor. SSProcessor has its own address counter that references the data in bank one thatCompressor writes to. This data is fed into a shift register, with the top five bits of one averaged luminance

Page 17: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 17

Figure 14: Transceiver state machine (transmitting)

Figure 15: SPI timing diagrams

value representing a black and white pixel pair. The line and column of each pixel pair can be extracted fromthe current address and this is used to form the header and line of each packet. When two complete bursts of

Page 18: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 18

Figure 16: RR processor block diagram

data have been extracted from RAM, the active session of SSProcessor ends. A valid signal goes high and theshift register containing all of the data can be viewed in parallel by the Communications module, with thedata guaranteed to be good until the next active session, that is, until the next time that Communicationsrequests a packet. SSProcessor has one final responsibility in signaling the Compressor module to writea new compressed frame of data whenever SSProcessor’s address counter reaches its final position beforewrapping to zero.

3.3 Communications Protocol

Finally, we tied together all of the communications with a state machine governing the communications pro-tocol for this project. This state machine handled toggling between master and slave, the initial handshakingbetween the master and slave, and the eventual back-and-forth communications process that allowed thevideo conferencing to work. The state machine is rather straightforward, but we’ve split it between Master

Page 19: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

3 WIRELESS COMMUNICATIONS 19

functionality and Slave functionality to discuss it more easily.

3.3.1 Master

Figure 17 shows the initialization (which both Master and Slave conduct) and the behavior of the Masterfollowing initialization. Immediately after reset, the state machine waits for the user to select either Masteror Slave functionality. If it is set as the master, it sends a MasterSeek packet and waits for a SlaveAckpacket. Upon receiving the SlaveAck, it sends a MasterReady packet and waits for a SlaveReady packet.Once it receives the SlaveReady packet, the handshake is complete and initialization is over.

Figure 17: Communications Master state machine

From here, the Master and Slave act very similarly in their exchange of packets. First, the Master sendsa packet, then it waits to receive a packet from the Slave. They exchange frames in this manner until thesystem is reset or communications is lost. We determine if communications is lost based on how long it hasbeen since we have last received a packet. If we time out too much (meaning too many packets are lost),

Page 20: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

4 USER INTERFACE 20

then we go to a Dead state where we inform the user that communications has been lost, requiring a resetof the system.

3.3.2 Slave

The Slave’s behavior fits in symmetrically with that of the Master. It initially waits to receive a MasterSeekpacket, then sends back a SlaveAck and waits for a MasterReady packet. Once it receives a MasterReadypacket, it sends SlaveReady and then waits for the first data packet to arrive. Then, the behavior followsthat of the Master in that it first waits for a packet, then sends a packet. This back-and-forth continuesuntil a reset or until communications is lost, as described previously. Figure 18 shows the state machine thatgoverns the Slave. In reality, the Master and Slave state machines are connected together and have onlybeen split here to illustrate different logical functionality.

Figure 18: Communications Slave state machine

4 User Interface

The user interface was rather simple. We created a text module (which utilized a provided character ROM)in order to print useful information on the screen about the status of the video conference.

4.1 Text Module

The text module simply masks off a portion of the screen and replaces it with text. It does so by keeping trackof where the VE Processor is reading from memory. If the location in memory corresponds to a portion ofthe screen that should contain text, it grabs data from the character ROM instead of from memory and slidesthat in place of the video data. Our choice of the interface was rather simple: we used a fixed 40-characterline on the screen and printed all of our status information in that line. Simplicity took precendence overaesthetics since time was a factor in completing the project. We label whether we’re a Master or Slave

Page 21: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

21

and indicate whether we’re handshaking (including which packet we’re sending, i.e. MasterSeek, SlaveAck,MasterReady, or SlaveReady), sending video packets, or if there is a communications failure. Further, weindicate the channel, sender, and recipient on screen as well. No further enhancements to the interface weremade beyond this simple functionality.

Part III

Conclusion

5 Design Metrics

Table 3 shows the design metrics for our project. Table 4 show the design and debugging time estimates foreach portion of the project (double times for total man-hours).

Logic Utilization Used Available UtilizationNumber of Slice Flip Flops 3,398 38,400 8%Number of 4 input LUTs 3,400 38,400 8%

Logic DistributionNumber of occupied Slices 3,169 19,200 18%

Total Number of 4 input LUTs 4,179 38,400 10%Number used as logic 3,400Number used as route-thru 471Number used as 16x1 ROMs 44Number used as Shift registers 264

Table 3: Design metrics

Checkpoint Design Time Coding Time Debugging Time Total Time0 6 hrs 3 hrs 7 hrs 16 hrs1 3 hrs 4 hrs 8 hrs 15 hrs2 6 hrs 5 hrs 10 hrs 21 hrs3 6 hrs 4 hrs 8 hrs 18 hrs4 8 hrs 15 hrs 20 hrs 43 hrsTotal 29 hrs 31 hrs 53 hrs 113 hrs

Table 4: Design and debugging time estimates

6 References

Aside from the provided Counter, Register, FIFOs, and FPGA TOP files, we did not obtain code from anyother sources. We did share information about debugging with other students throughout the semester, butthey are too many to name. The communal nature of the lab environment encouraged helping students thatwere having problems at many points, so we helped and received help from many along the way.

7 Conclusion

We’ve acheived, over the course of a semester, a working implementation of a simple video conferencingsystem. Though lacking audio and color, we have a two-way wireless system in which two people can seeeach other on monitors across distances measured in tens of feet. In compiling this document, it becomes clearhow truly massive this project was in scope. Along the way, we’ve learned how monitors handle video with

Page 22: CS150 Spring 2007 Wireless Video Conferencing …Wireless Video Conferencing Final Project Report Eudean Sun Nan Yu April 29, 2007 Contents I Overview 3 II System Description 3 1 SDRAM

8 SUGGESTIONS 22

luminance and chrominance values. We’ve learned how to interface with a physical SDRAM module in termsof timing. We’ve learned how to decompose extremely dense datasheets and timing diagrams into functionalstate machines and datapaths. We’ve learned how to communicate with a wireless chip and handshake acrossa wireless channel where packets may be lost or implementations may be running at different speeds.

We hit many road bumps along the way. The design of the SDRAM arbiter started as a complex pollingsystem that interacted with state machines within each port. In the end, we realized that some ports coulddo entirely without state machines, and only after we simplified our design and our code did everythingwork. When designing the wireless transceiver, we hit many road bumps as to how the state machine shouldlook. Figuring out when we could flush was rather complicated and only after debugging for quite some timedid it become apparent that our initial implementation of flushing FIFOs would not work.

To ask what we would do differently next time is an interesting question, because we completed theproject by the deadline—why risk changing anything if what we did worked? On the other hand, it isclear that we could have worked more prudently and efficiently. The most important change I would makewould be always reserving extra time to complete any intended feature of the project. Coming into labassuming it will take X hours to implement something is generally a bad idea if you only have X hours towork. Expecting it to take X hours, then giving yourself 1.5X or more hours is a wiser choice, since almostinevitably it won’t work the first time, or the second time, or the third time. Time management is perhapsthe most essential key to completing the project. The project isn’t difficult, only time consuming (thoughsome would call it difficult simply because it is time consuming), and that is perhaps the most importantfact to know heading into the project—being smart doesn’t help, putting time in does.

8 Suggestions

Nothing in this project was too difficult. I would say that it was perfectly reasonable to expect completionby the deadline, and considering how many groups finished by the early deadline, I would say that thenormal deadline should be treated as a safety by most rather than a normal deadline (i.e. all groups shouldaim for the early deadline because it is perfectly feasible to meet that deadline). There are a number ofrecommendations I would make nonetheless.

First, teaching us how to make FIFOs earlier on would have been extremely valuable. The asynchronousFIFO we were given for Checkpoint 2 was ridiculously large and slowed our debugging tremendously. I canonly imagine how many times we synthesized the project with a FIFO that tacked on 5 minutes or more tothe synthesis time.

Second, the specifications for the wireless transceiver were bad. When coding Checkpoint 3, we felt itwas rather easy. We designed our state machine, coded it, debugged it, and had it working in no time.However, once we got to Checkpoint 4 and everything sped up, it was absolutely necessary to know certainlittle details about the transceiver state machine in order to get correct functionality. We weren’t the onlygroup that experienced such troubles—at least two others we discussed this with also had issues with thewireless transceiver that only manifested themselves in Checkpoint 4. If a truly proper testing scheme weredevised for the wireless transceiver, or if certain details of its functionality were better specified, many hoursof needless debugging could be avoided entirely.

Lastly, we feel that more help needs to be provided initially. For example, almost no specifications weregiven for the design review at the beginning of the project and many groups went into the design reviewunprepared, only to be unfairly docked points. Those in later lab sections who heard about this were givenan unfair advantage as they were able to prepare based on knowledge available only after the initial groupswere told their design documents were insufficient (incidentally we were one of the groups given this unfairadvantage, but even so I’d like to complain for those that were cheated of those points).


Recommended