Reliable Storage using Reed-Solomon coding

Winter 2004/2005 Part B Final Presentation

Ilan Rosenfeld & Moshe KarlInstructor: Isaschar Walter

Overview

A problem: storage device

Setting is in space

Cosmic radiation is in abundance (no atmosphere)

Incident radiation upon storage

device may cause bit-flips

Valuable information might be lost

Possible Solutions

• Physical level– Thickening of shielding materials (problem -

weight)– Further separation of logic level voltages

(problem - energy)

• System level– Unit redundancy (e.g. majority vote)

• Logical level– Data redundancy with error-correcting coding

Solution: Reed-Solomon Coding

Reed Solomon Encoder

Reed Solomon Decoder

Storage Device

input data(possibly)corrupted

data

(hopefully)corrected

data

Other parameters

• Data is to be stored/retrieved at high rates (gigabits per second)

• The system must be as generic as possible in regard to the required data rate

Defined Project Goals

Building a system that is capable of:• Receiving raw data at high rates (gigabits per second)• Encoding of data in real time with Reed Solomon code

and sending it to a storage device• Retrieving the possibly corrupt data from the storage

device and decoding it.• All stages using minimum CPU resources.

In addition, a storage system capable of:• Receiving and transmitting data at such high rates.• Customizable error insertion.

Project Part A Goals

• Studying the system-on-chip design process.• Studying the EDK environment.• Studying the blocks that build our system:

– PowerPC– Reed Solomon Encoder/Decoder.– PLB IPIF and bus transaction protocol.– RocketIO

• Implementing the Reed Solomon cores as slaves on the bus (CPU still very involved in the flow).

• Sending a packet through RocketIO Reference Design

PLB

PowerPC

RSDecoder

RS Encoder

PLB2OPBBridge

OPB

UART

HyperTerminalon

DIGLABPC

IPIF

FIFO

IPIFIPIF

Part A Architecture

Part B Goals

Implementation of:

• Multigigabit data accelerator for data source.

• Real-time, generic Reed-Solomon encoding/decoding system

• Generic serial storage device

• Demonstration system

The Design

Design Resources

• Two Memec FF1152 development boards with Virtex II Pro XC2VP30 chip

• XC2VP30 includes 2 PowerPC processors, and more logic and memory than older chips we used

Data Flow

PC PowerPCGenericBRAM

Generic MGTTX

Generic MGTRX

RSEncoding

Serial Storage

Generic MGTTX

Generic MGTRX

RSDecoding

RS232 - UART

RocketIOSMA

RocketIOSMA

Data Accelerator

PLBIPIF

PLBIPIF

Generic Design

• The data flow consists of N multigigabit channels

• Each channel is independent in resources• All blocks in the multigagbit flow are

generically constructed with varying data widths (and resource usage)

• The parameter “N” is decided in system design stage (EDK level), prior to synthesis.

Performance limits per channel• Clock rate for encoding and decoding is set by

the MGT clock.• MGT may work at

– 62.5 MHz or 78.125 MHz (32-bit data path)– 125 MHz or 156.25 MHz (16-bit data path)– 250 MHz or 312.5 MHz (8-bit data path)

• 32-bit is too wide (too much logic)• 8-bit is much too fast for our blocks• Decoder cannot work at 156.25 MHZ

Thus a 125 MHz / 16-bit scheme is chosen.

Performance limits per channel

• With a 125 MHz / 16-bit scheme, data can be transmitted only up to RocketIO MGT limit – 2 gbps.

• RS Encoder produces 16 new symbols per 239 input symbols.

• Therefore a rough limit on data rate per channel is

2 gbps * 239/255 = 1.875 gbps

Design Blocks

Data flow revisited

PC PowerPCGenericBRAM

Generic MGT

TX

Generic MGTRX

RSEncodingSerial St

orage

Generic MGTTX

Generic MGTRX

RSDecoding

RS232 - UART

RocketIOSMA

RocketIOSMA

DataAccelerator

PLBIPIF

PLBIPIF

UART

• Controller core on the OPB bus• Configured via generics to the following:

– Baudrate: 115200 bps– Data bits: 8– Parity: None– Stop bits: 1– Flow control: None

• PC is configured with the same parameters.• PowerPC program accesses this block with

simple input/output functions.

PowerPC

• A 32-bit RISC embedded processor inside the Virtex II Pro FPGA (XC2VP30 chip includes two PowerPC processors)

• Operates at a rate of 300 MHz and connects directly, as master to the PLB.

• The processor has a JTAG interface, allowing debugging from a PC on runtime.

• In our system, PowerPC is not involved directly in the multigigabit data flow, but only configures the various blocks.

PLB IPIF

• As its initials suggest, an interface between the bus and the IP.

• Takes care of the transaction protocols on the bus and simplifies access to the IP.

• Also enables special features such as S/W Resetting, User Logic address ranges, interrupts, Bursting, DMA support and more.

Generic BRAM

• We created BRAM blocks of various port widths and interfaced them via IPIF according to the N parameter.

PLB IPIF

BRAM x132-bit 16-bit

BRAM x232-bit 32-bit

BRAM xN

32-bit

N*16-bit

To PLBTo Data

Accelerator

Data Accelerator• The block is intended to read data from the

generic BRAM in consecutive clock cycles and output it towards MGT TX.

• Interfaces to PowerPC via PLB IPIF• Includes configurable registers for:

– Base address in BRAM for data– Max address in BRAM– Data rate limitation– Packet Sending indication

• Generics: No. of channels, BRAM address width.

RocketIO MGT

• RocketIO can transmit and receive serial data at rates ranging from 620 Mbps to 3.125 Gbps.

• At these rates, in order to assure sampling at correct times, each byte is coded into a 10 bit sequence with enough edges.

• The 10 bit scheme allows for special “K-Characters”, which are used for various tasks, including packet start and end signaling (SOP, EOP) and byte-synchronization (Comma).

Mindspeed Physical Media Attachment

Xilinx Coding Sublayer

RocketIO MGT Diagram

loopback

loopback

8B/10BEncode

FIFOTXDATA 16 bit SerializerTransmit

Buffer

TXN

TXP

8B/10BDecode

Elastic Buffer

RXDATA 16 bitDeserializerComma Detect

ReceiveBuffer

RXN

RXP

RocketIO MGT Configuration

• 8/10 coding enabled• CRC disabled• Comma is K28.5 (x“BC”)• 125 MHz clock rate, 16-bit data path

– Data rate: 125 MHz x 16 bit = 2 Gbps– Bit rate: 125 MHz x 16 bit x 10b/8b = 2.5 Gbps

• Many more parameters…

Generic MGT

• According to the N, the number of channels, the number of parallel MGT transceivers also varies

Generic MGT

MGT#1

MGT#2

MGT#N

RXDATA(0:15)TXDATA(0:15)

RXN(0:0), TXN(0:0)RXP(0:0), TXP(0:0)

RXDATA(0:31)TXDATA(0:31)RXDATA(0:16*N-1)TXDATA(0:16*N-1)

RXN(0:1), TXN(0:1)RXP(0:1), TXP(0:1)RXN(0:N-1), TXN(0:N-1)RXP(0:N-1), TXP(0:N-1)

The Encoding System

Asynch.FIFO

Encoding Unit

Outgoing packet delivery

unit

Pattern Generator

PLB Interface Control

PLB

FromMGT

To Serial Storage

or MGT

Asynchronous FIFO

• We have generated asynchronous FIFO of various depths and data-width of 16 bits for our data path

• In the encoding system, the input FIFO is used as a buffer for incoming data

• Control register chooses whether FIFO input is from MGT (“User Mode”) or PLB (“Bus Mode”)

Outgoing Packet Delivery Unit

• The unit’s input data may arrive from the input FIFO or the pattern generator (control register chooses)

• Data for encoding is drawn in groups of 239 symbols, and the unit waits for encoder indication of readiness

• Adds a start-of-packet symbol• Outputs idle comma characters when no packet

is being sent• May send packets according to PowerPC

request or automatically (control register chooses)

Reed Solomon cores• When creating the cores using Xilinx

CoreGen, the following parameters are needed: – k: number of symbols per data block (to be

encoded)– n: total number of output symbols (original data +

check symbols)– s: number of bits per symbol

The Reed Solomon code can detect n-k symbol errors and correct (n-k)/2.

We have chosen k=239, n=255, s=8, which are similar to G. 709 standard.

RS cores

RS Cores – Detailed

Encoding Unit

• For our 16-bit wide data path, 2 encoders are put in parallel and considered an encoding unit

• The two blocks are enveloped in a black box, which has inputs similar to a RocketIO transceiver

• TXDATA is encoded when required• TXCHARISK is delayed to the output at

the same latency of the encoders

Encoding Unit PLB Interface

• Accessing each channel of the encoding unit in the following:– Input Data– Control register read– Control register write– “Send” command in case of bus flow control

Generic Encoding System

From Generic MGT

PLB Interface

To Serial Storage or

Generic MGT

Serial Storage

• Interfaces like MGT (with DATA, CHARISK lines)

• Consists of the following:– FIFO for data storage– RX state machine– TX state machine– Data corruption unit

• Sends data according to a “send” signal

Decoding System

Incoming Packet

ReceivingUnit

Asynch. FIFO

Decoding Unit

PLB InterfaceStatus / Statistics

PLB

FromMGT

Incoming Packet Receiving Unit

• Stores incoming data in a buffer and releases it to the decoder once a full 255-symbol block has been gathered.

Decoding Unit

• Two generated 8-bit decoders are put in parallel and consist of one 16-bit decoder

• Since the decoder has a very long latency period, a secondary 16-bit decoder receives the traffic when the primary one is busy.

• Statistics are gathered in a register

Decoding Unit

16-bitRS Dec

8-bitRS Dec

8-bitRS Dec

16-bitRS Dec

8-bitRS Dec

8-bitRS Dec

From Incoming Packet Receiving Unit

Arbitration ArbitrationTo buffer

Generic Decoding System

PLB Interface

PLB

FromMGT

Additional raw data receiver

• On the 1st chip, also present is a raw data receiver which upon receiving of packet puts a copy of the data before decoding in a BRAM memory

MGT RX

BRAM

RS Decoding

Raw data receiver

PLBIPIF

PLB

Software

Accelerator and decoder boardPowerPC program

• Receives data via UART from PC

• Configures the data accelerator (base, max addresses, rate limit and send order)

• Reads decoded and raw returned data and sends it via UART to PC

Encoder and storage boardPowerPC program

• Configures the encoding system control parameters (data source, flow)

• Configures error insertion

Problems and Results

Problems

• RocketIO synchronization on startup

• Due to large decoder size, 3 decoding channels are already too much for one chip

• When performing PAR for a system with 2 decoders, the task failed to finish in reasonable time

• Therefore the decoding system consists of one channel only

System performance

• Data generation at 4x1.875 gbps = 7.5 gbps

• RS encoding at 7.5 gbps in / 8 gbps out

• RS decoding at 2 gbps in.

Demonstration Flow

PowerPCGenericBRAM

Generic MGTTX

Generic MGTRX

RSEncoding

Serial Storage

Generic MGTTX

Generic MGTRX

RSDecoding

RS232 - UART

RocketIOSMA

RocketIOSMA

PLBIPIF

PLBIPIF

PCHyperTerminal

PCHyperTerminal

Thank you!