ECE 6332, Spring, 2017

Wireless Communication

Zhu Han

Department of Electrical and Computer Engineering

Class 20

Apr. 5th, 2017

                                                           

                                                           

Convolutional Code IntroductionConvolutional Code Introduction Convolutional codes map information to code bits sequentially

by convolving a sequence of information bits with “generator” sequences

A convolutional encoder encodes K information bits to N>K code bits at one time step

Convolutional codes can be regarded as block codes for which the encoder has a certain structure such that we can express the encoding operation as convolution

                                                           

Encoder Encoder Convolutional codes are applied in applications that require good

performance with low implementation cost. They operate on code streams (not in blocks)

Convolution codes have memory that utilizes previous bits to encode or decode following bits (block codes are memoryless)

Convolutional codes achieve good performance by expanding their memory depth

Convolutional codes are denoted by (n,k,L), where L is code (or encoder) Memory depth (number of register stages)

Constraint length C=n(L+1) is defined as the number of encoded bits a message bit can influence to

                                                           

ExampleExample Convolutional encoder, k = 1, n = 2, L=2

– Convolutional encoder is a finite state machine (FSM) processing information bits in a serial manner

– Thus the generated code is a function of input and the state of the FSM– In this (n,k,L) = (2,1,2) encoder each message bit influences a span of

C= n(L+1)=6 successive output bits = constraint length C– Thus, for generation of n-bit output, we require n shift registers in k = 1

convolutional encoders

                                                           

ExampleExample (3,2,1) Convolutional encoder

3 2' j j j jx m m m

3 1'' j j j jx m m m

2''' j j jx m m

Here each message bit influences a span of C = n(L+1)=3(1+1)=6 successive output bits

                                                           

Generator sequencesGenerator sequences

                                                           

Convolution point of view in encoding and Convolution point of view in encoding and generator matrixgenerator matrix

3

                                                           

Example: Using generator matrixExample: Using generator matrix

(1)

( 2 )

[1 0 11][111 1]

gg

                                                           

Representing convolutional codes: Code treeRepresenting convolutional codes: Code tree

2 1

2

'

''j j j j

j j j

x m m m

x m m

1 1 2 2 3 3' '' ' '' ' '' ...outx x x x x x x

This tells how one input bitis transformed into two output bits(initially register is all zero)

2 1 0 1j jm m

' 1

''

0

1

1

0j

j

x

x

' 0

''

0

0

1

0j

j

x

x

(n,k,L) = (2,1,2) encoder

                                                           

Representing convolutional codes Representing convolutional codes compactly: code trellis and state diagramcompactly: code trellis and state diagram

Shift register states

Input state ‘1’ indicated by dashed line

Code trellis State diagram

                                                           

Inspecting state diagram: Structural Inspecting state diagram: Structural properties of convolutional codesproperties of convolutional codes

Each new block of k input bits causes a transition into new state Hence there are 2k branches leaving each state Assuming encoder zero initial state, encoded word for any input of k bits can

thus be obtained. For instance, below for u=(1 1 1 0 1), encoded word v=(1 1, 1 0, 0 1, 0 1, 1 1, 1 0, 1 1, 1 1) is produced:

- encoder state diagram for (n,k,L)=(2,1,2) code- note that the number of states is 2L+1 = 8

                                                           

Distance for some convolutional codesDistance for some convolutional codes Lower the coding rate, larger the L, then larger the distance

                                                           

Puncture CodePuncture Code A sequence of coded bits is punctured by deleting some of the

bits in the sequence according to some fixed rule. The resulting coding rate is increased. So a lower rate code can

be extended to a sequence of higher rate codes.

                                                           

Decoding of convolutional codesDecoding of convolutional codes

                                                           

Example of exhaustive maximal likelihood detectionExample of exhaustive maximal likelihood detection Assume a three bit message is transmitted [and encoded by (2,1,2)

convolutional encoder]. To clear the decoder, two zero-bits are appended after message. Thus 5 bits are encoded resulting 10 bits of code. Assume channel error probability is p = 0.1. After the channel 10,01,10,11,00 is produced (including some errors). What comes after the decoder, e.g. what was most likely the transmitted code and what were the respective message bits?

a

b

c

d

states

decoder outputsif this path is selected

                                                           

Note also the Hamming distances!

correct:1+1+2+2+2=8;8 ( 0.11) 0.88false:1+1+0+0+0=2;2 ( 2.30) 4.6total path metric: 5.48

The largest metric, verifythat you get the same result!

                                                           

The Viterbi algorithmThe Viterbi algorithm Problem of optimum decoding is to find the minimum distance path

from the initial state back to initial state (below from S0 to S0). The minimum distance is the sum of all path metrics

that is maximized by the correct path Exhaustive maximum likelihood

method must search all the paths in phase trellis (2k paths emerging/entering from 2 L+1 states for an (n,k,L) code)

The Viterbi algorithm gets itsefficiency via concentrating intosurvivor paths of the trellis

0ln ( , ) ln ( | )jm j mjp p y xy x

Received codesequence

Decoder’s output sequencefor the m:th path

                                                           

The survivor pathThe survivor path Assume for simplicity a convolutional code with k=1, and up to 2k = 2

branches can enter each state in trellis diagram Assume optimal path passes S. Metric comparison is done by adding the

metric of S into S1 and S2. At the survivor path the accumulated metric is naturally smaller (otherwise it could not be the optimum path)

For this reason the non-survived path canbe discarded -> all path alternatives need notto be considered

Note that in principle whole transmittedsequence must be received before decision.However, in practice storing of states for input length of 5L is quite adequate

2 branches enter each nodek

2 nodes, determined L

by memory depth

                                                           

Example of using the Viterbi algorithmExample of using the Viterbi algorithm Assume the received sequence is

and the (n,k,L)=(2,1,2) encoder shown below. Determine the Viterbi decoded output sequence!

01101111010001y

(Note that for this encoder code rate is 1/2 and memory depth L = 2)

states

                                                           

The maximum likelihood pathThe maximum likelihood path

The decoded ML code sequence is 11 10 10 11 00 00 00 whose Hamming distance to the received sequence is 4 and the respective decoded sequence is 1 1 0 0 0 0 0 (why?). Note that this is the minimum distance path.(Black circles denote the deleted branches, dashed lines: '1' was applied)

(1)

(1)

(0)

(2)

(1)

(1)

1

1

Smaller accumulated metric selected

First depth with two entries to the node

After register length L+1=3branch pattern begins to repeat

(Branch Hamming distancesin parenthesis)

                                                           

How to end-up decoding?How to end-up decoding? In the previous example it was assumed that the register was finally filled with

zeros thus finding the minimum distance path In practice with long code words zeroing requires feeding of long sequence of

zeros to the end of the message bits: this wastes channel capacity & introduces delay

To avoid this path memory truncation is applied:– Trace all the surviving paths to the

depth where they merge– Figure right shows a common point

at a memory depth J– J is a random variable whose applicable

magnitude shown in the figure (5L) has been experimentally tested fornegligible error rate increase

– Note that this also introduces thedelay of 5L!

5 stages of the trellisJ L

                                                           

Concepts to LearnConcepts to Learn You understand the differences between cyclic codes and

convolutional codes You can create state diagram for a convolutional encoder You know how to construct convolutional encoder circuits

based on knowing the generator sequences You can analyze code strengths based on known code

generation circuits / state diagrams or generator sequences You understand how to realize maximum likelihood

convolutional decoding by using exhaustive search You understand the principle of Viterbi decoding

                                                           

Viterbi AlgorithmViterbi Algorithm As a youth, Life Fellow Andrew Viterbi never envisioned that he’d create an algorithm used in

every cellphone or that he would cofound Qualcomm, a Fortune 500 company that is a worldwide leader in wireless technology.

Viterbi came up with the idea for that algorithm while he was an engineering professor at the University of California at Los Angeles (UCLA) and then at the University of California at San Diego (UCSD), in the 1960s. Today, the algorithm is used in digital cellphones and satellite receivers to transmit messages so they won’t be lost in noise. The result is a clear undamaged message thanks to a process called error correction coding. This algorithm is currently used in most cellphones.

“The algorithm was originally created for improving communication from space by being able to operate with a weak signal but today it has a multitude of applications,” Viterbi says.

For the algorithm, which carries his name, he was awarded this year’s Benjamin Franklin Medal in electrical engineering by the Franklin Institute in Philadelphia, one of the United States’ oldest centers of science education and development. The institute serves the public through its museum, outreach programs, and curatorial work. The medal, which Viterbi received in April, recognizes individuals who have benefited humanity, advanced science, and deepened the understanding of the universe. It also honors contributions in life sciences, physics, earth and environmental sciences, and computer and cognitive sciences.

Qualcomm wasn’t the first company Viterbi started. In the late 1960s, he and some professors from UCLA and UCSD founded Linkabit, which developed a video scrambling system called Videocipher for the fledgling cable network Home Box Office. The Videocipher encrypts a video signal so hackers who haven’t paid for the HBO service can’t obtain it.

Viterbi, who immigrated to the United States as a four-year-old refugee from facist Italy, left Linkabit to help start Qualcomm in 1985. One of the company’s first successes was OmniTracs, a two-way satellite communication system used by truckers to communicate from the road with their home offices. The system involves signal processing and an antenna with a directional control that moves as the truck moves so the antenna always faces the satellite. OmniTracs today is the transportation industry’s largest satellite-based commercial mobile system.

Another successful venture for the company was the creation of code-division multiple access (CDMA), which was introduced commercially in 1995 in cellphones and is still big today. CDMA is a “spread-spectrum” technology—which means it allows many users to occupy the same time and frequency allocations in a band or space. It assigns unique codes to each communication to differentiate it from others in the same spectrum.

Although Viterbi retired from Qualcomm as vice chairman and chief technical officer in 2000, he still keeps busy as the president of the Viterbi Group, a private investment company specializing in imaging technologies and biotechnology. He’s also professor emeritus of electrical engineering systems at UCSD and distinguished visiting professor at Technion-Israel Institute of Technology in Technion City, Haifa. In March he and his wife donated US $52 million to the University of Southern California in Los Angeles, the largest amount the school ever received from a single donor.

To honor his generosity, USC renamed its engineering school the Andrew and Erna Viterbi School of Engineering. It is one of four in the nation to house two active National Science Foundation–supported engineering research centers: the Integrated Media Systems Center (which focuses on multimedia and Internet research) and the Biomimetic Research Center (which studies the use of technology to mimic biological systems).

                                                           

Interleaving to get time diversity Interleaving to get time diversity

Block interleaver where source bits are read into columns and out as n-bit rows

Combat burst error. Block and convolutional interleaver

                                                           

Turbo CodesTurbo Codes Backgound

– Turbo codes were proposed by Berrou and Glavieux in the 1993 International Conference in Communications.

– Performance within 0.5 dB of the channel capacity limit for BPSK was demonstrated.

Features of turbo codes– Parallel concatenated coding – Recursive convolutional encoders– Pseudo-random interleaving– Iterative decoding

                                                           

Motivation: Performance of Turbo CodesMotivation: Performance of Turbo Codes Comparison:

– Rate 1/2 Codes.– K=5 turbo code.– K=14 convolutional

code.

Plot is from: – L. Perez, “Turbo Codes”,

chapter 8 of Trellis Coding by C. Schlegel. IEEE Press, 1997

Gain of almost 2 dB!

Theoretical Limit!

                                                           

Concatenated CodingConcatenated Coding A single error correction code does not always provide enough

error protection with reasonable complexity. Solution: Concatenate two (or more) codes

– This creates a much more powerful code.

Serial Concatenation (Forney, 1966)

OuterEncoder

BlockInterleaver

InnerEncoder

OuterDecoder

De-interleaver

InnerDecoder

Channel

                                                           

Parallel Concatenated CodesParallel Concatenated Codes Instead of concatenating in serial, codes can also be

concatenated in parallel. The original turbo code is a parallel concatenation of two

recursive systematic convolutional (RSC) codes.– systematic: one of the outputs is the input.

Encoder#1

Encoder#2In

terle

aver MUX

Input

ParityOutput

Systematic Output

                                                           

Pseudo-random InterleavingPseudo-random Interleaving The coding dilemma:

– Shannon showed that large block-length random codes achieve channel capacity.

– However, codes must have structure that permits decoding with reasonable complexity.

– Codes with structure don’t perform as well as random codes.– “Almost all codes are good, except those that we can think of.”

Solution:– Make the code appear random, while maintaining enough structure to

permit decoding.– This is the purpose of the pseudo-random interleaver.– Turbo codes possess random-like properties.– However, since the interleaving pattern is known, decoding is possible.

                                                           

Recursive Systematic Convolutional EncodingRecursive Systematic Convolutional Encoding

An RSC encoder can be constructed from a standard convolutional encoder by feeding back one of the outputs.

An RSC encoder has an infinite impulse response.

An arbitrary input will cause a “good” (high weight) output with high probability.

Some inputs will cause “bad” (low weight) outputs.

D D

im )0(ix

)1(ix

ix

ir

Constraint Length K= 3

D Dim

)0(ix

)1(ix

ix

                                                           

Why Interleaving and Recursive Encoding?Why Interleaving and Recursive Encoding? In a coded systems:

– Performance is dominated by low weight code words.

A “good” code: – will produce low weight outputs with very low probability.

An RSC code:– Produces low weight outputs with fairly low probability.– However, some inputs still cause low weight outputs.

Because of the interleaver:– The probability that both encoders have inputs that cause low

weight outputs is very low.– Therefore the parallel concatenation of both encoders will produce

a “good” code.

                                                           

Iterative DecodingIterative Decoding There is one decoder for each elementary encoder. Each decoder estimates the a posteriori probability (APP) of each

data bit. The APP’s are used as a priori information by the other decoder. Decoding continues for a set number of iterations.

– Performance generally improves from iteration to iteration, but follows a law of diminishing returns.

Decoder#1

Decoder#2

DeMUX

Interleaver

Interleaver

Deinterleaver

systematic data

paritydata

APP

APP

hard bitdecisions

                                                           

The Turbo-PrincipleThe Turbo-Principle Turbo codes get their name because the decoder uses feedback,

like a turbo engine.

                                                           

Performance as a Function of Number of IterationsPerformance as a Function of Number of Iterations

K=5, r=1/2, L=65,536

0.5 1 1.5 210

-7

10-6

10-5

10-4

10-3

10-2

10-1

100

Eb/No in dB

BE

R

1 iteration

2 iterations

3 iterations6 iterations

10 iterations

18 iterations

                                                           

Performance Factors and TradeoffsPerformance Factors and Tradeoffs Complexity vs. performance

– Decoding algorithm.– Number of iterations.– Encoder constraint length

Latency vs. performance– Frame size.

Spectral efficiency vs. performance– Overall code rate

Other factors– Interleaver design.– Puncture pattern.– Trellis termination.

                                                           

Influence of Interleaver SizeInfluence of Interleaver Size

0.5 1 1.5 2 2.510

-7

10-6

10-5

10-4

10-3

10-2

10-1

Eb/No in dB

BE

RL = 1,024 L = 4,096 L = 16,384L = 65,536

Voice

VideoConferencing

ReplayedVideo

Data

Constraint Length 5. Rate r = 1/2. Log-MAP decoding. 18 iterations. AWGN Channel.

                                                           

Power Efficiency of Existing StandardsPower Efficiency of Existing Standards

                                                           

Turbo Code SummaryTurbo Code Summary Turbo code advantages:

– Remarkable power efficiency in AWGN and flat-fading channels for moderately low BER.

– Deign tradeoffs suitable for delivery of multimedia services. Turbo code disadvantages:

– Long latency.– Poor performance at very low BER.– Because turbo codes operate at very low SNR, channel estimation

and tracking is a critical issue. The principle of iterative or “turbo” processing can be applied

to other problems.– Turbo-multiuser detection can improve performance of coded

multiple-access systems.

                                                           

LDPC IntroductionLDPC Introduction Low Density Parity Check (LDPC) History of LDPC codes

– Proposed by Gallager in his 1960 MIT Ph. D. dissertation– Rediscovered by MacKay and Richardson/Urbanke in 1999

Features of LDPC codes– Performance approaching Shannon limit– Good block error correcting performance– Suitable for parallel implementation

Advantages over turbo codes– LDPC do not require a long interleaver– LDPC’s error floor occurs at a lower BER– LDPC decoding is not trellis based

                                                           

Tanner Graph (1/2)Tanner Graph (1/2) Tanner Graph (A kind of bipartite graph)

– LDPC codes can be represented by a sparse bipartite graph Si: the message nodes (or called symbol nodes) Ci: the check nodes Because G is the null space of H, H ． xT = 0 According to the equation above, we can define some relation

between the message bits– Example

n = 7, k=3, J = 4, λ=1

H =1 1 1 0 0 0 0 00 0 0 1 1 1 0 01 0 0 1 0 0 1 00 1 0 0 1 0 0 1

s0 s1 s2 s3 s4 s5 s6 s7

c0 c1 c2 c3

Symbol nodes

Check nodes

                                                           

Decoding of LDPC CodesDecoding of LDPC Codes For linear block codes

– If c is a valid codeword, we have c HT = 0

– Else the decoder needs to find out error vector e Graph-based algorithms

– Sum-product algorithm for general graph-based codes– MAP (BCJR) algorithm for trellis graph-based codes– Message passing algorithm for bipartite graph-based codes

                                                           

Pro and ConPro and Con ADVANTAGES

– Near Capacity Performance: Shannon’s Limit – Some LDPC Codes perform better than Turbo Codes– Trellis diagrams for Long Turbo Codes become very complex and

computationally elaborate– Low Floor Error – Decoding in the Log Domain is quite fast.

DISADVANTAGES– Long time to Converge to Good Solution– Very Long Code Word Lengths for good Decoding Efficiency– Iterative Convergence is SLOW

Takes ~ 1000 iterations to converge under standard conditions.

– Due to the above reason transmission time increases

i.e. encoding, transmission and decoding– Hence Large Initial Latency

(4086,4608) LPDC codeword has a latency of almost 2 hours

                                                           

Fountain CodeFountain Code Sender sends a potentially limitless stream of encoded bits.

Receivers collect bits until they are reasonably sure that they can recover the content from the received bits, and send STOP feedback to sender.

Automatic adaptation: Receivers with larger loss rate need longer to receive the required information.

Want that each receiver is able to recover from the minimum possible amount of received data, and do this efficiently.

Originalcontent

Encoded packetsUsers reconstruct

Original content as soon as they receive

enough packets

Enc

odin

gE

ngin

e

Tra

nsm

issi

on