CUHK, May 2010
Department of Electrical Engineering
École Polytechnique de Montréal
Professor of Electrical EngineeringLife Fellow of IEEEFellow , Engineering Institute of Canada
David Haccoun, Eng., Ph.D.
Engineering training in CanadaEngineering training in Canada
36 schools/faculties
32
2 1
1311
1
12
Undergraduate students
Canada: 55,000
Québec: 14,600
2
MontréalMontréal
VancouverVancouver
TorontoToronto
3
École Polytechnique, École Polytechnique, cradle of engineering in Québeccradle of engineering in Québec
The oldest engineering school in Canada . The third-largest in Canada for teaching and research. The first in Québec for the student body size. Operating budget $85 million Canadian Dollars (C$). Annual research budget $60.5 million C$. Annual grants and research contracts $38 million C$. 15 Industrial Research Chairs. 24 Canada Research Chairs. 7863 scientific publications over the last decade. 220 professors, and 1,100 employees. 1,000 graduates per year, and 30,000 since 1873.
4
11 engineering programs11 engineering programs
Biomedical Civil Chemical Electrical
Geological Industrial Computer Software
Mechanical Mining Engineering
physics
Our campusOur campus
5
Polytechnique
Novel Iterative Decoding Novel Iterative Decoding Using Convolutional Doubly Orthogonal CodesUsing Convolutional Doubly Orthogonal Codes
A simple approach to capacityA simple approach to capacity
David HaccounDavid HaccounÉric Roy, Christian CardinalÉric Roy, Christian Cardinal
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
Modern Error Control Coding Techniques Modern Error Control Coding Techniques Based on Differences FamiliesBased on Differences Families
A new class of threshold decodable codes leading to simple and efficient error control schemes.
No interleaver, at neither encoding nor decoding
Far less complex to implement than turbo coding schemes, attractive alternatives to turbo coding at moderate Eb/N0 values
High rate codes readily obtained by puncturing technique
Low complexity and high speed FPGA-based prototypes at bit rate >100 Mbps.
Extensions to recursive codes Capacity
– Rate adaptive schemes Punctured Codes– Reduced latency Simplified Codes – Reduced complexity
7
8
Two Problems for Usual Turbo Coding
One Convolutional Encoder No Interleaver
An alternative using CSO2C
Low Decoding Complexity (Iterative Threshold Decoder)
Smaller Latency with Simplified Self-Doubly-Orthogonal Codes (S-CSO2C)
Decoding Complexity (MAP Decoder)
Latency due to Interleavers
Further improvements while maintaining good error performance
Half Latency with Iterative BP Decoder
MOTIVATION
0 1 2 m-1 m D1
=0
j
J1 =m
D2... D Dm
...
j... ... ... ...
}{ tu
}{ tp
Shift register of length m
Information sequenceInformation sequence
Parity sequenceParity sequence
}{ uty
}{ pty
J
}{ tuj
AWGN Channel
– Set of connection positions– Number of connection positions– Memory length– Coding span
A Jm
J,01 mJ
} , ... ,1 ,0 { mj
},,,{ 21 JA
One-Dimensional NCDO CodesOne-Dimensional NCDO Codes
Nonrecursive systematic convolutional (NSC) encoder Nonrecursive systematic convolutional (NSC) encoder ( R = 1/2 )
9
Simple orthogonal properties : CSOC
Differences are distinct
ExampleExample of Convolutional Self-Orthogonal Code of Convolutional Self-Orthogonal Code
CSOC, R=1/2, CSOC, R=1/2, J=J=4, 4, m=m=1515,, 15 ,13 ,3 ,0 A
10
01 32 133 154
}{ tu
}{ tp
}{ tu
+
Example of CSOC, J=4,
DistinctDistinct Simple DifferencesSimple Differences
0 3 13 15
0 -3 -13 -15
3 3 -10 -12
13 13 10 -2
15 15 12 2
15 ,13 ,3 ,0 A
j
All the simple differences are distinct CSOC codes are suitable for threshold decoding
11
Threshold (TH) Decoding of CSOCThreshold (TH) Decoding of CSOC
Well known symbol decoding technique that exploits the simply-orthogonal properties of CSOC
Either hard or soft-input soft-output (SISO) decoding
Very simple implementation of majority logic procedure
12
CSOCCSOC are are Non iterativeNon iterative, , systematic and and non recursive
Example of One-Step Threshold DecoderExample of One-Step Threshold Decoder
JJ = 3, = 3, AA= {0, 1, 3}, = {0, 1, 3}, ddminmin= 4= 4
, are LLRs values representing the received symbols ,
t
ûi
> <
D D D
D D D1 =02 =13 =3
Soft outputsin LLR
ut J
w
pt J
w
= tanh/tanh-1 (sum-product) or add-min (min-sum) operator
Decoded bits
ut J
w pt J
w uty p
ty
01
0(2 -1)=1(3 -0)=3 (3 -1)=2
13
Extension to Extension to IterativeIterative Threshold Decoding Threshold Decoding Convolutional Self-Doubly-Orthogonal Codes : Convolutional Self-Doubly-Orthogonal Codes : CSOCSO22CC
Decoder exploits the doubly-orthogonal properties of CSO2C Asymptotic error performance (dmin=J+1 ) at moderate Eb/N0
1. All the differences (j - k ) are distinct ;
2. The differences of differences (j -k )–(l -n ), j k, k n, n l, l j, must be distinct from all the differences (r - s ), r s ;
3. The above differences of differences are distinct except for the unavoidable repetitions
Novel Iterative Error Control Coding SchemesNovel Iterative Error Control Coding Schemes
Issues : Search and determination of new CSO2Cs
Extention of Golomb rulers problem (unsolved)
14
(3,0,2,0)=((15)-(-13))= 28(3,0,3,0)=((15)-(-15))= 30(3,1,0,1)=((12)-( 3))= 9(3,1,2,0)=((12)-(-13))= 25(3,1,2,1)=((12)-(-10))= 22(3,1,3,0)=((12)-(-15))= 27(3,1,3,1)=((12)-(-12))= 24(3,2,0,1)=(( 2)-( 3))= -1(3,2,0,2)=(( 2)-( 13))= -11(3,2,1,0)=(( 2)-( -3))= 5(3,2,1,2)=(( 2)-( 10))= -8(3,2,3,0)=(( 2)-(-15))= 17(3,2,3,1)=(( 2)-(-12))= 14(3,2,3,2)=(( 2)-( -2))= 4
(0,1,0,1)=(( -3)-( 3))= -6(0,2,0,1)=((-13)-( 3))= -16(0,2,0,2)=((-13)-(13))= -26(0,3,0,1)=((-15)-( 3))= -18(0,3,0,2)=((-15)-(13))= -28(0,3,0,3)=((-15)-(15))= -30(1,0,1,0)=(( 3)-( -3))= 6(1,2,0,2)=((-10)-(13))= -23(1,2,1,0)=((-10)-( -3))= -7(1,2,1,2)=((-10)-(10))= -20(1,3,0,2)=((-12)-(13))= -25(1,3,0,3)=((-12)-(15))= -27(1,3,1,0)=((-12)-( -3))= -9(1,3,1,2)=((-12)-(10))= -22
(1,3,1,3)=((-12)-(12))= -24(2,0,1,0)=((13) -( -3))= 16(2,0,2,0)=((13)-(-13))= 26(2,1,0,1)=((10)-( 3))= 7(2,1,2,0)=((10)-(-13))= 23(2,1,2,1)=((10)-(-10))= 20(2,3,0,1)=(( -2)-( 3))= -5(2,3,0,3)=(( -2)-( 15))= -17(2,3,1,0)=(( -2)-( -3))= 1(2,3,1,3)=(( -2)-( 12))= -14(2,3,2,0)=(( -2)-(-13))= 11(2,3,2,1)=(( -2)-(-10))= 8(2,3,2,3)=(( -2)-( 2))= -4(3,0,1,0)=((15)-( -3))= 18
Differences of Differences
ExampleExample of CSO2C, of CSO2C, J=J=44,, 15 ,13 ,3 ,0 A
All the differences of differences are distinct These codes are suitable for iterative threshold or
belief propagation decoding
)()(),,,,( nlkjlnkj )()(),,,,( nlkjlnkj )()(),,,,( nlkjlnkj
15
Issue : minimization of memory length (span) m of encoders
Lower bound on span
Spans of some best known CSO2C encodersSpans of some best known CSO2C encoders
J m (span) J m (span) J m (span)
5 41 14 13774 23 402923
6 100 15 16503 24 502505
7 222 16 34908 25 643676
8 459 17 50071 26 965950
9 912 18 71858 27 1117924
10 1698 19 107528 28 1517378
11 3467 20 148787 29 1894067
12 5173 21 209013 30 2437586
13 9252 22 299126
4( )Jm f J
16
Approximate MAP value i:
Decision rule :ûi=1 if and only if i 0, otherwise ûi= 0
CSOC i is an equation of independent variables
Non-Iterative Threshold Decoding for CSOCs
J
jij
u
iiy
1,
Received Inform. Symb.
ExtrinsicInformation= +
: Addmin operator; ( )s ( ) min{ , }a b sign a ign b a b where
1
( ) ( )1 1 1
j j k j k
jJ Ju p u
i i i i ij k k j
y y y
17
IterativeIterative Threshold Decoding for CSO2Cs Threshold Decoding for CSO2Cs
Feedback for past symbols
Feedforward for future symbols
J
n
J
nli
n
li
pi
uii lnkjlnkjnkjkjkj
yy1 1
)1()(
1
1
)2()(
)1(
J
n
J
nli
n
li
pi
uii lnkjlnkjnkjkjkj
yy1 1
)()(
1
1
)1()(
)(
1 Iteration: Distinct Differences 2 Iterations: Distinct Differences of differences Distinct Differences of differences from Differences
J
j
J
jki
j
ki
pi
uii kjkjj
yy1 1
)()(
1
1
)1()(
)(
1
1
)1()(
j
ki kj
J
jki kj
1
)()(
General Expression:
Iterative Expressions:
18
Depends on the simple differencesDepends on the simple differences
Estimation of at Iteration uiy
depends on the simple differencesdepends on the simple differencesand onand on
the differences of differencesthe differences of differences
• No interleaver• One ( identical ) decoder per
iteration• Forward-only operation
Features :
Forward-Only Iterative DecoderForward-Only Iterative Decoder
Iterative Threshold Decoder Structure for CSO2CsIterative Threshold Decoder Structure for CSO2Cs
From channel Hard Decision
Delay m
......
......
......
Soft output
Softoutput
Softoutput
Softoutput
thresholddecoderIteration
=1
......
......
......
thresholddecoderIteration
=2
thresholddecoderIteration
=I
thresholddecoderIteration
=M
Last Iteration
DecodedInformation
symbols
Information symbols
Parity-check symbols
mt 2 Mmt t m mIt Delay m Delay m Delay m
Delay m Delay m
19
Block Diagram of Iterative Threshold Decoder
(CSO2Cs)
One-step TH decoding per iteration Iterative TH decoder ( M iterations M one-step decoders) Each one-step decoder for a distinct bit
Latency m bits
Input For
OutputFor
Total Latency M m bits
t
t
u
p
Latency m bits Latency m bits
20
Iterative Belief Propagation (BP) Decoder of CSO2CIterative Belief Propagation (BP) Decoder of CSO2C
DEC 1 DEC 2 DEC M
ptw utw
pMmtw
uMmtw
}{ )(,
MjMmtv
)1( mt )2(
2 mt )(
MMmt
Mmtu ˆ 0
1
DEC 1 DEC 2 DEC M
ptw utw
pMmtw
uMmtw
)1( mt )2(
2 mt )(
MMmt
Mmtu ˆ 0
1
(TH)
M(BP) ~ ½ M(TH)
BP Latency ~ ½ TH Latency1-step BP complexity ~ J X 1-step TH complexity
(BP)
Threshold Decoder Threshold Decoder
BP Decoder BP Decoder
Latency m bits
Latency m bits Latency m
bits
Latency m bits
Latency m bits
Latency m bits
21
1.5 2 2.5 3 3.5 4 4.5 510
-7
10-6
10-5
10-4
10-3
10-2
10-1
Eb/N0, dB
Bit
err
or
rate
Both BP and TH decoding approach the asymptotic error performance in error floor region
Error Performance Behaviors of CSO2CsError Performance Behaviors of CSO2Cs
J=9, A={0, 9, 21, 395, 584, 767, 871, 899, 912}
BP, 8 iterations
BP, 4 iterations
TH, 8 iterations
BPError floor region
THError floor region
THWaterfall region
BP Waterfall region
22
Reduce span by relaxing conditions on the double orthogonality at small degradation of the error performance Simplified S-CSO2C
Search and determination of new S-CSO2Cs with minimal spans
Analysis Results of CSO2CsAnalysis Results of CSO2Cs
With iterative decoding, error performance depends essentially on the number of connections , rather than on memory lengths (spans) .
Effects of Code Structure on Error Performance
Improvements : Span Reduction
J m
Best known codes: rapid increase of encoding spans with J :
Optimal codes unknown (Minimum span m )
Shortcomings of CSO2Cs
4( )Jm f J 4( )Jm f J
23
The set of connection positions A satisfies :1. All the differences (j - k ) are distinct ;
2. The differences of differences (j -k)-(l -n ), j k, k n, n l, l j, are distinct from all the differences (r - s ), r s ;
3. The differences of differences are distinct except for the unavoidable repetitions and a number of avoidable repetitions
Definition of S-CSO2CsDefinition of S-CSO2Cs
edN
edN
8
)232( 23
JJJJNd
Number of repeated differences of differences (excluding the unavoidable repetitions)
Maximal number of distinct differences of differences(excluding the unavoidable repetitions)
ed
d
N
N 2/10 , Normalized simplification factor
Search and determination of new short span S-CSO2Cs yielding value 24
Comparison of Spans of CSO2Cs and S-CSO2CsComparison of Spans of CSO2Cs and S-CSO2Cs
JCSO2C
m (span)S-CSO2Cm (span)
JCSO2C
m (span)S-CSO2Cm (span)
5 41 0.3818 23 14 13774 0.4269 1967
6 100 0.4333 45 15 16503 0.4253 2653
7 222 0.4416 82 16 34908 0.4313 3532
8 459 0.4828 129 17 50071 0.4246 4978
9 912 0.4895 208 18 71858 0.4002 6905
10 1698 0.4917 340 19 107528 0.4053 8748
11 3467 0.4539 588 20 148787 0.3923 9749
12 5173 0.4632 894
13 9252 0.4193 1217
25
Performance Comparison for J=10 S-CSO2CPerformance Comparison for J=10 S-CSO2C
26
Uncoded BPSKUncoded BPSK
coding gaincoding gain
asymptotic coding gainasymptotic coding gain
Performance Comparison for J=8 Codes (BP Decoding)Performance Comparison for J=8 Codes (BP Decoding)
27
CSO2C: A = { 0, 43, 139, 322, 422, 430, 441, 459 }S-CSO2C: A = { 0, 9, 22, 55, 95, 124, 127, 129 }
Performance Comparison CSO2Cs / S-CSO2Cs Performance Comparison CSO2Cs / S-CSO2Cs (TH Decoding)(TH Decoding)
Eb/No = 3.5 dB
8th iteration
BE
R
Latency (x 104 bits)3000 14000
CSO2C
S-CSO2C
28
29
Convolutional Self-Orthogonal Codes
(CSOC)
SimpleOrthogonality
Extension
Simplified CSO2C (S-CSO2C)
Relaxed DoubleOrthogonality
Relaxed Conditions
DoubleOrthogonality
Convolutional Self-Doubly-Orthogonal
Codes (CSO2C)4( )m f J
Orthogonalproperties
of set A
Substantial Span Reduction
Large Span
mSmall Span
Analysis of Orthogonality Properties (span)
30
Decoded symbol
)( nkjtp
jtp
Analysis of Orthogonality Properties (computational tree)Analysis of Orthogonality Properties (computational tree)
LLR for final hard decision
Simple orthogonality Independence of inputs over ONE iterationDouble orthogonality Independence of inputs over TWO iterations
The computational tree represents the symbols used by the decoder to estimate each information symbol in the iterative decoding process.
Error performances function of
Independency VS Short cycles
Analysis shows that the parity symbols are limiting the decoding performances of the iterative decoder because of their degree 11 in the computational tree
(no descendant nodes).
Impact : The decoder does not update these values over the iterative decoding process : limiting error performances.
)(i
J
j
J
jki
j
ki
pi
uii kjkjj
yy1 1
)()(
1
1
)1()(
)(
Iter (-1)
Iter (-2)
31
Distinct differences
Distinct differences of differences
Distinct differences
from difference of differences
CSOC
CSO2C
ConditionsConditionson associated setson associated setsCodesCodes
Analysis of Orthogonality Properties (cycles)Analysis of Orthogonality Properties (cycles)
No 4-cycles
Minimization ofNumber of
6-cycles
Minimization ofNumber of
8-cycles
Cycles on GraphsCycles on Graphs
Uniformly Distributed
Uniformly Distributed
A number of repetitions of differences of differences
A Number of Additional8-cycles
Approximately Uniformly Distributed
S-CSO2C
32
Asymptotic coding gain
Correspond to the minimum Hamming distance at moderate
Eb/N0 values.
Error performance
1min Jd
Summary of Single Register CSO2CsSummary of Single Register CSO2Cs
Structure of Tanner Graphs for Iterative Decoding No 4–cycles A minimal number of 6–cycles which are due to the unavoidable
repetitions A minimal number of 8–cycles Uniform distribution of the 6 and 8–cycles
Relaxing doubly orthogonal conditions of CSO2C adds some 8-cycles leading to codes with substantially reduced coding spans S-CSO2C
dB 2
1log10 )(log10 10min10
J
RdG
Extension : Recursive Convolutional Doubly-Orthogonal Codes (RCDO)
33
)( nkjtp
jtp
In order to improve the error performances of the iterative decoding algorithm the degree of the parity symbols must be increased
Solution : Use Recursive Convolutional Encoders
(RCDO)
)(i
RCDO codesRCDO codes
RCDO are systematic recursive convolutional encoder RCDO can be represented by their sparse parity-check matrix
HT(D)
Forward connections
Feedback connections
RCDO encoder example : R=3/6, 3 inputs 6 outputs
34
u(2)
u(1)
p(1)
p(2)
u(3)
p(3)
D D D D D D D D D
D D D D D D D D D D
D D D D D D D D
i
i
i
i
i
iD D
35
RCDO protograph structureRCDO protograph structure The parity-check matrix HT(D) completely defined the RCDO
codes.
The memory of the RCDO encoder m is defined by the largest shift register of the encoder
Each line of HT(D) represents one output symbol of the encoder.
Each column of HT(D) represents one constraint equation.
Protograph representation of a RCDO codes is defined by HT(D).
The degree distributions of the nodes in the protograph become important in the convergence behavior of the decoding algorithm.
Regular RCDO (dv, dc) : dv = degree of variable (rows)
dc = degree of constraint (col.) (same numbers of nonzero elements of HT(D) )
Irregular RCDO protograph
36
RCDO doubly-orthogonal conditionsRCDO doubly-orthogonal conditions The analysis of the computational tree of RCDO codes shows that, as for the
CSO2C, three conditions based on the differences must be respected by the connection positions of the encoder.
For RCDO the decoding equations are completely independent over 2 decoding iterations.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
Estimation of parity symbols are now improved from iteration to iteration Resulting in improving the error performances
5050thth iter iterLDPCLDPCn=1008n=1008
IncreasingIncreasingnumber of shift registersnumber of shift registers
decoder limit’ decoder limit’ RCDO (3,6)RCDO (3,6)
1.10 dB1.10 dB
Characteristics :
Small shift registers
Error performances VS number of shift registers
Low number of iterations compared to LDPC
RCDO codes error performancesRCDO codes error performances Error performances of RCDO (3,6) codes, R=1/2, 25th iteration
The complexity per decoded symbol of all the decoders associated with the RCDOs ( in this figure ) is smaller than the one offered by the LDPC decoder of block length 1008. Attractive for VLSI implementation.
38
RCDO codes error performancesRCDO codes error performances
Asymptotic error performances of RCDO close to BP decoder limit
Characteristics :
Coding rate-15/30 15 registers m = 149 Regular HT(D) (3,6) 40th Iteration
Close to optimal convergence behavior of the iterative decoder.
After 40 iterations 0.4 dB
Low error floor
ComparisonsComparisons
RCDOgood error performance at low SNR
CSO2Cgood error performances at moderate SNR
Pb = 10-5
Error performances comparisons with other existing techniquesError performances comparisons with other existing techniques
39 Figure from : C. Schlegel and L. Perez,Trellis and Turbo coding, Wiley, 2004.
Comparison of the techniquesComparison of the techniques
40
CSO2C RCDO LDPC
Implementation ComplexityEncoding
Decoding
Low
Low
Low
Low
High
High
Per Iteration ProcessingSize of operating window
Number of decoded bits
N/M
1
N/M
1
N
N
Error PerformanceEb/N0 (Waterfall region)
BER (Error floor)
Error floor tendency
Moderate
Moderate
Decreasing
Low
Low
Decreasing
Very small
Low
Flat
Block lengthBlock length NN , , IterationsIterations MM
ConclusionConclusion
41
New iterative decoding technique based on systematic doubly orthogonal convolutional codes : CSO2C, RCDO.
CSO2C : good error performances at moderate Eb/No : Single shift register encoder; J dominant
Recursive doubly orthogonal convolutional codes RCDO. Error performances improvement at low Eb/No. Multiple shift registers encoder ; m dominant Error performances comparable to those of LDPC block codes.
Simpler encoding and decoding processes.
Attractive for VLSI high speed implementations
Searching for optimal CSO2C & RCDO codes : open problem
42
