3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 1

Generalized LDPC Codes

Eran Sharon, Ran Zamir and Dudy Avraham

March 2017

3/8/2019©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 2

Background: Low Density Parity Check Codes• State of the art codes - Exhibit near Shannon limit performance under iterative decoding

• Linear block code characterized by a sparse parity-check matrix H

• Can also be represented by a sparse bipartite graph G– Left side nodes (variable nodes) represent the codeword bits

– Right side nodes (check nodes) represent the parity-check constraints that the bits should satisfy

Right nodes

(Check nodes)

Sparse bipartite graph (G)

Left nodes

(Bit nodes)

Sparse parity check matrix (H)

0

1

1

1

1

1

1

0

1

1

1

1

Right nodes

(Check nodes)

Sparse bipartite graph (G)

Left nodes

(Bit nodes)

Sparse parity check matrix (H)

1

1

0

0

0

1

1

1

1

1

1

0

1

1

1

1

Right nodes

(Check nodes)

Sparse bipartite graph (G)

Left nodes

(Bit nodes)

Sparse parity check matrix (H)

1

1

0

0

0

0

0

0

1

1

1

1

11

0

1

1

1

1

1

1

0

1

1

1

1

H·cT = 0

c = [0 1 0 1]

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 3

Generalized Low Density Parity Check Codes

LDPC code

1

1

0

1

1

0

0

0

1

0

GLDPC code

1

1

0

Extended Hamming code

[k = 4, n = 8]

Parity check code [k = 2, n = 3]

Simplex code [k = 3, n = 7 ]

1

1

0

0Repetition code

[k = 1, n = 2 ]

1,1,0

1

0

LDPC code

Repetition code [k = 1, n = 3]

000

111

111

01

1

01

1

01

1

0

0

00

Parity check code [k = 3, n = 4]

Parity check code [k = 2, n = 3]

Repetition code [k = 1, n = 2]

1

1

0

1

1

0

0

0

1

0

Csimplex =

{0000000, 0001111,

0110011, 1010101,

0111100, 1011010,

1100110, 1101001}

Edge bit assignments satisfy a Simplex code constraint:

• An equivalent view of an LDPC code: – Each left node represents a repetition code constraint (some may be punctured nodes)– Each right node represents a parity check code constraint

• Why should we limit ourselves to Repetition and Parity Check codes?

• Generalized LDPC (GLDPC) Codes– We can use general constituent codes for imposing node constraints:

Repetition, Parity Check, Simplex, Hamming, 1st order Reed Muller, Extended Hamming,…

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 4

Iterative Decoding of GLDPC Codes

• Iterative Belief Propagation decoding

LDPC code

A-priori LLRs from the NAND

Lva-priori

Extrinsic LLRs from bit nodes to check nodes

Lvc

Extrinsic LLRs from check nodes to bit nodes

Lcv

A-posteriori LLRs

Lva-posteriori

Log Likelihood Ratio (LLR) – common “soft” metric:

𝐿 = logPr(𝑏𝑖𝑡=0| observations & constraints)Pr(𝑏𝑖𝑡=1| observations & constraints)

vv3

3( , )\j

j

a priori

vc v c v

c N v G c

L L L

v

Lv

a-priori

vcL1 vc1

vcL

2 vc2

cvL

1cv

1

cvL

2cv

2

33cv

Lcv

cvL

4cv

4

cvL

5cv

5

cc6

6

1

( , )\

2 tanh tanh2

j

j

v c

cv

v N c G v

LL

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 5

Iterative Decoding of GLDPC Codes

• Iterative Belief Propagation decoding

LDPC code

A-priori LLRs from the NAND

Lva-priori

Extrinsic LLRs from bit nodes to check nodes

Lvc

Extrinsic LLRs from check nodes to bit nodes

Lcv

A-posteriori LLRs

Lva-posteriori

Log Likelihood Ratio (LLR) – common “soft” metric:

𝐿 = logPr(𝑏𝑖𝑡=0| observations & constraints)Pr(𝑏𝑖𝑡=1| observations & constraints)

vv3

3( , )\j

j

a priori

vc v c v

c N v G c

L L L

v

Lv

a-priori

vcL1 vc1

vcL

2 vc2

cvL

1cv

1

cvL

2cv

2

33cv

Lcv

cvL

4cv

4

cvL

5cv

5

cc6

6

1

( , )\

2 tanh tanh2

j

j

v c

cv

v N c G v

LL

Repetition code MAP decoder

Parity Check code MAP decoder

Maximum A-Posteriori Decoder (MAP):

𝐿𝑗𝑀𝐴𝑃= log

Pr(𝑏𝑗 = 0| 𝐿, 𝑏 ∈ 𝑪)

Pr(𝑏𝑗 = 1| 𝐿, 𝑏 ∈ 𝑪)= log

σ𝑏 𝑠.𝑡. 𝑏𝑗=0Pr(𝐿|𝑏)

σ𝑏 𝑠.𝑡. 𝑏𝑗=1Pr(𝐿|𝑏)

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 6

Iterative Decoding of GLDPC Codes

• Iterative Belief Propagation decoding

LDPC code

A-priori LLRs from the NAND

Lva-priori

Extrinsic LLRs from bit nodes to check nodes

Lvc

Extrinsic LLRs from check nodes to bit nodes

Lcv

A-posteriori LLRs

Lva-posteriori

Repetition code MAP decoder

Parity Check code MAP decoder

Interleaver +Memory for

Edge Messages

IA,c

IE,c

IA,v

IE,v

Iterative decoding: exchange of information

between MAP decoders of small constituent codes

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 7

Iterative Decoding of GLDPC Codes

• Iterative Belief Propagation decoding

A-priori LLRs from the NAND

Lva-priori

Extrinsic LLRs from bit nodes to check nodes

Lvc

Extrinsic LLRs from check nodes to bit nodes

Lcv

A-posteriori LLRs

Lva-posteriori

Repetition code MAP decoder

Parity Check code MAP decoder

Interleaver +Memory for

Edge Messages

IA,c

IE,c

IA,v

IE,v

Iterative decoding: exchange of information

between MAP decoders of small constituent codes

eHammingcode MAP decoder

IA,c

IE,c

Simplex code MAP decoder

IA,v

IE,v

GLDPC code

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 8

Iterative Decoding of GLDPC Codes

• Iterative Belief Propagation decoding

LDPC code

A-priori LLRs from the NAND

Lva-priori

Extrinsic LLRs from bit nodes to check nodes

Lvc

Extrinsic LLRs from check nodes to bit nodes

Lcv

A-posteriori LLRs

Lva-posteriori

Log Likelihood Ratio (LLR) – common “soft” metric:

𝐿 = logPr(𝑏𝑖𝑡=0| observations & constraints)Pr(𝑏𝑖𝑡=1| observations & constraints)

Repetition code MAP decoder

Parity Check code MAP decoder

Interleaver +Memory for

Edge Messages

IA,c

IE,c

IA,v

IE,v

Maximum A-Posteriori Decoder (MAP):

𝐿𝑗𝑀𝐴𝑃= log

Pr(𝑏𝑗 = 0| 𝐿, 𝑏 ∈ 𝑪)

Pr(𝑏𝑗 = 1| 𝐿, 𝑏 ∈ 𝑪)= log

σ𝑏 𝑠.𝑡. 𝑏𝑗=0Pr(𝐿|𝑏)

σ𝑏 𝑠.𝑡. 𝑏𝑗=1Pr(𝐿|𝑏)

Iterative decoding: exchange of information

between MAP decoders of small constituent codes

GLDPC code

eHammingcode MAP decoder

IA,c

IE,c

Simplex code MAP decoder

IA,v

IE,v

Decoding complexity:

• MAP decoding for general length n code hasexponential complexity O(2n)

• Repetition & Parity-Check codes have linearcomplexity O(n) (special cases)

• We limit ourselves to several additional structuredconstituent code families:

• Hamming• Extended Hamming• Simplex• 1st order Reed Muller

• We developed low complexity encoding O(n) anddecoding O(n·log(n)) based on a Fast Walsh-HadamardTransform

[1] A.Ashikhmin and S. Litsyn “Simple MAP decoding of First Order

Reed Muller and Hamming Codes,” Trans. Info. Theory, vol. 50, pp 1812-1818, Aug. 2004.

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 9

Repetition code MAP decoder

Lva-priori

Lc1v

Lc2v

Lvc3 = Lva-priori+Lc1v+Lc2v

Designing GLDPC Codes

[2] E. Sharon, A. Ashikhmin, S. Litsyn, ``EXIT functions for binary input memoryless symmetric channels," Trans. on Communications, vol. 54, pp. 1207--1214, July, 2006. [3] E. Sharon, A. Ashikhmin, S. Litsyn, ``Analysis of low-density parity-check codes based on EXIT functions,"Trans. on Comm, vol. 54, pp. 1407-1414, Aug. 2006.

Repetition code MAP decoder

Parity Check code MAP decoder

Interleaver

IA,c

IE,c

IA,v

IE,veHamming code MAP decoder

IA,c

IE,c

Simplex code MAP decoder

IA,v

IE,v

• Main challenge: How to design good GLDPC codes?– How to select the optimal mixture of constituent codes?

– How to connect them?

• GLDPC spectrum optimizer:– Asymptotic analysis by tracking the Extrinsic Information

Transfer (EXIT) between the constituent codes

EXIT function characterizes how a-priori information is amplified to extrinsic information by a MAP decoder

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 10

IA IE

Repetition code MAP decoder

Designing GLDPC Codes

[2] E. Sharon, A. Ashikhmin, S. Litsyn, ``EXIT functions for binary input memoryless symmetric channels," Trans. on Communications, vol. 54, pp. 1207--1214, July, 2006. [3] E. Sharon, A. Ashikhmin, S. Litsyn, ``Analysis of low-density parity-check codes based on EXIT functions,"Trans. on Comm, vol. 54, pp. 1407-1414, Aug. 2006.

Repetition code MAP decoder

Parity Check code MAP decoder

Interleaver

IA,c

IE,c

IA,v

IE,veHamming code MAP decoder

IA,c

IE,c

Simplex code MAP decoder

IA,v

IE,v

• Main challenge: How to design good GLDPC codes?– How to select the optimal mixture of constituent codes?

– How to connect them?

• GLDPC spectrum optimizer:– Asymptotic analysis by tracking the Extrinsic Information

Transfer (EXIT) between the constituent codes

EXIT function characterizes how a-priori information is amplified to extrinsic information by a MAP decoder

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 11

IE

Repetition code MAP decoder

Designing GLDPC Codes

[2] E. Sharon, A. Ashikhmin, S. Litsyn, ``EXIT functions for binary input memoryless symmetric channels," Trans. on Communications, vol. 54, pp. 1207--1214, July, 2006. [3] E. Sharon, A. Ashikhmin, S. Litsyn, ``Analysis of low-density parity-check codes based on EXIT functions,"Trans. on Comm, vol. 54, pp. 1407-1414, Aug. 2006.

Repetition code MAP decoder

Parity Check code MAP decoder

Interleaver

IA,c

IE,c

IA,v

IE,veHamming code MAP decoder

IA,c

IE,c

Simplex code MAP decoder

IA,v

IE,v

• Main challenge: How to design good GLDPC codes?– How to select the optimal mixture of constituent codes?

– How to connect them?

• GLDPC spectrum optimizer:– Asymptotic analysis by tracking the Extrinsic Information

Transfer (EXIT) between the constituent codes

EXIT function characterizes how a-priori information is amplified to extrinsic information by a MAP decoder

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 12

Designing GLDPC Codes

[2] E. Sharon, A. Ashikhmin, S. Litsyn, ``EXIT functions for binary input memoryless symmetric channels," Trans. on Communications, vol. 54, pp. 1207--1214, July, 2006. [3] E. Sharon, A. Ashikhmin, S. Litsyn, ``Analysis of low-density parity-check codes based on EXIT functions,"Trans. on Comm, vol. 54, pp. 1407-1414, Aug. 2006.

Repetition code MAP decoder

Parity Check code MAP decoder

Interleaver

IA,c

IE,c

IA,v

IE,veHamming code MAP decoder

IA,c

IE,c

Simplex code MAP decoder

IA,v

IE,v

• Main challenge: How to design good GLDPC codes?– How to select the optimal mixture of constituent codes?

– How to connect them?

• GLDPC spectrum optimizer:– Asymptotic analysis by tracking the Extrinsic Information

Transfer (EXIT) between the constituent codes

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 13

Designing and Generating QC-GLDPC Codes

[4] E. Sharon and S. Litsyn, ``Constructing LDPC codes by error minimization progressive edge growth,“ Trans. on Communications, vol. 56, pp. 359-368, March 2008.

• GLDPC codes can be designed based on a lifted graph QC-GLDPC– Enables high throughput decoder design

• Additional code design and generation tools for QC-GLDPC:– EXIT based GLDPC protograph optimizer (via annealing)

– EMPEG based QC-GLDPC code generator [4]• Generates the graph edge by edge, avoiding detrimental configurations that may lead

to an error floor.

v1

v2

v3

v4

v5

v6

c1

c2

c3

Waterfall optimization

Error floor optimization

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 14

Simulation Results

• Comparing three [N = 4608B,K = 4096B, R = 8/9] codes:– (4,36)-regular LDPC code

– Optimized Irregular LDPC code

– Optimized Irregular GLDPC code

• All codes are Quasi-Cyclic (Z = 256)

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 15

Summary• GLDPC codes: sparse bipartite graph with general constituent codes:

Repetition, Parity Check, Hamming, Simplex,…

• Low complexity iterative decoding:

– Exchanging information between low complexity MAP decoders (O(n), O(n·log(n)))

– Quasi-Cyclic structure for supporting high throughput decoders

• Code design and generation tools

– EXIT based GLDPC spectrum optimizer (via linear programming)

– EXIT based GLDPC protograph optimizer (via annealing)

– EMPEG based GLDPC code generator

GLDPC codes can outperform state of the art LDPC codes

• GLDPC framework opens a huge search space and extra degrees of freedom• Enable improvements across various ECC metrics

3/8/2019©2018 Western Digital Corporation or its affiliates. All rights reserved. 16

Thank You

© 2018 Western Digital Corporation or its affiliates. All rights reserved. 3/8/2019