On Linear-Programming Decoding of Nonbinary Expander Codes · LP decoding of nonbinary expander...

On Linear-Programming Decoding

of Nonbinary Expander Codes

Vitaly SkachekClaude Shannon Institute

University College Dublin

Supported by SFI Grant 06/MI/006

University College CorkMay 18, 2009

Vitaly Skachek LP Decoding of Nonbinary Expander Codes

Literature Survey

[Gallager ’62]Low-Density Parity-Check (LDPC) codes.


Literature Survey


→ Very efficient in practice.


Literature Survey



→ Difficult to analize.


Literature Survey




[Wiberg ’96] [Koetter Vontobel ’03–’05]Graph covers, pseudocodewords and pseudoweights.


Literature Survey





[Feldman Wainwright Karger ’03–’05]Decoding of binary LDPC codes using linear-programming.


Literature Survey





[Feldman Wainwright Karger ’03–’05]Decoding of binary LDPC codes using linear-programming.

[Feldman et al. ’04] [Feldman Stein ’04]LP decoding on expander codes corrects a fraction oferrors, achieves capacity.


This Work

LP decoding of nonbinary expander codes.


This Work


The decoder corrects a number of errors which isapproximately a quarter of a lower bound on the minimumdistance.


This Work



We consider:

Bipartite expander graph.Two different types of constituent codes.


This Work



We consider:

Bipartite expander graph.Two different types of constituent codes.

The proof does not use a separate assumption on thesymmetry of the LP polytope.


Code Construction

[Sipser Spielman ’95] [Barg Zemor ’01–’02]


Code Construction


Graph G = (V, E) is a ∆-regular bipartite undirectedgraph.


Code Construction



Vertex set V = A ∪ B such that A ∩ B = ∅ and|A| = |B| = n.Edge set E of size n∆ such that every edge in E has oneendpoint in A and one endpoint in B.


Code Construction



Vertex set V = A ∪ B such that A ∩ B = ∅ and|A| = |B| = n.Edge set E of size n∆ such that every edge in E has oneendpoint in A and one endpoint in B.Linear [∆, rA∆, δA∆] and [∆, rB∆, δB∆] codes CA and CB ,respectively, over F = Fq.


Code Construction




C is a linear code of length |E| over F:

C =

{

c ∈ F|E| :

(c)E(v) ∈ CA for every v ∈ A and

(c)E(v) ∈ CB for every v ∈ B

}

,


Code Construction




C is a linear code of length |E| over F:

C =

{

c ∈ F|E| :

(c)E(v) ∈ CA for every v ∈ A and

(c)E(v) ∈ CB for every v ∈ B

}

,

where (c)E(v) = the sub-word of c that is indexed by theset of edges incident with v.


Example

Take k = 2, ∆ = 3, n = 4.Let GA and GB be generatingmatrices of CA and CB

(respectively) overF22 = {0, 1, α, α2}:

GA =

(

1 1 11 α 0

)

,

GB =

(

1 0 10 1 α

)

.


Example

Take k = 2, ∆ = 3, n = 4.Let GA and GB be generatingmatrices of CA and CB

(respectively) overF22 = {0, 1, α, α2}:

GA =

(

1 1 11 α 0

)

,

GB =

(

1 0 10 1 α

)

. v4

v3

v2

v1

u4

u3

u2

u1

A B0α

α2

0α

α2

α2

0α

1α

0

0

α2

1

α0

α

α2

α0

α2

α0


Eigenvalues of Expander Graph

Assume that all vertices in G have degree ∆. The largesteigenvalue of the adjacency matrix AG of G is ∆.




Let λG be the second largest eigenvalue of AG .





Lower ratios of γG = λG

∆ imply greater values ζ ofexpansion. [Alon ’86]





Lower ratios of γG = λG

∆ imply greater values ζ ofexpansion. [Alon ’86]

Expander graph with

λG ≤ 2√

∆ − 1

is called a Ramanujan graph. Constructions are due to[Lubotsky Philips Sarnak ’88], [Margulis ’88].


Parameters of Expander Codes

Code Rate

RC ≥ rA + rB − 1.


Parameters of Expander Codes

Code Rate

RC ≥ rA + rB − 1.

Relative Minimum Distance

δC ≥ δAδB − γG√

δAδB

1 − γG.


General Notation

Let the codeword c = (ce)e∈E ∈ C be transmitted and theword y = (ye)e∈E ∈ F

|E| be received.


General Notation


|E| be received.

Define the mapping

ξ : F −→ {0, 1}q ⊂ Rq ,

byξ(α) = x = (x(ω))ω∈F ,

such that, for each ω ∈ F,

x(ω) =

{

1 if ω = α0 otherwise.


General Notation


|E| be received.

Define the mapping

ξ : F −→ {0, 1}q ⊂ Rq ,

byξ(α) = x = (x(ω))ω∈F ,

such that, for each ω ∈ F,

x(ω) =

{

1 if ω = α0 otherwise.

Let Ξ(c) = (ξ(ce1) | ξ(ce2) | · · · | ξ(ce|E|)).


Objective Function

For vectors f ∈ Rq|E|, we adopt the notation

f = (f e1| f e2

| · · · | f e|E|) ,

where∀e ∈ E, f e = (f (α)

e )α∈F .


Objective Function


f = (f e1| f e2

| · · · | f e|E|) ,


e )α∈F .

For all e ∈ E, α ∈ F, we use the variables f(α)e ≥ 0.


Objective Function


f = (f e1| f e2

| · · · | f e|E|) ,


e )α∈F .


Variables wv,b for all v ∈ V and all b ∈ C(v): relative

weights of local codewords b associated with E(v).


Objective Function


f = (f e1| f e2

| · · · | f e|E|) ,


e )α∈F .


Variables wv,b for all v ∈ V and all b ∈ C(v): relative

weights of local codewords b associated with E(v).

The objective function is∑

e∈E

∑

α∈Fγ

(α)e f

(α)e , where γ

(α)e

is a function of the channel output.


Objective Function (cont.)

For each α ∈ F we set

γ(α)e =

{

−1 if α = ye

1 if α 6= ye.




γ(α)e =

{

−1 if α = ye

1 if α 6= ye.

Let f e = ξ(β) for some e ∈ E, β ∈ F. Then,

∑

α∈F

γ(α)e f (α)

e =

{

−1 if β = ye

1 if β 6= ye.




γ(α)e =

{

−1 if α = ye

1 if α 6= ye.

Let f e = ξ(β) for some e ∈ E, β ∈ F. Then,

∑

α∈F

γ(α)e f (α)

e =

{

−1 if β = ye

1 if β 6= ye.

Suppose now that f = Ξ(z) for some z ∈ F|E|. It follows

that∑

e∈E

∑

α∈F

γ(α)e f (α)

e + |E| = 2d(y, z) ,

where d(y, z) is the Hamming distance between y and z.


Primal Problem

Maximize∑

e∈E,α∈F

(

−γ(α)e

)

· f (α)e


Primal Problem

Maximize∑

e∈E,α∈F

(

−γ(α)e

)

· f (α)e

subject to ∀v ∈ V :∑

b∈C(v) wv,b = 1 ;


Primal Problem

Maximize∑

e∈E,α∈F

(

−γ(α)e

)

· f (α)e


b∈C(v) wv,b = 1 ;

∀e = {v, u} ∈ E, ∀α ∈ F : f(α)e =

∑

b∈C(v) : be=α wv,b ,

f(α)e =

∑

b∈C(u) : be=α wu,b ;


Primal Problem

Maximize∑

e∈E,α∈F

(

−γ(α)e

)

· f (α)e


b∈C(v) wv,b = 1 ;

∀e = {v, u} ∈ E, ∀α ∈ F : f(α)e =

∑

b∈C(v) : be=α wv,b ,

f(α)e =

∑

b∈C(u) : be=α wu,b ;

∀e ∈ E, α ∈ F : f(α)e ≥ 0 ;

∀v ∈ V, b ∈ C(v) : wv,b ≥ 0 .


Dual Witness

Dual Witness Approach [Feldman et al. ’04]


Dual Witness


The codeword c ∈ C was transmitted.


Dual Witness



There is a feasible combination of values of the variableswv,b that corresponds to c.


Dual Witness



There is a feasible combination of values of the variableswv,b that corresponds to c.

Decoding Success

The sufficient criteria for the decoding success is that thissolution is the unique optimum of the primal LP decodingproblem.


Decoding Success

Primal Polytope

Dual Polytope


Decoding Success

Primal Polytope

Dual Polytope

Max

Min


Decoding Success

Primal Polytope

Dual Polytope

Max

Min

Feasible Point


Decoding Success

Primal Polytope

Dual Polytope

Max

Min

Feasible Point

Dual Problem Variables

For each ω ∈ F, e ∈ E, and v ∈ V , such that v is an

endpoint of e, there is a variable τ(ω)v,e .


Decoding Success

Primal Polytope

Dual Polytope

Max

Min

Feasible Point

Dual Problem Variables

For each ω ∈ F, e ∈ E, and v ∈ V , such that v is an

endpoint of e, there is a variable τ(ω)v,e .

For each v ∈ V , there is a variable σv.


Dual Problem

Minimize∑

v∈V σv


Dual Problem

Minimize∑

v∈V σv

subject to ∀e = {v, u} ∈ E, ∀ω ∈ F : τ(ω)v,e + τ

(ω)u,e ≤ γ

(ω)e ;


Dual Problem

Minimize∑

v∈V σv

subject to ∀e = {v, u} ∈ E, ∀ω ∈ F : τ(ω)v,e + τ

(ω)u,e ≤ γ

(ω)e ;

∀v ∈ V, ∀b ∈ C(v) :∑

e∈E(v) τ(be)v,e + σv ≥ 0 .


Uniqueness

Minimize∑

v∈V σv

subject to ∀e = {v, u} ∈ E, ∀ω ∈ F\{ce} : τ(ω)v,e + τ

(ω)u,e <γ

(ω)e ;


Uniqueness

Minimize∑

v∈V σv


(ω)u,e <γ

(ω)e ;

∀e = {v, u} ∈ E : τ(ce)v,e + τ

(ce)u,e ≤ γ

(ce)e ;


Uniqueness

Minimize∑

v∈V σv


(ω)u,e <γ

(ω)e ;

∀e = {v, u} ∈ E : τ(ce)v,e + τ

(ce)u,e ≤ γ

(ce)e ;

∀v ∈ V, ∀b ∈ C(v) :∑

e∈E(v) τ(be)v,e + σv ≥ 0 .


Feasible Point for the Dual: Value Assignments

We aim at the objective value to be |E| − 2d(y, c). This can beachieved by setting, for all v ∈ V , σv = 1

2∆ − d((y)E(v), (c)E(v)).


Feasible Point for the Dual: Value Assignments

We aim at the objective value to be |E| − 2d(y, c). This can beachieved by setting, for all v ∈ V , σv = 1

2∆ − d((y)E(v), (c)E(v)).

ω = ce ω 6= ce

ye is correct τ(ω)v,e = −1

2 τ(ω)v,e = 1

2 − ǫ

ye is in error τ(ω)v,e = 1

2 τ(ω)v,e = −5

2 − ǫ or τ(ω)v,e = 3

2depends on the structure of the error


Uniqueness (again)

Minimize∑

v∈V σv


(ω)u,e < γ

(ω)e ;

∀e = {v, u} ∈ E : τ(ce)v,e + τ

(ce)u,e ≤ γ

(ce)e ;


Uniqueness (again)

Minimize∑

v∈V σv


(ω)u,e < γ

(ω)e ;

∀e = {v, u} ∈ E : τ(ce)v,e + τ

(ce)u,e ≤ γ

(ce)e ;

∀v ∈ V, ∀b ∈ C(v) :∑

e∈E(v) τ(be)v,e ≥ −σv .


Uniqueness (again)

Minimize∑

v∈V σv


(ω)u,e < γ

(ω)e ;

∀e = {v, u} ∈ E : τ(ce)v,e + τ

(ce)u,e ≤ γ

(ce)e ;

∀v ∈ V, ∀b ∈ C(v) :∑

e∈E(v) τ(be)v,e ≥ −σv .

Here for all v ∈ V , σv = 12∆ − d((y)E(v), (c)E(v)).


Error Orientation

Definition

The assignment of the directions to theedges of the subgraph H = (UA ∪ UB, E)is called a (ρA, ρB)-orientation if eachvertex v ∈ UA and each vertex u ∈ UB

has at most ρA∆ and ρB∆ incomingedges in E, respectively.


Error Orientation

Definition



ρA∆

ρB∆


Error Orientation

Definition



Error pattern orientation: for ω 6= ce and

ye in error, the value τ(ω)v,e = −5

2 − ǫ willbe assigned if the edge e enters thevertex v.

ρA∆

ρB∆


Error Orientation in Expander Graphs

Existence of (< 14δA, < 1

4δB)-orientation yields a sufficientlysmall number of assignments −5

2 − ǫ. This, in turn, yields that

∑

e∈E(v)

τ (be)v,e ≥ −σv .


Error Orientation in Expander Graphs

Existence of (< 14δA, < 1

4δB)-orientation yields a sufficientlysmall number of assignments −5

2 − ǫ. This, in turn, yields that

∑

e∈E(v)

τ (be)v,e ≥ −σv .

Lemma

Let H = (UA ∪ UB, E) be a subgraph of G = (A ∪ B, E).Assume that |E| ≤ (αβ − 1

2γG)∆n for some α, β ∈ (0, 1], suchthat γG ≤ √

αβ, and 12α∆, 1

2β∆ are both integers. Then, E

contains a (β/2, α/2)-orientation.


Fraction of Correctable Errors

Theorem

Let θA > 0 (θB > 0) be the largest number such thatθA < δA (θB < δB) and 1

4θA∆ (14θB∆, respectively) is

integer.



Theorem



integer.

Let C be as above, and assume that γG ≤ 12

√θAθB.



Theorem



integer.


√θAθB.

Then, the LP decoder corrects any error pattern of a size lessthan or equal to (1

4θAθB − 12γG)∆n.



Theorem



integer.


√θAθB.

Then, the LP decoder corrects any error pattern of a size lessthan or equal to (1

4θAθB − 12γG)∆n.

Remark

It is possible to improve (slightly) on the low-order term in theexpression for the number of correctable errors in the statementof the main result. In the present work, we omit this analysis.


Date post:	31-Oct-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

On Linear-Programming Decoding of Nonbinary Expander Codes · LP decoding of nonbinary expander...

Documents