1

Network Source Coding

Lee Center Workshop 2006Wei-Hsin Gu (EE, with Prof. Effros)

2

Outline

Introduction Previously Solved Problems Our Results Summary

3

Problem Formulation (1)

General network : A directed graph G = (V,E). Source inputs and reproduction demands.

All links are directed and error-free. Source sequences

Distortion measures are given.

X

Y

X

Y

ˆ ˆ,X Y

,,1

( , ) ( , )n n

nn n

X Y i iX Yi

P x y P x y

4

Problem Formulation (2)

Rate Distortion Region Given a network. : source pmf. : distortion vector. A rate vector is - achievable if there exists a sequence of length n

codes of rate whose reproductions satisfy the distortion constraints asymptotically.

The closure of the set consisting of all achievable is called the rate distortion region, .

A rate vector is losslessly achievable if the error probability of reproductions can be arbitrarily small. Lossless rate region .

D

D

R

R

R

XP

( , )X

R P D

( )L XR P

5

Example

Decoder

EncoderEncoder Decoder( )

1nf ( )

2nf

( )1

ng

( )2ng

Black : sourcesRed : reproductions

,X Y XR YR

Z

ˆXX D

ˆYY D

is achievable if and only if

( , )X YR R ( , )X YD D

6

Outline

Introduction Previously Solved Problems Our Results Summary

7

Known Results

Black : sourcesRed : reproductions

DecoderEncoder XX XR

Lossless : Source Coding Theorem

Lossy : Rate-Distortion Theorem

( )XR H X

ˆ( , )

ˆmin ( ; )Xd X X D

R I X X

Black : sourcesRed : reproductions

Encoder

DecoderEncoder XX

Y

XR

YR

Lossless : Solved [Ahlswede and Korner `75].

Encoder1

Encoder3

Encoder2

Decoder 1

Decoder 2

X

Y

XR

YR

XYR XD

YD

Black : sourcesRed : reproductions

X

,X Y

Y

Lossy : [Gray and Wyner ’74]

W|X,Y

|{ } |{ }

For some P

( ) ( )

( , ; )

X X W X Y Y W Y

XY

R R D R R D

R I X Y W

ˆ ˆ,X YX

Y

XR

YR

Encoder1

Encoder2

Decoder

Lossless : Slepian-Wolf problem

( | ) ( | )

( , )X Y

X Y

R H X Y R H Y X

R R H X Y

8

Outline

Introduction Previously Solved Problems Our Results

Two Multi-hop NetworksProperties of Rate Distortion Regions

Summary

9

Multi-Hop Network (1)Achievability Result

Source coding for the following multihop network

Black : sourcesRed : reproductionsDecoder

EncoderEncoder Decoder

Node 1Node 2

Node 3

ˆXX D

ˆYY D

XR YR,X Y

Z

Achievability Result, | ,

|

For some ( ( , ) ( , ) is a Markov chain)

( ) ( , ; ) ( , ; | , )

( , ; | ) ( , ; | , )

and

ˆ ˆ( , , ) . . ( , ( , , ))

U V X Y

X X U X

Y

Y

P Z X Y U V

R R D I X Y U I X Y V U Z

R I X Y U Z I X Y V U Z

Y U V Z s t Ed Y Y U V Z D

10

Multi-Hop Network (1)Converse Result

Source coding for the following multihop network

Black : sourcesRed : reproductionsDecoder

EncoderEncoder Decoder

Node 1Node 2

Node 3

ˆXX D

ˆYY D

XR YR,X Y

Z

Converse Result, | ,For some ( ( , ) ( , ) is a Markov chain)

( , ; ) ( , ; | , )

( , ; | )

and

ˆ ˆ( ) . . ( , ( )) .

U V X Y

X

Y

X

P Z X Y U V

R I X Y U I X Y V U Z

R I X Y V Z

X U s t Ed X X U D

11

Multi-Hop Network (2)

Diamond Network

Encoder1

Encoder3

Encoder2

Decoder2

Decoder1

Decoder3

1X

2X

Y

1R

2R

3R

4R

1 2, ,X X Y

1X

2X

Y

1R

2R

3R

4R

1 2, ,X X Y

12

Diamond NetworkSimpler Case

When are independent Lossless (by Fano’s inequality) :

1 2 1 2 3 4

1 4 1 2 3 2

1 1 2 2

( ) ( ) ( ) ( )

( ) ( ) ( ) ( )

( ) ( )

R R H X H X H Y R R H Y

R R H X H Y R R H X H Y

R H X R H X

1 1

2 2

3

4

( ) ( )

( ) (1 ) ( )

( )

(1 ) ( )

[0,1]

R H X H Y

R H X H Y

R H Y

R H Y

Is optimal

1 2, ,X X Y

1X

2X

Y

1R

2R

3R

4R

1 2, ,X X Y

13

Diamond NetworkAchievability Result

1 1,U V

2 2,U V

1X

2X

Y

1R

2R

3R

4R

1 2, ,X X Y

14

Diamond NetworkConverse Result

1X

2X

Y

1R

2R

3R

4R

1 2, ,X X Y

15

Outline

Introduction Previously Solved Problems Our Results

Two Multi-hop NetworksProperties of Rate Distortion Regions

Summary

16

Properties of RD Regions

: Rate distortion region. : Lossless rate region. is continuous in for finite-alphabet sources. Conjecture

is continuous in source pmf

( , )X

R P D

( , )X

R P D

D

XP( , )

XR P D

( )L XR P

17

Continuity

Can allow small errors estimating source pmf. Trivial for point-to-point networks – rate distortion function is continu

ous in probability distribution. Two convex subsets of are if and only if

Continuity means that

Proved to be true for those networks whose one-letter characterizations have been found.

e1 2,C C -close1 1 2 2 1 2, . . ( , )c C c C s t d c c

2 2 1 1 1 2, . . ( , )c C c C s t d c c

0, 0 . . ( , ) and ( , ) are - when

|| ||X X

X X

s t R P D R Q D closed

P Q

( is fixed)D

18

Continuity - Example

Slepian-Wolf problem

X

Y

ˆ ˆ,X YXR

YR

( | ) ( | )

( , )X Y

X Y

R H X Y R H Y X

R R H X Y

( | )H X Y

( | )H Y X

( , )H X Y

( , )H X Y

( , )P X Y

( , )Q X Y

( , ) ( , )Q X Y P X Y

19

Continuity - Example

Coded side information problem

Since alphabet of is bounded, and are uniformly continuous in over all

X

Y

XXR

YR

|

( | ) ( ; )

For some X Y

U Y

R H X U R I Y U

P

U ( | )H X U ( ; )I Y U

|U YP,X YP

20

Outline

Introduction Previously Solved Problems Our Results Summary

21

Summary Study the RD regions for two multi-hop networks.

Solved for independent sources. Achievability and converse results are not yet known to

be tight.

Study general properties of RD regions RD regions are continuous in distortion vector for finit

e-alphabet sources. Conjecture that RD regions are continuous in pmf for fi

nite-alphabet sources.