A SINKHORN-KNOPP FIXED POINT PROBLEM

Outline

The Sinkhorn-Knopp Algorithm and Fixed Point Problem

Solutions for 2 × 2 and special n × n cases

Circulant matrices for 3 × 3 case Ongoing work

Sinkhorn-Knopp Problem

In the 1960s Sinkhorn and Knopp developed an algorithm which transforms any positive matrix A into a doubly stochastic matrix by pre- and post-multiplication of diagonal matrices

where is a solution to

where (-1) is the entry-wise inverse.

)( ))(( )1( xdiagAxAdiag

)1()1( ))(( xAAx T x

Examples

11

41A)1()1( ))(( xAAx T

Solutions to for

Examples

11

41A

1

2

1

2

62

64

64

62

12

21

)1()1( ))(( xAAx T

)( ))(( )1( xdiagAxAdiag x

Solutions to for

Examples

11

41A)1()1( ))(( xAAx T

Solutions to for

Examples

11

41A

1

2i

1

2i

)1()1( ))(( xAAx T

)( ))(( )1( xdiagAxAdiag x

Solutions to for

ii

ii

52

51

52

54

52

54

52

51

ii

ii

52

51

52

54

52

54

52

51

Examples

11

41 iA)1()1( ))(( xAAx T

Solutions to for

Examples

11

41 iA

1

2 i

)1()1( ))(( xAAx T

)( ))(( )1( xdiagAxAdiag x

Solutions to for

1

2 i

121

122

424

422

iii

iii

iii

121

122

424

422

iii

iii

iii

2 x 2 Case

or any other multiple thereof.

0,,, dcba

ac

bd

bd

acac

bdx

or

1or

1

dc

baA)1()1( ))(( xAAx T

Solutions to for

Observations

All solutions to

result in matrices

with row and column sums of 1.

Exactly one solution is guaranteed to result in a doubly stochastic matrix if A is all positive. It is the unique solution found by the Sinkhorn-Knopp Fixed Point Algorithm

)1()1( ))(( xAAx T

)( ))(( )1( xdiagAxAdiag

)1()1( ))((1 kk xAAx T

Formulae for solutions

The general formula for the 2 × 2 case is simple.

The general formula for the 3 × 3 case is far more complicated. Its only real use thus far is to verify that there are at most 6 solutions, which we had already predicted by numerically finding solutions.

We have guesses for how many solutions there are in the general n × n case.

Some general rules

For all non-zero diagonal matrices, every non-zero vector is a solution.

For constant matrices, the only solution is the vector of all 1’s.

Any nonzero multiple of a solution is also a solution—this is especially important.

)1()1(

)1()1( )))((())((

xcAAxcxAAx TT

)1()1( ))(( xAAx T

Block matrices

If

, and

then .

)1()1( ))(( jjTjj xAAx

kA

A

A 1

kx

x

x

1

)1()1( ))(( xAAx T

Row and column swaps

For . The matrix formed by swapping any two

rows of A has the same solutions. The matrix formed by swapping two

columns of A has the same solutions, but with corresponding elements swapped. That is, if columns i and j are swapped in A, then the solution is the original solution, but with elements i and j swapped.

)1()1( ))(( xAAx T

Larger matrices

For 3 x 3 and larger matrices, the general case is too complicated.

We considered special cases: Already mentioned diagonal and constant

matrices Upper and lower triangular Patterned matrices, including circulant

matrices

Circulant Matrices

We have shown that any eigenvector for any non-zero n × n circulant matrix is a solution:

This include the vector of all 1’s, the only solution that results in a doubly stochastic matrix.

unity. ofroot theis where

)(

)(

1

th

1

mx m

nm

km

m

m

Circulant Matrices

Observations:

since .

If , then .

)1(mnm xx

mmm xxA mmnm

T xxA

kk

k)(

1

)(

1 m-n

mm

Circulant Matrices

Eigenvectors: if ,

then )1()1()1()1( ))(())(( mmT

mT xAxAA

)1()1(1 ))(( m

T xAm

)1(1 )( mn

T xAm

)1()(

1 )( mnmnn x

m

.mx

mmm xxA

)( mnnx )1()(

mnx

Circulant Matrices

There are other solutions as well. We consider the 3 × 3 case

which has three other (non-e.vector)

solutions:

abc

cab

bca

A

cabcbacbabca

x2

2

2

1

bacbabcacabc

x2

2

2

2

abcacabcbacb

x2

2

2

3

Circulant Matrices

There are other solutions as well. We consider the 3 × 3 case

which has three other (non-e.vector)

solutions:

abc

cab

bca

A

cabcbacbabca

x2

2

2

1

bacbabcacabc

x2

2

2

2

abcacabcbacb

x2

2

2

3

Circulant Matrices

The columns of those solutions form another circulant matrix, which we label A1:

The solutions for this matrix are similar to those for the original circulant matrix, A, which we now label as A0.

abca

bacb

cabc

cabc

abca

bacb

bacb

cabc

abca

A222

222

222

1

Circulant Matrices

For

the first element of the first solution vector is

)(

3

))(()(

2

222333333

2

1

1121

2

222

bcaabc

cbacbcaba

a

cba

abca

cabc

bacb

abca

abca

bacb

cabc

cabc

abca

bacb

bacb

cabc

abca

A222

222

222

1

Circulant Matrices

With similar results for the second and third elements, the first solution vector is

)(

3)(

3)(

3

2

222333333

2

222333333

2

222333333

abcabc

cbacbcabaacbabc

cbacbcababcaabc

cbacbcaba

Circulant Matrices

With similar results for the second and third elements, the first solution vector is

after factoring out the common term.

abc

acb

bca

2

2

2

1

1

1

)(

3)(

3)(

3

2

222333333

2

222333333

2

222333333

abcabc

cbacbcabaacbabc

cbacbcababcaabc

cbacbcaba

Circulant Matrices

With similar results for the other two solutions, we find all three solutions and form a new (and again circulant) matrix

bcaacbabc

abcbcaacb

acbabcbca

A

222

222

222

2

111

111

111

Circulant Matrices

For

the first element of the first solution vector is

)3(

))(()(

333

1

1121

2

2222

2

222

cbaabca

a

cba

bca

abcacbbca

bcaacbabc

abcbcaacb

acbabcbca

A

222

222

222

111

111

111

2

Circulant Matrices

With similar results for the second and third elements, the first solution is

)3(

)3(

)3(

333

333

333

cbaabcc

cbaabcb

cbaabca

Circulant Matrices

With similar results for the second and third elements, the first solution is

after factoring out the common term.

c

b

a

)3(

)3(

)3(

333

333

333

cbaabcc

cbaabcb

cbaabca

Circulant Matrices

With similar results for the other two solutions, we find all three solutions and form a new (yet again circulant) matrix

which is, of course, the original matrix A.

abc

cab

bca

A3

The cycle of matrices

abc

cab

bca

A3

bcaacbabc

abcbcaacb

acbabcbca

A

222

222

222

2

111

111

111

abca

bacb

cabc

cabc

abca

bacb

bacb

cabc

abca

A222

222

222

1

abc

cab

bca

A

Current work

Solutions for n × n cases: Circulant Upper/lower triangular Other patterned matrices Numerical solutions for general case Maximum number of solutions

How to characterize solutions: do doubly stochastic solutions minimize some sort of energy function for a given matrix, while the non-doubly stochastic solutions maximize the energy function?

Questions?