Stopping by Matrix Rigidity on a snowy day Introduction to ... · independently by Grigoriev(1976)....

Stopping by Matrix Rigidity on a snowy dayIntroduction to Matrix Rigidity - I

C RamyaTata Institute of Fundamental Research

Mumbai, INDIA

Workshop on Matrix RigidityFSTTCS 2020

1 / 22

Matrix Rigidity

I Matrix Rigidity was introduced by Valiant(1977) in thecontext of computing linear transformations and was studiedindependently by Grigoriev(1976).

Rigidity of a matrix

Rigidity of a matrix A for rank r is the minimum number of entriesto be changed in A so that rank(A) is at most r .

I Rigidity of a matrix A ∈ Fn×n for rank r is denoted by RFA(r).For the n × n identity matrix In, RFIn(r) ≤ (n − r).

I A matrix is rigid if it is far from any matrix of low rank.I RA(r) is hamming distance between A and rank ≤ r matrices.

Rigidity intertwines combinatorial & algebraic property.

Rigidity has connections to communication complexity,data structure lower bounds and coding theory.

2 / 22

Matrix Rigidity




I Rigidity of a matrix A ∈ Fn×n for rank r is denoted by RFA(r).

For the n × n identity matrix In, RFIn(r) ≤ (n − r).I A matrix is rigid if it is far from any matrix of low rank.I RA(r) is hamming distance between A and rank ≤ r matrices.



2 / 22

Matrix Rigidity








2 / 22

Matrix Rigidity





I A matrix is rigid if it is far from any matrix of low rank.

I RA(r) is hamming distance between A and rank ≤ r matrices.



2 / 22

Matrix Rigidity








2 / 22

Interpreting Matrix Rigidity

Let A ∈ Fn×n. Suppose rigidity of matrix A for rank r is ≤ s.

3 / 22



A =

n× n

aij 7→ bij

No. of ’s here = s

3 / 22



A =

n× n

aij 7→ bij


n× n

C =

bij − aij0

(i,j) =

otherwise

ifcij =

sparsity(C) ≤ srank(A + C) ≤ r

3 / 22



A =

n× n

aij 7→ bij


n× n

C =

bij − aij0

(i,j) =

otherwise

ifcij =


I When RFA(r) ≤ s, there is a matrix C ∈ Fn×n of sparsity ≤ ssuch that rank(A + C ) ≤ r .

3 / 22



A =

n× n

aij 7→ bij


n× n

C =

bij − aij0

(i,j) =

otherwise

ifcij =



I If there is a matrix C ∈ Fn×n of sparsity ≤ s such thatrank(A + C ) ≤ r then RFA(r) ≤ s.

3 / 22



A =

n× n

aij 7→ bij


n× n

C =

bij − aij0

(i,j) =

otherwise

ifcij =



I If there is a matrix C ∈ Fn×n of sparsity ≤ s such thatrank(A + C ) ≤ r then RFA(r) ≤ s.

Rigidity of a matrix A for rank r

RFA(r) = minC{sparsity(C) | C ∈ Fn×n, rankF(A + C ) ≤ r}.

3 / 22

Toy Example I: Identity Matrix

RFA(r) = minC{sparsity(C) | C ∈ Fn×n, rankF(A + C ) ≤ r}

I If RA(r) ≤ s then A = S + L such that S has sparsity ≤ s andL has rank ≤ r .

I If A = S + L with S has sparsity of S ≤ s and rank(L) ≤ rthen RA(r) ≤ s.

Example

Rigidity of n × n identity matrix is (n − r) for any r ≤ n.For any r ≤ n, RIn(r) ≤ (n − r).Suppose, RIn(r) < (n − r). Then, there exists C ∈ Fn×n ofsparsity < (n − r) such that rank(In + C ) ≤ r .

rank(In + C ) ≥ rank(In)− rank(C ) ≥ n − (n − r) > r(⇐⇒)

4 / 22





Example

Rigidity of n × n identity matrix is (n − r) for any r ≤ n.For any r ≤ n, RIn(r) ≤ (n − r).

Suppose, RIn(r) < (n − r). Then, there exists C ∈ Fn×n ofsparsity < (n − r) such that rank(In + C ) ≤ r .


4 / 22





Example

Rigidity of n × n identity matrix is (n − r) for any r ≤ n.For any r ≤ n, RIn(r) ≤ (n − r).Suppose, RIn(r) < (n − r).

Then, there exists C ∈ Fn×n ofsparsity < (n − r) such that rank(In + C ) ≤ r .


4 / 22





Example

Rigidity of n × n identity matrix is (n − r) for any r ≤ n.For any r ≤ n, RIn(r) ≤ (n − r).Suppose, RIn(r) < (n − r). Then, there exists C ∈ Fn×n ofsparsity < (n − r) such that rank(In + C ) ≤ r .


4 / 22

Toy Example II: Building over Identity matrices

Theorem (Midrijānis (2005))

For any n divisible by 2r , RFMn(r) =n2

4r .

n/2r

n× n

I2r I2r

I2r

I2r I2r

I2r

blocksn/2r

blocks

No. of blocks = n24r2

5 / 22




4r .

Proof.

I By changing r entries in each blockconsistently, rank(Mn) is at most r . Thus,

RMn(r) ≤ n2

4r .

n/2r

n× n

I2r I2r

I2r

I2r I2r

I2r

blocksn/2r

blocks


5 / 22




4r .

Proof.

I Clearly, by changing r entries in each blockconsistently rank(Mn) ≤ r . Thus,RMn(r) ≤ n

2

4r .

I Suppose, rank(Mn) can be reduced to r bychanging fewer than n

2

4r entries. Then, ∃ I2rblock whose rank can be reduced to r bychanging fewer than r entries. (⇐⇒)

n/2r

n× n

I2r I2r

I2r

I2r I2r

I2r

blocksn/2r

blocks


5 / 22

Upper Bounds on Matrix Rigidity

Theorem (Valiant(1977))

For any matrix A ∈ Fn×n and any r ≤ n, RFA(r) ≤ (n − r)2.

6 / 22



For any matrix A ∈ Fn×n and any r ≤ n, RFA(r) ≤ (n − r)2.Proof.

I If rank(A) ≤ r then RA(r) = 0.I If rank(A) > r there exists an full rank

r × r submatrix B in A.

6 / 22




Proof.


r × r submatrix B in A.

B C

D E

r

(n− r) (n− r)

(n− r)

r

r

r

(n− r)

6 / 22




Proof.


r × r submatrix B in A.I Every row of D can be expressed as a

linear combination of the r rows of B.

B C

D E

r

r (n− r)

A =

α1row1(B) + α2row2(B) + · · ·+ αrrowr(B)

6 / 22







I Edit every row of E by correspondinglinear combination of the r rows of C .

B C

D E

r

r (n− r)

A =

α1row1(C) + α2row2(C) + · · ·+ αrrowr(C)

6 / 22







I Edit every row of E by correspondinglinear combination of the r rows of C .

B C

D E

r

r (n− r)

A =

α1row1(C) + α2row2(C) + · · ·+ αrrowr(C)

Now, every row of A is a linear combination of the first r rows.By changing (n − r)2 entries in E , rank(A) is reduced to r .Thus, RFA(r) ≤ (n − r)2.

6 / 22

Linear Circuits

I Linear circuits are a computational model involving additionsand scalar multiplications.

7 / 22

Linear Circuits


I A linear circuit C over F is a DAG wherein-degree 0 gates: labelled by variables;internal gates: labelled by +;edges: labelled by constants in F. x1 x1x2 x2 x1

+ +

+ +

+2 31

1

11

421

1

8x1 + 3x2 7x1 + 5x2

8 3

7 5A =

x 7→ A · x x = x1x2

7 / 22

Linear Circuits


I A linear circuit C over F is a DAG wherein-degree 0 gates: labelled by variables;internal gates: labelled by +;edges: labelled by constants in F.

x1 x1x2 x2 x1

+ +

+ +

+2 31

1

11

421

1

8x1 + 3x2 7x1 + 5x2

8 3

7 5A =

x 7→ A · x x = x1x2

I Linear circuits have n inputs, n outputs and fan-in 2 gates.

7 / 22

Linear Circuits



x1 x2 x3 xn−1 xn

`n−1`1 `3`2 `n


7 / 22

Linear Circuits



x1 x2 x3 xn−1 xn`n

`n−1`1

`2

`3

A =

`1

...

`2 `n


I C computes a linear transformation represented by A ∈ Fn×n.

7 / 22

Linear Circuits



`n−1`1

`2

`3

A =

`1

...

`2 `n

I Linear circuits have n inputs, n outputs and fan-in 2 gates.I C computes a linear transformation represented by A ∈ Fn×n.

size(C): # of edgesdepth(C): length of longest path from i/p to o/p.

I Any linear transformation Fn → Fn can be computed by alinear circuit of size O(n2) and depth O(log n).

7 / 22

Linear Circuits



`n−1`1

`2

`3

A =

`1

...

`2 `n


size(C): # of edges

depth(C): length of longest path from i/p to o/p.

I Best known size lower bound: 3n − o(n) (Chashkin 1994).7 / 22

Linear Circuits and Matrix Rigidity

I Can we prove super-linear lower bounds for linearcircuits of logarithmic depth?

I What is the linear circuit complexity of rigid matrices?Can a matrix of high rigidity be computed by linear sizelogarithmic depth linear circuits?


For any A ∈ Fn×n if RA(�n) > n1+δ for some �, δ > 0 then anylinear circuit of depth O(log n) computing the transformationA : x 7→ A · x must have size Ω(n log log n).

I Rigid matrices cannot be computed by linear circuits havingsmall depth as well as small size.

8 / 22

Proof of Valiant’s Theorem

I Consider a linear circuit of size s, depth d , n inputs, n outputsand fan-in 2.

Edge Removal Lemma (Erdös, Graham, and Szemerédi 1976)

Let G be a directed acyclic graph with s edges and every pathhaving length at most d . Then, by removing at most s/ log dedges every path in the resulting graph has length at most d/2.

I Repeating the edge removal process � times, length of everypath at most d/2� and no. of edges removed is s�log d .

`i

removed edges

b1

b2

b3

b1, . . . , bk: tails of removed edgesk ≤ s�log d

9 / 22

Proof(contd.)

I Each ì is a linear combination of the tails b1, . . . , bk and atmost 2d/2

�input variables.

ì

b1 b2 bk

ì =∑kj=1 αijbj + ci

αij ∈ F

ci ∈ Fn,

bj ∈ Fn

2d/2�-sparse

10 / 22

Proof(contd.)


�input variables.

`i

b1 b2 bk


`i = i

j

bi + ci

αij

10 / 22

Proof(contd.)


�input variables.

`i

b1 b2 bk


`i = i

j

bi + ci

αij

I A = B1B2 + C where B1 ∈ Fn×k ,B2 ∈ Fk×n,C ∈ Fn×n.

10 / 22

Proof(contd.)


�input variables.

`i

b1 b2 bk


`i = i

j

bi + ci

αij

I A = B1B2 + C where B1 ∈ Fn×k ,B2 ∈ Fk×n,C ∈ Fn×n.I Then, rank(B1B2) ≤ k ≤ s�log d and sparsity(C ) ≤ n2

d/2� .

10 / 22

Proof(contd.)


�input variables.

`i

b1 b2 bk


`i = i

j

bi + ci

αij


d/2� .

I Thus, rigidity of A for rank s�log d is at most n2d/2� .

10 / 22

Proof(contd.)


�input variables.

`i

b1 b2 bk


`i = i

j

bi + ci

αij


d/2� .

I Thus, rigidity of A for rank s�log d is at most n2d/2� .

I If A ∈ Fn×n is computed by a linear circuit of size n log log nand depth log n then RA(�n) ≤ n1+δ.

10 / 22

Valiant’s Question

I For any A ∈ Fn×n if RA(�n) > n1+δ for some �, δ > 0 thenany linear circuit of depth O(log n) computing A must havesize Ω(n log log n).

Valiant’s Question

Find an explicit sequence of matrices Mn ∈ Fn×n such thatRFMn(�n) ≥ Ω(n

1+δ) for �, δ > 0.

I Explicit: There exists a poly(n) time deterministic algorithmon input 1n outputs the n × n matrix Mn.

This Workshop

Recent Progress towards answering Valiant’s Question (andbeyond).

11 / 22

Existence of Rigid Matrices


Let Fq be a finite field. For any 0 ≤ r ≤ n − Ω(√n) there is a

matrix M ∈ Fn×nq such that RFqM (r) = Ω((n − r)

2/ log n).

12 / 22





2/ log n).

Proof. (via counting)

I Count no. of matrices A ∈ Fn×nq with RA(r) ≤ s.

set of n× n

over Fq

set of matrices ofrigidity ≤ s for rank ≤ r

M

matrices

12 / 22





2/ log n).


I Count no. of matrices A ∈ Fn×nq with RA(r) ≤ s.I If RA(r) ≤ s then A = S +L, sparsity(S) ≤ s and rank(L) ≤ r .

12 / 22





2/ log n).



No.of RA(r) ≤ s matrices:(n2

s

)· qs︸︷︷︸

no. of s-sparse matrices

·(n

r

)2· qn2−(n−r)2︸︷︷︸

no. of rank-r matrices

.

12 / 22





2/ log n).



No.of RA(r) ≤ s matrices:(n2

s

)· qs︸︷︷︸

no. of s-sparse matrices

·(n

r

)2· qn2−(n−r)2︸︷︷︸

no. of rank-r matrices

.

I When r < n − c1√n and s < c2(n − r)2/ log n almost all

matrices have rigidity (n − r)2.

12 / 22

Super-exponential time construction of Rigid Matrices

Super-exponential time construction

For every n × n matrices A with entries in Fq, test ifthere exists any s-sparse matrix C such thatrankFq(A + C ) ≤ r .Running time: qO(n

2) · qs · nO(1).

13 / 22

Super-exponential time construction of Rigid Matrices

Super-exponential time construction

For every n × n matrices A with entries in Fq, test ifthere exists any s-sparse matrix C such thatrankFq(A + C ) ≤ r .Running time: qO(n

2) · qs · nO(1).


Let F be an infinite field. For any 0 ≤ r ≤ n there is a matrixM ∈ Fn×n such that RFM(r) = (n − r)2.

13 / 22

Untouched Minor Argument

I Consider an n × n matrix M all of whose r × rsubmatrices have full rank.

14 / 22



I Suppose few entries of M are changed, there is atleast one untouched submatrix contributing rank r .

14 / 22




I Cauchy matrix: C = {cij}ni ,j=1; cij =1

xi+yjfor 2n distinct

elements x1, . . . , xn, y1, . . . , yn ∈ F.

14 / 22




I Cauchy matrix: C = {cij}ni ,j=1; cij =1

xi+yjfor 2n distinct

elements x1, . . . , xn, y1, . . . , yn ∈ F.

Theorem (Shokrollahi, Spielman, Stemann(1997))

Let F be a field with at least 2n distinct elements and Mn be n× nCauchy matrix. Then, RFMn(r) = Ω(

n2

r lognr ) for log n ≤ r ≤ n/2.

14 / 22

Proof of SSS‘97

Suppose not, RFMn(r) = o(n2

r lognr ).

That is, by changing

o(n2

r lognr ) entries in M, rank can be reduced to r .

I Consider a bipartite graph G = (U,V ,E )with |U| = |V | = n such that(i , j) ∈ E (G ) iff Mij is untouched.

I G has at least n2 − o(n2r lognr ) edges.

U V

1

i j

1

nn

Mij untouched

Theorem (Kovári-Sós-Turán (1954))

The maximum number of edges in any n × n bipartite graphwithout Kr+1,r+1 is at most n

2 − n(n−r)2(r+1) lognr .

I G contains a (r + 1)× (r + 1) complete bipartite subgraph.I If fewer than n

2

4(r+1) lognr entries in M are changed an

(r + 1)× (r + 1) submatrix of Mn remains untouched.

15 / 22

Proof of SSS‘97


r lognr ). That is, by changing

o(n2




U V

1

i j

1

nn

Mij untouched



2 − n(n−r)2(r+1) lognr .


2



15 / 22

Proof of SSS‘97



o(n2




U V

1

i j

1

nn

Mij untouched



2 − n(n−r)2(r+1) lognr .

I G contains a (r + 1)× (r + 1) complete bipartite subgraph.

I If fewer than n2



15 / 22

Proof of SSS‘97



o(n2




U V

1

i j

1

nn

Mij untouched



2 − n(n−r)2(r+1) lognr .


2


(r + 1)× (r + 1) submatrix of Mn remains untouched.15 / 22

Matrices with Algebraically Independent Entries

I a1, . . . , an ∈ R are algebraically independent over Q if there isno polynomial P ∈ Q[x1, . . . , xn] such that P(a1, . . . , an) = 0.

I {π, eπ} are algebraically independent over Q.I Any set of n + 1 polynomials p1, . . . , pn+1 on n variables is

algebraically dependent.

Theorem

Let A ∈ Rn×n with n2 algebraically independent elements over Qas its entries. Then, for any r ≤ n, RRA (r) = (n − r)2.

Proof. Upper bound via Valiant’s theorem.

16 / 22



I {π, eπ} are algebraically independent over Q.

I Any set of n + 1 polynomials p1, . . . , pn+1 on n variables isalgebraically dependent.

Theorem



16 / 22



I {π, eπ} are algebraically independent over Q.I Any set of n + 1 polynomials p1, . . . , pn+1 on n variables is

algebraically dependent.

Theorem



16 / 22


Lower Bound: Suppose not, RRA (r) < (n − r)2. Then A = S + Lsuch that S has sparsity s < (n − r)2 and L has rank ≤ r .

Every entry of A is a function of the n2 − (n − r)2 manyentries of L and s entries of S .

These are n2 polynomials each on n2 − (n − r)2 + s variables.The entries of A are algebraically dependent. (⇒⇐)

The matrix A is not explicit. The degree of theextension [Q(a11, . . . , ann) : Q] = 2n

2.

Can we reduce the amount of algebraic independenceamong the entries while maintaining rigidity?

17 / 22




These are n2 polynomials each on n2 − (n − r)2 + s variables.

The entries of A are algebraically dependent. (⇒⇐)


2.


17 / 22




These are n2 polynomials each on n2 − (n − r)2 + s variables.The entries of A are algebraically dependent. (⇒⇐)


2.


17 / 22

Non-explicit Rigid Matrices

Theorem (Lokam(2000, 2006))

I Let x1, . . . , xn ∈ C be algebraically independent over Q andV = (x ji )1≤i ,j≤n be Vandermonde matrix in C

n×n. Then,RCV (r) = Ω(n

2) for r ≤ O(√n).

I Let A ∈ Cn×n with aij =√pij for distinct primes p11, . . . , pnn.

Then, RCA (r) = Ω(n2) for r ≤ n/32.

Square root of distinct primes are linearly independent over Q.Proof via algebraic dimension argument(Shoup, Smolensky).

18 / 22





2) for r ≤ O(√n).



Square root of distinct primes are linearly independent over Q.

Proof via algebraic dimension argument(Shoup, Smolensky).

18 / 22





2) for r ≤ O(√n).



Square root of distinct primes are linearly independent over Q.Proof via algebraic dimension argument(Shoup, Smolensky).

18 / 22

Rigidity of Random Matrices

I Random matrices are rigid with high probability.

19 / 22



[Goldreich, Tal 2013] Rigidity of Random Toeplitz matrix

For every r ∈ [√n, n/32], RF2T (r) = Ω

(n3

r2 log n

)with probability

1− o(1) where T ∈ Fn×n2 is a random Toeplitz/Hankel matrix.

Toeplitz T =

a0 a1 a2a−1 a0 a1a−2 a−1 a0

and Hankel H =a−2 a−1 a0a−1 a0 a1a0 a1 a2

19 / 22





(n3

r2 log n

)with probability


Toeplitz T =

a0 a1 a2a−1 a0 a1a−2 a−1 a0


Asymptotically better than Ω(n

2

r lognr ) if r = o(

nlog n log log n ).

19 / 22





(n3

r2 log n

)with probability


Toeplitz T =

a0 a1 a2a−1 a0 a1a−2 a−1 a0


Asymptotically better than Ω(n

2

r lognr ) if r = o(

nlog n log log n ).

Explicit construction in ENP

Run over all n × n Hankel/Toeplitz matrices with{0, 1} entries.

For each such matrix test if RF2T (r) = Ω(

n3

r2 log n

).

19 / 22

Designing TESTs,r(H)

TESTs,r (H)

(1) If H is not rigid then reject H.

(2) If H is random Hankel matrix, accept H w.p 1− o(1).

20 / 22


TESTs,r (H)



+=

H S Lsparsity(S) ≤ s rank(L) ≤ r

20 / 22


TESTs,r (H)



+=

H S L

2r

2r

(n/2r)2 submatrices

20 / 22


TESTs,r (H)



+=

H S L

2r

2r

(n/2r)2 submatrices sparsity(S ′) ≤ s(n/2r)2

rank(L′) ≤ r

H’ S’ L’

20 / 22


TESTs,r (H)



TESTs,r (H)

Partition H into submatrices of dimension 2r × 2r each.For every such submatrix H ′ of H

For every s(n/2r)2

-sparse matrix S ′ in F2r×2r2If rank(H ′ − S ′) ≤ r then reject H

Accept H

20 / 22


TESTs,r (H)



TESTs,r (H)


For every s(n/2r)2


Accept H

Pr[TEST rejects H] = Pr[∃H ′∃S ′ rank(H ′-S ′) ≤ r ]Need to bound Pr[rank(H ′-S ′) ≤ r ].

20 / 22


TESTs,r (H)


For every s(n/2r)2


Accept H

Pr[TEST rejects H] = Pr[∃H ′∃S ′ rank(H ′-S ′) ≤ r ]Need to bound Pr[rank(H ′-S ′) ≤ r ].

2r

n/2r

n/2r

H

2r

2r

(n/2r)2 submatrices

H

H’

20 / 22

Explicit Rigid matrices beyond exponential time

(Folklore) Sub-exponential time construction ofM ∈ Fn×n2 with R

F2M (n

1/2−�) ≥ Ω(n2/ log n).(Alman, Chen ‘20) M ∈ Fn×n2 in PNP such that thereexists a δ > 0 with RM(2

(log n)1/4−�) ≥ δn2 for all � > 0.(Bhangale, Harsha, Paradise, Tal ‘20) M ∈ Fn×n2 inPNP such that there exists a δ > 0 withRM(2

log n/Ω(log log n)) ≥ δn2.

Works for any finite field for large n.

Proof via linear circuit lower bounds & PCPs.

21 / 22

The Road Thus Taken

Thank You! Questions?22 / 22

Date post:	25-Jan-2021
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Stopping by Matrix Rigidity on a snowy day Introduction to ... · independently by Grigoriev(1976)....

Documents