Lower Bounds for Depth Three Circuits with small bottom fanin

Neeraj Kayal Chandan SahaIndian Institute of Science

A lower bound

Theorem: Consider representations of a degree d polynomial of the form

If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such that s is at least .

A lower boundTheorem: Consider representations of a degree d polynomial of the form

If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such that s is at least .

Remark: For a generic , s must be at least

Any asymptotic improvement in the exponent will imply .

A lower boundCorollary (via Kumar-Saraf): Consider representations of a degree d polynomial of the form

If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such for s is at least .

Remark: Bad news.

Good news.

Background/Motivation

Arithmetic Circuits

…𝑥2𝑥1𝑥𝑛

…

     …

-1

𝑥1+𝑥2 𝑥1−𝑥2

𝑥2𝑥1𝑥𝑛

Arithmetic Circuits

…

     …

     …

-1

𝑥1+𝑥2 𝑥1−𝑥2

𝑥2𝑥1𝑥𝑛

Arithmetic Circuits

…

     …

     …

-1

𝑥2𝑥1𝑥𝑛

Arithmetic Circuits

…

     …

     …

-1

𝑥2𝑥1𝑥𝑛

Arithmetic Circuits

…

     …

     …

-1

 

)(xHn

𝑥2𝑥1𝑥𝑛

…

     …

     …

-1

 

)(xHn

𝑥2𝑥1𝑥𝑛

Size = Number of Edges

…

     …

     …

-1

 

)(xHn

𝑥2𝑥1𝑥𝑛

Depth

…

     …

     …

-1

 

)(xHn

𝑥2𝑥1𝑥𝑛

This talk.

> Field is .

> Gates have unbounded fanin

>

Two Fundamental Questions

Can explicit polynomials be efficiently computed?

Can computation be efficiently parallelized?

Two Fundamental Questions

Can explicit polynomials be efficiently computed? Does VP equal VNP?

Can computation be efficiently parallelized? Can every efficient computation be also done by small, shallow circuits?

Can computation be efficiently parallelized?

Question: How efficiently can we simulate circuits of

size by circuits of depth ?

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,

GKKS13, Tavenas-13, Wigderson/Tavenas): Any

circuit of size s and degree can be simulated

by a -depth circuit of size

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,

GKKS13, Tavenas-13, Wigderson/Tavenas): Any

circuit of size s and degree can be simulated

by a -depth circuit of size … also by a regular,

homogeneous -depth circuit of size

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,

GKKS13, Tavenas-13, Wigderson/Tavenas): Any

circuit of size s and degree can be simulated

by a -depth circuit of size … also by a regular,

homogeneous -depth circuit of size

Question: Is this optimal?

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,

GKKS13, Tavenas-13, Wigderson/Tavenas): Any

circuit of size s and degree can be simulated

by a -depth circuit of size … also by a regular,

homogeneous -depth circuit of size

Question: Is this optimal?

(KS15 + Ramprasad): For yes, but with caveats.

Can computation be efficiently parallelized?

Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,

GKKS13, Tavenas-13, Wigderson/Tavenas): Any

circuit of size s and degree can be simulated

by a -depth circuit of size … also by a regular,

homogeneous -depth circuit of size

Corollary: Strong enough lower bounds for low-depth circuits imply VP VNP.

A possible way to approach VP vs VNP

• Strong enough lower bounds for low-depth circuits imply VP VNP.

Low Depth Circuit (’s simple)

Low-depth circuits are easy to analyze..

A possible way to approach VP vs VNP

• Strong enough lower bounds for low-depth circuits imply VP VNP.

Low Depth Circuit (’s simple)

Low-depth circuits are easy to analyze..

Lots of work on lower bounds for low depth

arithmetic circuits in recent years – hope to

discover general patterns and technical

ingredients

’s are: Lower Bound

has degree ~ in VNP GKKS13+KSS14

has degree ~ IMM FLMS14

is sparse ~ in VNP KLSS14

is sparse ~ IMM KS14

has degree one and arity IMM This work

have homogeneous depth three circuits of bottom fanin

in VNP This work

have homogeneous depth five circuits of bottom fanin

IMM Next talk

A possible way to approach VP vs VNP

Prove strong enough lower bounds for low depth

circuits.

Low Depth Circuit (’s simple)

Low-depth circuits are easy to analyze..

Lots of work on low depth arithmetic circuits recently

This talk:

• common pattern/proof strategy

• Technical ingredients

A common Proof Strategy and some technical

ingredients

Proof Strategy (’s simple ). Let .

shallow circuit C

Find a geometric property GP of the ’s.

Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M()) is “relatively large”.

Proof Strategy (’s simple ). Let .

shallow circuit C

Find a geometric property GP of the ’s.

Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M()) is “relatively large”.

Lower Bounding rank of large matrices

• If a matrix M(f) has a large upper triangular submatrix, then it has large rank

• (Alon): If the columns of M(f) are almost orthogonal then M(f) has large rank.

When the ’s have low degree (’s have low degree). Let .

shallow circuit C

Find a geometric property GP of the ’s.

Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M()) is “relatively large”.

Finding a geometric property GP of T

is a product of low degree polynomials

V(T) is a union of low-degree hypersurfaces

V(T) has lots of high-order singularities

Finding a geometric property GP of T

𝑇 (𝑥 , 𝑦 )=𝑔(𝑥 , 𝑦) ∙ h(𝑥 , 𝑦)

Finding a geometric property GP of T

is a product of low degree polynomials

V(T) is a union of low-degree hypersurfaces

V(T) has lots of high-order singularities

V( T) has lots of points

When the ’s have low degree (’s have low degree). Let .

shallow circuit C

Find a geometric property GP of the ’s.

Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M()) is “relatively large”.

When the ’s have low degree (’s have low degree). Let .

shallow circuit C

Find a geometric property GP of the ’s.

Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M()) is “relatively large”.

Expressing largeness of a variety in terms of rank

is a variety.

Let = set of degree- polynomials.

Expressing largeness of a variety in terms of rank

is a variety.

Let = set of degree- polynomials. Let = set of degree- polynomials which vanish at every point of V.

Hilbert’s Theorem (Informal): If V is “large” then has small dimension.

Expressing largeness of a variety in terms of rank

is a variety.

Let = set of degree- polynomials. Let = set of degree- polynomials which vanish at every point of V.

Hilbert’s Theorem (Informal): If V is “large” then has small dimension.

Let = { () of deg } .

Hilbert’s Theorem (Formal): If V has dimension r then has asymptotic dimension .

When the ’s have low degree (’s have low degree). Let .

shallow circuit C

Find a geometric property GP of the ’s.

Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”

Show that rank(M()) is “relatively large”.

Restrictions

Example: is a function. f(0, 1, z) is a restriction of f. Geometrically, f(0, 1, z) is same as restricting f to the axis-parallel line W = V(x, y-1). Algebraically, f(0, 1, z) is same as mod

More generally: Let be any ideal and a polynomial. Call mod as an algebraic restriction of f.

Obs: Hilbert’s theorem can be generalized for algebraic restrictions of polynomials.

Employing restrictions

Lemma (KLSS14): If Q is a sparse polynomial then for a suitable random algebraic restriction mod is a low degree polynomial.

Yields lower bounds for homogeneous depth four (KLSS14 and KS14).

Lemma (KLSS14): If T is a sparse polynomial a sum of product of low arity polynomials then for a suitable random algebraic restriction mod is a low degree polynomial.

Employing Restrictions

Lemma: If T is a sparse polynomial a sum of product of low arity polynomials then for a suitable random algebraic restriction mod is a low degree polynomial.

Employing Restrictions

Yields lower bounds for homogeneous depth five with low bottom fanin (KS15 and BC15).

Lemma (Shpilka-Wigderson): A depth three circuit C of size s can be converted to a homogeneous depth circuit C’ of size . Further if C has bottom fanin t then C’ also has bottom fanin t.

A lemma by Shpilka and Wigderson

Yields lower bounds mentioned earlier.

Conclusion

Proving lower bounds for low depth circuits is a potential way to prove lower bounds for more general circuits.

There is a meta-strategy common to many recent (and older) lower bounds. We don’t understand the power or limitations of this meta-strategy.

Open: lower bounds for homogeneous depth three circuits for polynomials of degree .