Discrepancy Berlin

8/12/2019 Discrepancy Berlin

1/64

1/30

Discrepancy and SDPs

Nikhil Bansal (TU Eindhoven)


2/64

Outline

Discrepancy: definitions and applications

Basic results: upper/lower bounds

Partial Coloring method (non-constructive)

SDPs: basic method

Algorithmic Spencers ResultLovett-Meka result

Lower bounds via SDP duality (Matousek)2/30


3/64

Material

Classic: Geometric Discrepancy by J. Matousek

Papers:

Bansal. Constructive algorithms for discrepancy minimization,

FOCS 2010

Matousek. The determinant lower bound is almost tight

Lovett, Meka. Discrepancy minimization by walking on the

edges

Survey with fewer technical details:

Bansal. 3/30


4/64

4/30

Discrepancy: What is it?

Study of gaps in approximating the continuous by the discrete.

Original motivation: Numerical Integration/ Sampling

Problem: How wellcan you approximate a region by discrete points

Discrepancy:Max over intervals I

|(# points in I)(length of I)|


5/64

5/30

Discrepancy: What is it?

Study of gaps in approximating the continuous by the discrete.

Problem: How uniformlycan you distribute points in a grid.

Uniform: For every axis-parallel rectangle R

| (# points in R) - (Area of R) | should be low.

n1/2

n1/2

Discrepancy:

Max over rectangles R|(# points in R)(Area of R)|


6/64

6/30

Distributing points in a grid

Problem: How uniformlycan you distribute points in a grid.

Uniform: For every axis-parallel rectangle R

| (# points in R) - (Area of R) | should be low.

Uniform Random Van der Corput Set

n= 64points

n1/2discrepancy n1/2(loglog n)1/2O(log n)discrepancy!


7/64

Quasi-Monte Carlo Methods

With N random samples: Error \prop 1/\sqrt{n}

Quasi-Monte Carlo Methods: \prop Disc/n

Can discrepancy be O(1) for 2d grid?

No. \Omega(log n) [Schmidt ]

d-dimensions: O(log^{d-1} n) [Halton-Hammersely ]

\Omega(log^{(d-1)/2} n) [Roth ]

\Omega(log^{(d-1)/2 + \eta} n

[Bilyk,Lacey,Vagharshakyan08]

7/30


8/64

8/30

Discrepancy: Example 2

Input: n points placed arbitrarilyin a grid.Color them red/bluesuch that each rectangle is colored as

evenly as possible

Discrepancy: max over rect. R ( | # redin R - #bluein R | )

Continuous: Color each element

1/2 red and 1/2 blue (0 discrepancy)

Discrete:

Random has about O(n1/2log1/2n)

Can achieve O(log2.5n)


9/64

9/30

Combinatorial Discrepancy

Universe:U= [1,,n]

Subsets:S1,S2,,Sm

Color elements red/blue so eachset is colored as evenlyas possible.

Find : [n] !{-1,+1} to

Minimize |(S)|1= maxS| i 2S(i) |

If A is m \times n incidence matrix.

Disc(A) = min_{x \in {-1,1}^n} |Ax|_\infty

S1

S2

S3

S4


10/64

10/30

Applications

CS: Computational Geometry, Comb. Optimization, Monte-Carlosimulation, Machine learning, Complexity, Pseudo-Randomness,

Math: Dynamical Systems, Combinatorics, Mathematical Finance,

Number Theory, Ramsey Theory, Algebra, Measure Theory,


11/64

Hereditary Discrepancy

11/30


12/64

Rounding

Lovasz-Spencer-Vesztermgombi86

Given any matrix A, and x \in R^n

can round x to \tilde{x} \in Z^n s.t.

|AxA\tilde{x}|_\infty < Herdisc(A)

Proof: Round the bits one by one.

12/30


13/64

Can we find it efficiently?

Nothing known until recently.

Thm [B10]. Can efficiently round so thatError \leq O(\sqrt{log m log n})

Herdisc(A)

13/30


14/64

More rounding approaches

Bin Packing

Refined further by Rothvoss (Entropy roundingmethod)

14/30


15/64

Dynamic Data Structures

N points in a 2-d region.

Weights update over time.

Query: Given an axis-parallel rectangle R,

determine the total weight on points in R.

Preprocess:1) Low query time

2) Low update time (upon weight change)

15/30


16/64

Example

Line:

Query = O(n) Update = 1

Query = 1 Update = O(n^2)Query = 2 Update = O(n)

Query = O(log n) Update = O(log n)

Recursively can get for 2-d.

16/30


17/64

What about other objects?

Query

Circles arbitrary rectangles aligned triangle

Turns out t_q t_u \geq n^{1/2}/log^2 n ?

Larsen-Green: t_q t_u \geq disc(S)^n/log^2 n

17/30


18/64

Sketch of idea

A good data structure impliesD = A P

A = row sparse P = Column sparse

(low query time) (low update time)18/30


19/64

Outline again

19/30


20/64

Basic Results

20/30


21/64

21/30

Best Known Algorithm

Random: Color each element i independently as

x(i) = +1 or -1 with probability each.

Thm: Discrepancy = O (n log n)1/2

Pf: For each set, expect O(n1/2)discrepancy

Standard tail bounds: Pr[ | i 2Sx(i) | cn1/2] e-c

2

Union bound + Choose c (log n)1/2

Analysis tight:Random actually incurs ((n log n)1/2

).


22/64

22/30

Better Colorings Exist!

[Spencer 85]: (Six standard deviations suffice)Always exists coloring with discrepancy6n1/2

(In general for arbitrary m, discrepancy = O(n1/2log(m/n)1/2)

Tight: For m=n, cannot beat 0.5 n1/2 (Hadamard Matrix, orthogonal sets)

Inherentlynon-constructiveproof(pigeonhole principle on exponentially large universe)

Challenge:Can we find italgorithmically?

Certain algorithms do not work [Spencer]

Conjecture[Alon-Spencer]:May not be possible.


23/64

23/30

Beck Fiala Thm

U = [1,,n] Sets: S1,S2,,Sm

Suppose each element lies in at most tsets (t


24/64

24/30

Approximating Discrepancy

Question:If a set system has low discrepancy (say


25/64

25/30

Our Results

Thm 1: Can get Spencers bound constructively.

That is, O(n1/2)discrepancy for m=n sets.

Thm 2: If each element lies in at mosttsets, get bound of

O(t1/2log n) constructively (Srinivasans bound)

Thm 3: For any set system, can find

Discrepancy O(log (mn))Hereditary discrepancy.

Other Problems:Constructive bounds (matching current best)

k-permutation problem [Spencer, Srinivasan,Tetali]

Geometric problems ,


26/64

26/30

Relaxations: LPs and SDPs

Not clear how to use.

Linear Programis useless. Can color each element redandblue. Discrepancy of each set = 0!

SDPs (LP on vivj, cannot control dimension of vs)

| i 2Svi|2 n 8S

|vi|2= 1

Intended solution vi= (+1,0,,0) or (-1,0,,0).

Trivially feasible: vi = ei (all vis orthogonal)

Yet, SDPs will be a major tool.


28/64

28/30

Talk Outline

Introduction

The Method

Low Hereditary discrepancy -> Good coloring

Additional Ideas

Spencers O(n1/2

) bound


29/64

Partial Coloring Method

29/30


30/64

A Question

30/30

-n n


31/64

Slight improvement

Can be improved to O(\sqrt{n})/2^n

If you pick a random {-1,1} coloring sw.p. say >= |a \cdot s| \leq c \sqrt{n}

2^{n-1} colorings s, with |a\cdot s| \leq c\sqrt{n}

31/30


32/64

Algorithmically

Easy: 1/poly(n) (How?)

Answer: Pick any poly(n) colorings.

[Karmarkar-Karp81]: \approx 1/n^log n

Huge gap: Major open question

Remark: {-1,+1} not enough. Really need

color 0 also. 32/30


33/64

Yet another enhancement

There is a {-1,0,1} coloring with at least

n/2 {-1,1}s s.t. \sum_i a_i s_i \leq n/2^{n/5}

Make buckets of size 2n/2^{n/5}At least 2^{4n/5} sums fall in same bucket

Claim: Some two s and s in same bucket and differ in at

least n/2 coordinates

Again consider s = (s-s)/2

33/30


34/64

Proof of Claim

Claim: Any set of 2^{4n/5} vertices of the

boolean cube has

[Kleitman66] Isoperimetry for cube.

Hamming ball B(v,r) has the smallest

diameter for a given number of vertices.

|B(v,n/4)| < 2^{4n/5}34/30


35/64

Spencers proof

35/30


36/64

36/30

Our Approach


37/64

37/30

Algorithm (at high level)

Cube: {-1,+1}n

Analysis: Few steps to reach a vertex (walk has high variance)

Disc(Si) does a random walk (with low variance)

start

finish

Algorithm: Sticky random walk

Each step generated by rounding a suitable SDP

Move in various dimensions correlated, e.g. t

1+ t

2 0

Each dimension: An Element

Eachvertex: A Coloring


38/64

38/30

An SDP

Hereditary disc. ) the following SDP is feasible

SDP:

Low discrepancy: |i 2Sj

vi|2 2

|vi|2 = 1

Rounding:Pick random Gaussian g =(g1,g2,,gn)

each coordinate giis iid N(0,1)

For each i, consider i = gvi

Obtain vi2Rn


39/64

39/30

Properties of Rounding

Lemma:If g 2Rnis random Gaussian. For any v 2Rn,

g v is distributed as N(0, |v|2)

Pf: N(0,a2) + N(0,b2) = N(0,a2+b2) gv = i v(i) gi N(0, iv(i)2)

1. Each i N(0,1)

2. For each set S,

i 2Si = g (i2Svi) N(0, 2)(std deviation )

SDP:

|vi|2 = 1

|i2Svi|

22

Recall: i = g vi

s mimics a low discrepancy coloring (but is not {-1,+1})


40/64

40/30

Algorithm Overview

Construct coloring iteratively.

Initially: Start with coloring x0= (0,0,0, ,0) at t = 0.

At Time t: Update coloring as xt= xt-1+ (t1,,

tn)

(tiny: 1/n suffices)

x(i)

xt(i) = (1i+ 2i+ + ti)

Color of element i: Does random walk

over time with step size N(0,1)

Fixedif reaches -1 or +1.

time

-1

+1

Set S: xt(S) = i 2S xt(i) does a random walk w/ step N(0,2)


41/64

41/30

Analysis

Consider time T = O(1/2)

Claim 1:With prob. , at least n/2 elementsreach -1 or +1.Pf: Each element doing random walk with size .

Recall: Random walk with step 1, is O(t1/2) away in t steps.

A Trouble:Various element updates are correlated

Consider basic walk x(t+1) = x(t) 1 with prob

Define Energy (t)= x(t)2

E[(t+1)] = (x(t)+1)2+ (x(t)-1)2 = x(t)2+ 1 = (t)+1

Expected energy = n at t= n.

Claim 2: Each set has O() discrepancyin expectation.Pf: For each S, x

t(S) doing random walk with step size


42/64

42/30

Analysis

Consider time T = O(1/2)

Claim 1:With prob. , at least n/2 variablesreach -1 or +1.

) Everything colored in O(log n)rounds.

Claim 2: Each set has O() discrepancyin expectation per round.

) Expected discrepancy of a set at end = O(log n)

Thm: Obtain a coloring with discrepancy O(log (mn))Pf: By Chernoff, Prob. that disc(S) >= 2 Expectation + O(log m)

= O(log (mn))is tiny (poly(1/m)).


43/64

43/30

Recap

At each step of walk, formulate SDP on unfixed variables.

Use some (existential) property to argue SDP is feasible

Rounding SDP solution -> Step of walk

Properties of walk:

HighVariance -> Quick convergence

Lowvariance for discrepancy on sets -> Low discrepancy


44/64

44/30

Refinements

Spencers six std deviations result:

Goal: Obtain O(n1/2)discrepancy for any set system on m = O(n) sets.

Random coloring has n1/2(log n)1/2 discrepancy

Previous approach seems useless:

Expected discrepancy for a set O(n1/2),

but some random walks willdeviateby up to (log n)1/2factor

Need an additional idea to prevent this.


45/64

45/30

Spencers O(n1/2) result

Partial Coloring Lemma:For any system with m sets, there exists acoloring on n/2elements with discrepancy O(n1/2log1/2(2m/n))[For m=n, disc = O(n1/2)]

Algorithm for total coloring:

Repeatedly apply partial coloring lemma

Total discrepancy

O( n1/2 log1/22 ) [Phase 1]

+ O( (n/2)1/2log1/24 ) [Phase 2]

+ O((n/4)1/2log1/2 8 ) [Phase 3]

+ = O(n1/2)


46/64

46/30

Proving Partial Coloring Lemma

Beautiful Counting argument (entropy method + pigeonhole)Idea:Too many colorings (2n), but few discrepancy profiles

Key Lemma: There existk=24n/5colorings X1,,Xksuch that

everytwo Xi, Xj are similar foreveryset S1,,Sn.

Some X1,X2 differ on n/2positions

Consider X = (X1X2)/2

Pf: X(S) = (X1(S)X2(S))/2 2 [-10 n1/2, 10 n1/2]

X1= ( 1,-1, 1 , ,1,-1,-1)

X2= (-1,-1,-1, ,1, 1, 1)X = ( 1, 0, 1 , ,0,-1,-1)


47/64

47/30

A useful generalization

There exists a partial coloring with non-uniformdiscrepancy bound Sfor set S

Even if S = ( n1/2) in some average sense


48/64

48/30

An SDP

Suppose there existspartialcoloring X:

1. On n/2 elements2. Each set S has |X(S)| S

SDP:

Low discrepancy: |i 2Sj vi|

2 S2

Many colors: i |vi|2n/2

|vi|2 1

Pick random Gaussian g =(g1,g2,,gn)

each coordinate giis iid N(0,1)

For each i, consider i = g vi

Obtain vi2Rn

Al i h


49/64

49/30

Algorithm

Initially write SDP with S= c n1/2

Each set S does random walk and expectsto reach

discrepancy of O(S) = O(n1/2)

Some sets will becomeproblematic.

Reducetheir Son the fly.

Not many problematic sets, and entropy penalty low.

0 20n1/2 30n1/2 35n1/2

Danger 1 Danger 2 Danger 3


50/64

50/30

Concluding Remarks

Construct coloring over timeby solving sequence of SDPs(guided by existence results)

Works quite generally

Can be derandomized [Bansal-Spencer]

(use entropy method itself for derandomizing + usual tech.)

E.g. Deterministic six standard deviations can be viewed as a way to

derandomize something stronger than Chernoff bounds.


51/64

51/30

Thank You!


52/64

52/30


53/64

53/30


54/64

54/30

Rest of the talk

1. How to generate i with required properties.

2. How to update Sover time.

Show n1/2(log log log n)1/2bound.


55/64

55/30

Why so few algorithms?

Often algorithms rely on continuous relaxations.

Linear Program is useless. Can color each element

redand blue.

Improved results of Spencer, Beck, Srinivasan,

based on clever counting(entropy method).

Pigeonhole Principle on exponentially large systems(seems inherently non-constructive)

P ti l C l i L


56/64

56/30

Partial Coloring Lemma

Suppose we have discrepancy bound Sfor set S.

Consider 2n possible colorings

Signatureof a coloring X: (b(S1), b(S2),, b(Sm))

Want partial coloring with signature (0,0,0,,0)


57/64

57/30

Progress Condition

Energy increases at each step:

E(t) = \sum_i x_i(t)^2

Initially energy =0, can be at most n.

Expected value of E(t) = E(t-1) + \sum_i

\gamma_i(t)^2

Markovs inequality.


58/64

58/30

Missing Steps

1. How to generate the \eta_i

2. How to update \Delta_S over time

P ti l C l i


59/64

59/30

Partial Coloring

If exist two colorings X1,X2

1. Samesignature (b1,b2,,bm)

2. Differ in at least n/2positions.

Consider X = (X1X2)/21. -1 or 1 on at least n/2positions, i.e.partial coloring

2. Has signature (0,0,0,,0)

X(S) = (X1(S)X2(S)) / 2, so |X(S)| S for all S.

Can show that there are 24n/5colorings with same signature.

So, some twowill differ on > n/2positions. (Pigeon Hole)

X1= (1,-1, 1 , , 1,-1,-1)

X2= (-1,-1,-1, , 1,1, 1)


60/64

60/30

S O( 1/2) lt


61/64

61/30

Spencers O(n1/2) result

Partial Coloring Lemma:For any system with m sets,

there exists a coloring on n/2elements withdiscrepancy O(n1/2log1/2(2m/n))

[For m=n, disc = O(n1/2)]

Algorithm for total coloring:

Repeatedly apply partial coloring lemma

Total discrepancy

O( n1/2

log1/2

2 ) [Phase 1]+ O( (n/2)1/2log1/24 ) [Phase 2]

+ O((n/4)1/2log1/2 8 ) [Phase 3]

+ = O(n1/2)Let us prove the lemma for m = n

P i P ti l C l i L


62/64

62/30

Proving Partial Coloring Lemma

Pf: Associate with coloring X, signature = (b1,b2,,bn)

(bi= bucket in which X(Si) lies )

Wish to show: There exist 24n/5colorings with same signature

Choose X randomly: Induces distribution on signatures.

Entropy () n/5 implies some signature hasprob. 2-n/5.

Entropy () iEntropy( bi) [Subadditivity of Entropy]

bi = 0 w.p. 1- 2 e-50,

= 1 w.p. e-50

= 2 w.p. e-450

-10 n1/2-30 n1/2 10 n1/2 30 n1/2

0 21-1-2

Ent(b1) 1/5


63/64

63/30

A useful generalization

Partial coloring with non-uniformdiscrepancy Sfor set S

S-3S -S 3S 5S

0 1 2-1-2

For each set S, consider the bucketing

Suffices to have s Ent (bs) n/5

Or, if S= sn1/2, then s g(s) n/5

g() e-2/2 > 1

ln(1/) < 1

Bucket of n1/2/100has penalty ln(100)

Recap


64/64

Recap

Partial Coloring: S 10 n1/2 gives low entropy) 24n/5 colorings exist with same signature.

) some X1,X2with large hamming distance.

(X1X2) /2 gives the desired partial coloring.

Trouble: 24n/5/2n is an exponentially small fraction.

Only if we could find the partial coloring

efficiently

Date post:	03-Jun-2018
Category:	Documents
Upload:	daniel-situngkir
View:	230 times
Download:	0 times

Discrepancy Berlin

Documents