Divide and Conquer II

Lecture slides by Kevin WayneCopyright © 2005 Pearson-Addison Wesley

http://www.cs.princeton.edu/~wayne/kleinberg-tardos

Last updated on 3/26/18 9:51 AM

DIVIDE AND CONQUER II

‣ master theorem

‣ integer multiplication

‣ matrix multiplication

‣ convolution and FFT


‣ master theorem




SECTIONS 4.4–4.6

Divide-and-conquer recurrences

Goal. Recipe for solving common divide-and-conquer recurrences: with T(0) = 0 and T(1) = Θ(1).

Terms.

独a ≥ 1 is the number of subproblems.

独b ≥ 2 is the factor by which the subproblem size decreases.

独f (n) ≥ 0 is the work to divide and combine subproblems.

Recursion tree. [ assuming n is a power of b ]

独a = branching factor.

独ai = number of subproblems at level i.

独1 + logb n levels.

独n / bi = size of subproblem at level i.

3

T (n) = a T�n

b

�+ f(n)

T (n)

T (n / b) T (n / b) T (n / b)...

.........

Fact 1. If r ≠ 1, then

Fact 2. If r = 1, then

Fact 3. If ⎥ r⎥ < 1, then 1 + r + r2 + r3 + . . . =

1

1 � r

Geometric series

4

1 + r + r2 + r3 + . . . + rk�1 = k

1 + r + r2 + r3 + . . . + rk�1 =1 � rk

1 � r

1

1/2

1/4

1/8

1/161/32

1 + 1/2 +1/4 + 1/8 + … = 2

Case 1: total cost dominated by cost of leaves

Ex 1. If T (n) satisfies T (n) = 3 T (n / 2) + n, with T (1) = 1, then T (n) = .

5

1 + log2 n

n

3 (n / 2)

3i (n / 2i)

⋮

32 (n / 22)

T (n)

T (n / 2) T (n / 2) T (n / 2)

T (n / 4)T (n / 4) T (n / 4) T (n / 4)T (n / 4) T (n / 4) T (n / 4)T (n / 4) T (n / 4)

⋮

T (1) T (1)T (1) T (1) T (1) T (1) T (1) T (1)T (1) T (1) T (1) ... T (1)T (1)

r = 3 / 2 > 1

3log2 n = nlog2 3

r1+log2 n � 1

r � 1n = 3nlog2 3 � 2n

3log2 n(n / 2log2 n)

T (n) = (1 + r + r2 + r3 + . . . + rlog2 n) n =

�(nlog2 3)

Case 2: total cost evenly distributed among levels

Ex 2. If T (n) satisfies T (n) = 2 T (n / 2) + n, with T (1) = 1, then T (n) = Θ(n log n).

6

1 + log2 n

n

2 (n / 2)

23 (n / 23)

⋮

22 (n / 22)

T (n)

T (n / 2) T (n / 2)

T (n / 8) T (n / 8)T (n / 8) T (n / 8) T (n / 8) T (n / 8)T (n / 8) T (n / 8)

T (n / 4) T (n / 4) T (n / 4) T (n / 4)

⋮

T (1) T (1)T (1) T (1) T (1) T (1) T (1) T (1)T (1) T (1) T (1) ... T (1)T (1)

n (1)

= n (log2 n + 1)r = 1

2log2 n = n

T (n) = (1 + r + r2 + r3 + . . . + rlog2 n) n

Case 3: total cost dominated by cost of root

Ex 3. If T (n) satisfies T (n) = 3 T (n / 4) + n5, with T (1) = 1, then T (n) = Θ(n5).

7

1 + log4 n

n5

3 (n / 4)5

3i (n / 4i)5

⋮

32 (n / 42)5

T (n)

T (n / 4) T (n / 4) T (n / 4)

T (n / 16)T (n / 16) T (n / 16) T (n / 16)T (n / 16) T (n / 16) T (n / 16)T (n / 16) T (n / 16)

⋮

T (1) T (1)T (1) T (1) T (1) T (1) T (1) T (1)T (1) T (1) T (1) ... T (1)T (1)

n5 ≤ T (n) ≤ (1 + r + r 2 + r 3 + … ) n5 ≤r = 3 / 45 < 1 1 – r1

n 5

3log4 n = nlog4 3

3log4 n(n/4log4 n)5

Master theorem (divide-and-conquer)

Master theorem. Suppose that T (n) is a function on the non-negative

integers that satisfies the recurrence with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then,

Case 1. If f (n) = O(nlogba−ε) for some constant ε > 0, then T (n) = Θ(nlogba). Ex. T (n) = 3 T(n / 2) + 5 n.

独a = 3, b = 2, logb a = 1.58…, f (n) = 5 n, ε = 0.58. 独T(n) = Θ(nlogba) = Θ(nlog23).

8

T (n) = a T�n

b

�+ f(n)




Case 2. If f (n) = Θ(nlogba), then T (n) = Θ(nlogba log n). Ex. T (n) = 2 T(⎡n / 2⎤) + 17 n.

独a = 2, b = 2, logb a = 1, f (n) = 17 n.

独T (n) = Θ(n log n).

9

T (n) = a T�n

b

�+ f(n)




Case 3. If f (n) = Ω(nlogba+ε) for some constant ε > 0, and if a f (n / b) ≤ c f (n) for

some constant c < 1 and all sufficiently large n, then T (n) = Θ ( f (n)). Ex. T (n) = 3 T(n / 4) + n5

.

独a = 3, b = 4, logb a = 0.7924…, f (n) = n5.

独Regularity condition: 3(n / 4)5 ≤ c n5 for c = 3/45 < 1.

独T (n) = Θ ( f (n)) = Θ(n5).

10

T (n) = a T�n

b

�+ f(n)

“regularity condition”

holds if f(n) = Θ(nk)




Case 1. If f (n) = O(nlogba−ε) for some constant ε > 0, then T (n) = Θ(nlogba). Case 2. If f (n) = Θ(nlogba), then T (n) = Θ(nlogba log n).

Case 3. If f (n) = Ω(nlogba+ε) for some constant ε > 0, and if a f (n / b) ≤ c f (n) for some constant c < 1 and all sufficiently large n, then T (n) = Θ ( f (n)).

Pf sketch.

独Use recursion tree to sum up terms (assuming n is an exact power of b).

独Three cases for geometric series.

独Deal with floors and ceilings.11

T (n) = a T�n

b

�+ f(n)

at most 2 extra levels in recursion tree

Consider the following recurrence. Which case of the master theorem?

A. Case 1: T(n) = Θ(nlog23) = O(n1.585).

B. Case 2: T(n) = Θ(n log n).

C. Case 3: T(n) = Θ(n).

D. Master theorem not applicable.

12

Divide-and-conquer II: quiz 1

T (n) =

��

�(1) B7 n = 1

3T (�n/2�) + �(n) B7 n > 1<latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit>

Consider the following recurrence. Which case of the master theorem?

A. Case 1: T(n) = Θ(n).

B. Case 2: T(n) = Θ(n log n).

C. Case 3: T(n) = Θ(n).

D. Master theorem not applicable.

13


C(n) =

��

0 B7 n � 1

C(�n/5�) + C(n � 3�n/10�) + 115 n B7 n > 1

<latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit><latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit><latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit><latexit sha1_base64="Y96HaCEvN4aOrD6hbv5tphcOY0Y=">AAAC0nicbVHLbtNAFB2bVwmPpmXJ5ooIlAoRbCCiKKJUyoZlkTCtlLGi8fg6HXU8tmbGKJGVBWLLj/BJ/A1j1xJNwl0dnXOf5yalFMYGwR/Pv3X7zt17e/d7Dx4+erzfPzj8ZopKc4x4IQt9kTCDUiiMrLASL0qNLE8knidX00Y//47aiEJ9tasS45wtlMgEZ9ZR8/7v6VAdAZ18pJMeTXAhVM1dO7Pu0UkAL4BaXFqoQWSwBgVUIoRA6SzEPHYp0yGVmSwKDer1GKhu8RG8BNcWXsFb+CeHwU2dZprxOgzX9bjpuzPopB3To6jSbqF5fxCMgjZgF4QdGJAuzuYH3iFNC17lqCyXzJhZGJQ2rpm2gkt0F1YGS8av2AJnDiqWo4nr1tM1PHdMCplbPSuUhZa9WVGz3JhVnrjMnNlLs6015P+0WWWz47gWqqwsKn49KKsk2AKaB0EqNHIrVw4wroXbFfglc25Z98aNKW3vEvnGJfWyUoIXKW6x0i6tZo2L4bZnuyB6M/owCr+8G5wed3bukafkGRmSkLwnp+QzOSMR4d6+N/ZOvE9+5Nf+D//ndarvdTVPyEb4v/4Ck0DZRw==</latexit>

Master theorem need not apply

Gaps in master theorem.

独Number of subproblems is not a constant.

独Number of subproblems is less than 1.

独No polynomial separation between f(n) and nlogba.

独f(n) is not positive.

独Regularity condition does not hold.

14

T (n) = nT (n/2) + n2

T (n) =1

2T (n/2) + n2

T (n) = 2 T (n/2) � n2

T (n) = T (n/2) + n(2 � cos n)

T (n) = 2 T (n/2) + n log n<latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit><latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit><latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit><latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit>

Akra–Bazzi theorem

Theorem. [Akra–Bazzi 1998] Given constants ai > 0 and 0 < bi < 1,functions ⎜hi (n)⎜ = O(n / log2 n) and g(n) = O(nc). If T(n) satisfies the recurrence: then, T(n) = , where p satisfies . Ex. T(n) = T(⎣n / 5⎦) + T(n – 3⎣n / 10⎦) + 11/5 n, with T(0) = 0 and T(1) = 0.

独a1 = 1, b1 = 1/5, a2 = 1, b2 = 7/10 ⇒ p = 0.83978… < 1.

独h1(n) = ⎣n / 5⎦ – n / 5, h2(n) = 3/10 n – 3⎣n / 10⎦.

独g(n) = 11/5 n ⇒ T(n) = Θ(n).

15

T (n) =

k�

i=1

ai T (bin + hi(n)) + g(n)

k�

i=1

ai bpi = 1�

�np

�1 +

� n

1

g(u)

up+1du

��

ai subproblems

of size bi nsmall perturbation to handle

floors and ceilings


‣ master theorem




SECTION 5.5

Integer addition and subtraction

Addition. Given two n-bit integers a and b, compute a + b.

Subtraction. Given two n-bit integers a and b, compute a – b.

Grade-school algorithm. Θ(n) bit operations.

Remark. Grade-school addition and subtraction algorithms are optimal.

17

1 1 1 1 1 1 0 1

1 1 0 1 0 1 0 1

+ 0 1 1 1 1 1 0 1

1 0 1 0 1 0 0 1 0

“bit complexity”

(instead of word RAM)

Integer multiplication

Multiplication. Given two n-bit integers a and b, compute a × b.

Grade-school algorithm (long multiplication). Θ(n2) bit operations.

Conjecture. [Kolmogorov 1956] Grade-school algorithm is optimal.

Theorem. [Karatsuba 1960] Conjecture is false.18

1 1 0 1 0 1 0 1

× 0 1 1 1 1 1 0 1

1 1 0 1 0 1 0 1

0 0 0 0 0 0 0 0

1 1 0 1 0 1 0 1

1 1 0 1 0 1 0 1

1 1 0 1 0 1 0 1

1 1 0 1 0 1 0 1

1 1 0 1 0 1 0 1

0 0 0 0 0 0 0 0

0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1

Divide-and-conquer multiplication

To multiply two n-bit integers x and y:

独Divide x and y into low- and high-order bits.

独Multiply four ½n-bit integers, recursively.

独Add and shift to obtain result.

19

1

x y = (2m a + b) (2m c + d) = 22m ac + 2m (bc + ad) + bd2 3 4

c = ⎣ y / 2m ⎦ d = y mod 2m

m = ⎡ n / 2 ⎤

Ex. x = 1 0 0 0 1 1 0 1 y = 1 1 1 0 0 0 0 1

a b c d

use bit shifting

to compute 4 terms

a = ⎣ x / 2m ⎦ b = x mod 2m

Divide-and-conquer multiplication

20

MULTIPLY(x, y, n) _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (n = 1)RETURN x ! y.

ELSE

m ← ⎡ n / 2 ⎤.a ← ⎣ x / 2m ⎦; b ← x mod 2m.c ← ⎣ y / 2m ⎦; d ← y mod 2m.e ← MULTIPLY(a, c, m).f ← MULTIPLY(b, d, m).g ← MULTIPLY(b, c, m).h ← MULTIPLY(a, d, m).RETURN 22m e + 2m (g + h) + f.

_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Θ(n)

4 T(⎡n / 2⎤)

Θ(n)

How many bit operations to multiply two n-bit integers using the divide-and-conquer multiplication algorithm?

A. T(n) = Θ(n1/2).

B. T(n) = Θ(n log n).

C. T(n) = Θ(nlog23) = O(n1.585).

D. T(n) = Θ(n2).

21


T (n) =

��

�(1) B7 n = 1

4T (�n/2�) + �(n) B7 n > 1<latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit><latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit><latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit><latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit>

Case 1 of master theorem

Karatsuba trick

To multiply two n-bit integers x and y:

独Divide x and y into low- and high-order bits.

独To compute middle term bc + ad, use identity:

独Multiply only three ½n-bit integers, recursively.

22

bc + ad = ac + bd – (a – b) (c – d)

a = ⎣ x / 2m ⎦ b = x mod 2m

c = ⎣ y / 2m ⎦ d = y mod 2m

m = ⎡ n / 2 ⎤

1 231 3

x y = (2m a + b) (2m c + d) = 22m ac + 2m (bc + ad ) + bd

x = 1 0 0 0 1 1 0 1

a b

y = 1 1 1 0 0 0 0 1

c d

middle term

= 22m ac + 2m (ac + bd – (a – b)(c – d)) + bd

Karatsuba multiplication

23

KARATSUBA-MULTIPLY(x, y, n) _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (n = 1)RETURN x ! y.

ELSE

m ← ⎡ n / 2 ⎤.a ← ⎣ x / 2m ⎦; b ← x mod 2m.c ← ⎣ y / 2m ⎦; d ← y mod 2m.e ← KARATSUBA-MULTIPLY(a, c, m).f ← KARATSUBA-MULTIPLY(b, d, m).g ← KARATSUBA-MULTIPLY(⎢a – b ⎢, ⎢c – d ⎢, m).Flip sign of g if needed.RETURN 22m e + 2m (e + f – g) + f.

_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Θ(n)

3 T(⎡n / 2⎤)

Θ(n)

Karatsuba analysis

Proposition. Karatsuba’s algorithm requires O(n1.585) bit operations to

multiply two n-bit integers.

Pf. Apply Case 1 of the master theorem to the recurrence:

Practice.

独Use base 32 or 64 (instead of base 2).

独Faster than grade-school algorithm for about 320–640 bits.

24

T (n) = 3T (n/2) + �(n) =� T (n) = �(nlog2 3) = O(n1.585)<latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit><latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit><latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit><latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit>

T (n) =

��

�(1) B7 n = 1

3T (�n/2�) + �(n) B7 n > 1<latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit>

Integer arithmetic reductions

Integer multiplication. Given two n-bit integers, compute their product.

25

arithmetic problem formula bit complexity

integer multiplication a × b M(n)

integer square a2 Θ(M(n))

integer division ⎣a / b⎦, a mod b Θ(M(n))

integer square root ⎣√a ⎦ Θ(M(n))

integer arithmetic problems with the same bit complexity M(n) as integer multiplication

ab =(a + b)2 � a2 � b2

2<latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit><latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit><latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit><latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit>

History of asymptotic complexity of integer multiplication

Remark. GNU Multiple Precision library uses one of first

five algorithms depending on n.

26

year algorithm bit operations

12xx grade school O (n2)

1962 Karatsuba–Ofman O(n1.585)

1963 Toom-3, Toom-4 O (n1.465), O (n1.404)

1966 Toom–Cook O (n1 + ε)

1971 Schönhage–Strassen O (n log n ⋅ log log n)

2007 Fürer n log n 2O(log*n)

2018 Harvey–van der Hoeven O(n log n ⋅ 22 lg* n )

O(n)

number of bit operations to multiply two n-bit integers

used in Maple, Mathematica, gcc, cryptography, ...


‣ master theorem




SECTION 4.2

Dot product

Dot product. Given two length-n vectors a and b, compute c = a ⋅ b.

Grade-school. Θ(n) arithmetic operations.

Remark. “Grade-school” dot product algorithm is asymptotically optimal.

28

€

a = .70 .20 .10[ ]b = .30 .40 .30[ ]a ⋅ b = (.70 × .30) + (.20 × .40) + (.10 × .30) = .32

a · b =

n�

i=1

ai bi

Matrix multiplication

Matrix multiplication. Given two n-by-n matrices A and B, compute C = AB.

Grade-school. Θ(n3) arithmetic operations.

Q. Is “grade-school” matrix multiplication algorithm asymptotically optimal?

29

€

c11 c12 ! c1n

c21 c22 ! c2n

" " # "cn1 cn2 ! cnn

"

#

$ $ $ $

%

&

' ' ' '

=

a11 a12 ! a1n

a21 a22 ! a2n

" " # "an1 an2 ! ann

"

#

$ $ $ $

%

&

' ' ' '

×

b11 b12 ! b1n

b21 b22 ! b2n

" " # "bn1 bn2 ! bnn

"

#

$ $ $ $

%

&

' ' ' '

€

.59 .32 .41

.31 .36 .25

.45 .31 .42

"

#

$ $ $

%

&

' ' '

=

.70 .20 .10

.30 .60 .10

.50 .10 .40

"

#

$ $ $

%

&

' ' '

× .80 .30 .50.10 .40 .10.10 .30 .40

"

#

$ $ $

%

&

' ' '

cij =

n�

k=1

aik bkj

Block matrix multiplication

30

€

C11

= A11 ×B11 + A12 ×B21 = 0 14 5#

$ %

&

' ( ×

16 1720 21#

$ %

&

' ( +

2 36 7#

$ %

&

' ( ×

24 2528 29#

$ %

&

' ( =

152 158504 526#

$ %

&

' (

€

152 158 164 170504 526 548 570856 894 932 970

1208 1262 1316 1370

"

#

$ $ $ $

%

&

' ' ' '

=

0 1 2 34 5 6 78 9 10 11

12 13 14 15

"

#

$ $ $ $

%

&

' ' ' '

×

16 17 18 1920 21 22 2324 25 26 2728 29 30 31

"

#

$ $ $ $

%

&

' ' ' '

C11A11 A12 B11

B21

Block matrix multiplication: warmup

To multiply two n-by-n matrices A and B:

独Divide: partition A and B into ½n-by-½n blocks.

独Conquer: multiply 8 pairs of ½n-by-½n matrices, recursively.

独Combine: add appropriate products using 4 matrix additions.

Running time. Apply Case 1 of the master theorem.

31

€

C11 = A11 × B11( ) + A12 × B21( )C12 = A11 × B12( ) + A12 × B22( )C21 = A21 × B11( ) + A22 × B21( )C22 = A21 × B12( ) + A22 × B22( )

€

T (n) = 8T n /2( )recursive calls! " # $ #

+ Θ(n2 )add, form submatrices! " # # $ # # ⇒ T (n) =Θ(n3)

€

C11 C12

C21 C22

"

# $

%

& ' =

A11 A12

A21 A22

"

# $

%

& ' ×

B11 B12

B21 B22

"

# $

%

& '

½n-by-½n matrices

C = A � B

n-by-n matrices8 matrix multiplications(of ½n-by-½n matrices)

4 matrix additions(of ½n-by-½n matrices)

Strassen’s trick

Key idea. Can multiply two 2-by-2 matrices via 7 scalar multiplications (plus 11 additions and 7 subtractions).

Pf. C12 = P1 + P2

= A11 ! (B12 – B22) + (A11 + A12) ! B22

= A11 ! B12 + A12 ! B22. ✔32

€

C11 C12

C21 C22

"

# $

%

& ' =

A11 A12

A21 A22

"

# $

%

& ' ×

B11 B12

B21 B22

"

# $

%

& '

P1 ← A11 ! (B12 – B22)

P2 ← (A11 + A12) ! B22

P3 ← (A21 + A22) ! B11

P4 ← A22 ! (B21 – B11)

P5 ← (A11 + A22) ! (B11 + B22)

P6 ← (A12 – A22) ! (B21 + B22)

P7 ← (A11 – A21) ! (B11 + B12)

C11 = P5 + P4 – P2 + P6

C12 = P1 + P2

C21 = P3 + P4

C22 = P1 + P5 – P3 – P7

7 scalar multiplications

scalars

Strassen’s trick

Key idea. Can multiply two 2-by-2 matrices via 7 scalar multiplications (plus 11 additions and 7 subtractions).

Pf. C12 = P1 + P2

= A11 ! (B12 – B22) + (A11 + A12) ! B22

= A11 ! B12 + A12 ! B22. ✔33

€

C11 C12

C21 C22

"

# $

%

& ' =

A11 A12

A21 A22

"

# $

%

& ' ×

B11 B12

B21 B22

"

# $

%

& '

P1 ← A11 ! (B12 – B22)

P2 ← (A11 + A12) ! B22

P3 ← (A21 + A22) ! B11

P4 ← A22 ! (B21 – B11)

P5 ← (A11 + A22) ! (B11 + B22)

P6 ← (A12 – A22) ! (B21 + B22)

P7 ← (A11 – A21) ! (B11 + B12)

C11 = P5 + P4 – P2 + P6

C12 = P1 + P2

C21 = P3 + P4

C22 = P1 + P5 – P3 – P7

7 matrix multiplications(of ½n-by-½n matrices)

½n-by-½n matrices

n-by-n ½n-by-½n matrix

Strassen’s algorithm

34

STRASSEN(n, A, B) ______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (n = 1) RETURN A ! B.Partition A and B into ½n-by-½n blocks.P1 ← STRASSEN(n / 2, A11, (B12 – B22)).P2 ← STRASSEN(n / 2, (A11 + A12), B22).P3 ← STRASSEN(n / 2, (A21 + A22), B11).

P4 ← STRASSEN(n / 2, A22, (B21 – B11)).P5 ← STRASSEN(n / 2, (A11 + A22), (B11 + B22)).P6 ← STRASSEN(n / 2, (A12 – A22), (B21 + B22)).P7 ← STRASSEN(n / 2, (A11 – A21), (B11 + B12)).C11 = P5 + P4 – P2 + P6.C12 = P1 + P2.C21 = P3 + P4.C22 = P1 + P5 – P3 – P7.RETURN C.

€

C11 C12

C21 C22

"

# $

%

& ' =

A11 A12

A21 A22

"

# $

%

& ' ×

B11 B12

B21 B22

"

# $

%

& '

7 T(n / 2) + Θ(n2)

Θ(n2)

assume n is a power of 2

Analysis of Strassen’s algorithm

Theorem. Strassen’s algorithm requires O(n2.81) arithmetic operations to

multiply two n-by-n matrices.

35

Numer. Math. t3, 354--356 (t969)

Gaussian Elimination is not Optimal VOLKER ~TRASSEN *

Received December 12, t 968

t. Below we will give an algorithm which computes the coefficients of the product of two square matrices A and B of order n from the coefficients of A and B with tess than 4 . 7 - n l°g7 arithmetical operations (all logarithms in this paper are for base 2, thus tog 7 ~ 2.8; the usual method requires approximately 2n 3 arithmetical operations). The algorithm induces algorithms for invert ing a matr ix of order n, solving a system of n linear equations in n unknowns, com- put ing a determinant of order n etc. all requiring less than const n l°g 7 arithmetical operations.

This fact should be compared with the result of KLYUYEV and KOKOVKIN- SHCHERBAK [1 ] tha t Gaussian elimination for solving a system of l inearequations is optimal if one restricts oneself to operations upon rows and columns as a whole. We also note tha t WlNOGRAD [21 modifies the usual algorithms for matr ix multiplication and inversion and for solving systems of linear equations, trading roughly half of the multiplications for additions and subtractions.

I t is a pleasure to thank D. BRILLINGER for inspiring discussions about the present subject and ST. COOK and B. PARLETT for encouraging me to write this paper.

2. We define algorithms e~, ~ which mult iply matrices of order m2 ~, by in- duction on k: ~ , 0 is the usual algorithm, for matr ix multiplication (requiring m a multiplications and m 2 ( m - t) additions), e~,k already being known, define ~ , ~ +t as follows:

If A, B are matrices of order m 2 k ~ to be multiplied, write

(All A~2 t (B~I B12~ (Cll C1~ A = \A21 A~2], B --- \B.21 B2~], A B = \C21 C22],

where the Ai~ ,, Bik, Cik are matrices of order m2 ~. Then compute

I = (All + A ~ ) (Bit + Be2 ) , I I = ( A ~ I + A ~ 2 )Bl I , I I I = A l l (Bt~ - - B~.~), IV = A ~ ( - - Bl l + B21 ), V = (Alx+AI~)B~2, VI = (--Ax~ + A~) (Bll + B~2 ), VII = (AI~--A~2 ) (B21 + B22 ),

* The results have been found while the author was at the Department of Statistics of the University of California, Berkeley. The author wishes to thank the National Science Foundation for their support (NSF GP-7454).

Analysis of Strassen’s algorithm

Theorem. Strassen’s algorithm requires O(n2.81) arithmetic operations to

multiply two n-by-n matrices.

Pf.

独When n is a power of 2, apply Case 1 of the master theorem:

独When n is not a power of 2, pad matrices with zeros to be nʹ-by-nʹ,where n ≤ nʹ < 2n and nʹ is a power of 2.

36

€

T (n) = 7T n /2( )recursive calls! " # $ #

+ Θ(n2 )add, subtract! " # $ # ⇒ T (n) =Θ(n log2 7 ) =O(n2.81)

��

1 2 3 04 5 6 07 8 9 00 0 0 0

��

��

10 11 12 013 14 15 016 17 18 00 0 0 0

�� =

��

84 90 96 0201 216 231 0318 342 366 00 0 0 0

��

Strassen’s algorithm: practice

Implementation issues.

独Sparsity.

独Caching.

独n not a power of 2.

独Numerical stability.

独Non-square matrices.

独Storage for intermediate submatrices.

独Crossover to classical algorithm when n is “small.”

独Parallelism for multi-core and many-core architectures.

Common misperception. “Strassen’s algorithm is only a theoretical curiosity.”独Apple reports 8x speedup when n ≈ 2,048.

独Range of instances where it’s useful is a subject of controversy.

37

SC16; Salt Lake City, Utah, USA; November 2016978-1-4673-8815-3/16/$31.00 c� 2016 IEEE

Strassen’s Algorithm ReloadedJianyu Huang⇤, Tyler M. Smith⇤†, Greg M. Henry‡, Robert A. van de Geijn⇤†

⇤Department of Computer Science and †Institute for Computational Engineering and Sciences,The University of Texas at Austin, Austin, TX 78712

Email: jianyu,tms,[email protected]‡Intel Corporation, Hillsboro, OR 97124

Email: [email protected]

Abstract—We dispel with “street wisdom” regarding thepractical implementation of Strassen’s algorithm for matrix-matrix multiplication (DGEMM). Conventional wisdom: it is onlypractical for very large matrices. Our implementation is practicalfor small matrices. Conventional wisdom: the matrices beingmultiplied should be relatively square. Our implementation ispractical for rank-k updates, where k is relatively small (a shapeof importance for libraries like LAPACK). Conventional wisdom:it inherently requires substantial workspace. Our implementationrequires no workspace beyond buffers already incorporatedinto conventional high-performance DGEMM implementations.Conventional wisdom: a Strassen DGEMM interface must passin workspace. Our implementation requires no such workspaceand can be plug-compatible with the standard DGEMM inter-face. Conventional wisdom: it is hard to demonstrate speedupon multi-core architectures. Our implementation demonstratesspeedup over conventional DGEMM even on an Intel R� XeonPhiTM coprocessor1 utilizing 240 threads. We show how a dis-tributed memory matrix-matrix multiplication also benefits fromthese advances.

Index Terms—Strassen, numerical algorithm, performancemodel, matrix multiplication, linear algebra library, BLAS.

I. INTRODUCTION

Strassen’s algorithm (STRASSEN) [1] for matrix-matrixmultiplication (DGEMM) has fascinated theoreticians and prac-titioners alike since it was first published, in 1969. That paperdemonstrated that multiplication of n ⇥ n matrices can beachieved in less than the O(n3) arithmetic operations requiredby a conventional formulation. It has led to many variants thatimprove upon this result [2], [3], [4], [5] as well as practicalimplementations [6], [7], [8], [9]. The method can yield ashorter execution time than the best conventional algorithmwith a modest degradation in numerical stability [10], [11],[12] by only incorporating a few levels of recursion.

From 30,000 feet the algorithm can be described as shiftingcomputation with submatrices from multiplications to addi-tions, reducing the O(n3) term at the expense of adding O(n2)complexity. For current architectures, of greater consequenceis the additional memory movements that are incurred whenthe algorithm is implemented in terms of a conventionalDGEMM provided by a high-performance implementationthrough the Basic Linear Algebra Subprograms (BLAS) [13]interface. A secondary concern has been the extra workspacethat is required. This simultaneously limits the size of problem

1Intel, Xeon, and Intel Xeon Phi are trademarks of Intel Corporation in theU.S. and/or other countries.

that can be computed and makes it so an implementation is notplug-compatible with the standard calling sequence supportedby the BLAS.

An important recent advance in the high-performance imple-mentation of DGEMM is the BLAS-like Library InstantiationSoftware (BLIS framework) [14], a careful refactoring of thebest-known approach to implementing conventional DGEMMintroduced by Goto [15]. Of importance to the present paperare the building blocks that BLIS exposes, minor modifica-tions of which support a new approach to implementatingSTRASSEN. This approach changes data movement betweenmemory layers and can thus mitigate the negative impact ofthe additional lower order terms incurred by STRASSEN. Thesebuilding blocks have similarly been exploited to improve uponthe performance of, for example, the computation of theK-Nearest Neighbor [16] and Tensor Contraction [17], [18]problem. The result is a family of STRASSEN implementations,members of which attain superior performance depending onthe sizes of the matrices.

The resulting family improves upon prior implementationsof STRASSEN in a number of surprising ways:

• It can outperform classical DGEMM even for small squarematrices.

• It can achieve high performance for rank-k updates(DGEMM with a small “inner matrix size”), a case ofDGEMM frequently encountered in the implementation oflibraries like LAPACK [19].

• It needs not require additional workspace.• It can incorporate directly the multi-threading in tradi-

tional DGEMM implementations.• It can be plug-compatible with the standard DGEMM

interface supported by the BLAS.• It can be incorporated into practical distributed memory

implementations of DGEMM.Most of these advances run counter to conventional wisdomand are backed up by theoretical analysis and practical imple-mentation.

II. STANDARD MATRIX-MATRIX MULTIPLICATION

We start by discussing naive computation of matrix-matrixmultiplication (DGEMM), how it is supported as a library rou-tine by the Basic Linear Algebra Subprograms (BLAS) [13],how modern implementations block for caches, and how thatimplementation supports multi-threaded parallelization.

Suppose that you could multiply two 3-by-3 matrices with 21 scalar multiplications. How fast could you multiply two n-by-n matrices?

A. Θ(n log321)

B. Θ(n log221)

C. Θ(n log921)

D. Θ(n 2)

38


€

Θ (n log3 21) = O(n 2.77 )

Is it possible to multiply two 3-by-3 matrices using only 21 scalar multiplications?

A. Yes.

B. No.

C. Unknown.

39

Divide-and-conquer II: quiz 5 Fast matrix multiplication: theory

Q. Multiply two 2-by-2 matrices with 7 scalar multiplications?

A. Yes! [Strassen 1969] Θ(nlog27 ) = O(n2.81) Q. Multiply two 2-by-2 matrices with 6 scalar multiplications?

A. Impossible. [Hopcroft–Kerr, Winograd 1971] Θ(nlog26 ) = O(n2.59) Begun, the decimal wars have. [Pan 1978, Bini et al., Schönhage, …]

独Two 70-by-70 matrices with 143,640 scalar multiplications. O(n2.7962)

独Two 48-by-48 matrices with 47,217 scalar multiplications. O(n2.7801)

独A year later. O(n2.7799)

独December 1979. O(n2.521813)

独January 1980. O(n2.521801)

40

History of arithmetic complexity of matrix multiplication

41

year algorithm arithmetic operations

1858 “grade school” O (n3 )

1969 Strassen O (n2.808 )

1978 Pan O (n2.796 )

1979 Bini O (n2.780 )

1981 Schönhage O (n2.522 )

1982 Romani O (n2.517 )

1982 Coppersmith–Winograd O (n2.496 )

1986 Strassen O (n2.479 )

1989 Coppersmith–Winograd O (n2.3755 )

2010 Strother O (n2.3737 )

2011 Williams O (n2.372873 )

2014 Le Gall O (n2.372864 )

O (n2 + ε )

number of arithmetic operations to multiply two n-by-n matrices

galactic

algorithms

Numeric linear algebra reductions

Matrix multiplication. Given two n-by-n matrices, compute their product.

42

linear algebra problem expression arithmetic complexity

matrix multiplication A × B MM(n)

matrix squaring A2 Θ(MM(n))

matrix inversion A–1 Θ(MM(n))

determinant ⎢A ⎢ Θ(MM(n))

rank rank(A) Θ(MM(n))

system of linear equations Ax = b Θ(MM(n))

LU decomposition A = L U Θ(MM(n))

least squares min ⎢⎢Ax – b ⎢⎢2 Θ(MM(n))

numerical linear algebra problems with the samearithmetic complexity MM(n) as matrix multiplication


‣ master theorem




SECTION 5.6

Fourier theorem. [Fourier, Dirichlet, Riemann] Any (sufficiently smooth)

periodic function can be expressed as the sum of a series of sinusoids.

n = 1n = 5n = 10n = 100

Fourier analysis

44

t

y(t) =2

�

n�

k=1

sin kt

k<latexit sha1_base64="Wx3hIxObyD9jVmjGbChXYBtOR90=">AAACZXicbVBda9swFFXcfWTdliXZ6MseJhYGHYxgh8EyukBgL33sYFkLcRZk5boVlmUjXY8E4T/TX9PX9m3/YD9jsuOHJd0BicM59+rqniiXwqDv/255Bw8ePnrcfnL49Nnzzotur//DZIXmMOOZzPRFxAxIoWCGAiVc5BpYGkk4j5KvlX/+C7QRmfqOmxwWKbtUIhacoZOW3S+bY3xPwxM6qa4w1ozbUWnDXJQ0/EBDU6RLm0yC8qdq3NAIRRMsbVIuuwN/6Neg90nQkAFpcLbstfrhKuNFCgq5ZMbMAz/HhWUaBZdQHoaFgZzxhF3C3FHFUjALW69Z0ndOWdE40+4opLX6b4dlqTGbNHKVKcMrs+9V4v+8eYHxeGGFygsExbeD4kJSzGiVGV0JDRzlxhHGtXB/pfyKuTDQJbszpX47B76ziV0XSvBsBXuqxDVqVqUY7Gd2n8xGw8/D4NvHwXTcxNkmr8lbckwC8olMySk5IzPCyTW5IbfkrvXH63ivvKNtqddqel6SHXhv/gLMbrqD</latexit><latexit sha1_base64="Wx3hIxObyD9jVmjGbChXYBtOR90=">AAACZXicbVBda9swFFXcfWTdliXZ6MseJhYGHYxgh8EyukBgL33sYFkLcRZk5boVlmUjXY8E4T/TX9PX9m3/YD9jsuOHJd0BicM59+rqniiXwqDv/255Bw8ePnrcfnL49Nnzzotur//DZIXmMOOZzPRFxAxIoWCGAiVc5BpYGkk4j5KvlX/+C7QRmfqOmxwWKbtUIhacoZOW3S+bY3xPwxM6qa4w1ozbUWnDXJQ0/EBDU6RLm0yC8qdq3NAIRRMsbVIuuwN/6Neg90nQkAFpcLbstfrhKuNFCgq5ZMbMAz/HhWUaBZdQHoaFgZzxhF3C3FHFUjALW69Z0ndOWdE40+4opLX6b4dlqTGbNHKVKcMrs+9V4v+8eYHxeGGFygsExbeD4kJSzGiVGV0JDRzlxhHGtXB/pfyKuTDQJbszpX47B76ziV0XSvBsBXuqxDVqVqUY7Gd2n8xGw8/D4NvHwXTcxNkmr8lbckwC8olMySk5IzPCyTW5IbfkrvXH63ivvKNtqddqel6SHXhv/gLMbrqD</latexit><latexit sha1_base64="Wx3hIxObyD9jVmjGbChXYBtOR90=">AAACZXicbVBda9swFFXcfWTdliXZ6MseJhYGHYxgh8EyukBgL33sYFkLcRZk5boVlmUjXY8E4T/TX9PX9m3/YD9jsuOHJd0BicM59+rqniiXwqDv/255Bw8ePnrcfnL49Nnzzotur//DZIXmMOOZzPRFxAxIoWCGAiVc5BpYGkk4j5KvlX/+C7QRmfqOmxwWKbtUIhacoZOW3S+bY3xPwxM6qa4w1ozbUWnDXJQ0/EBDU6RLm0yC8qdq3NAIRRMsbVIuuwN/6Neg90nQkAFpcLbstfrhKuNFCgq5ZMbMAz/HhWUaBZdQHoaFgZzxhF3C3FHFUjALW69Z0ndOWdE40+4opLX6b4dlqTGbNHKVKcMrs+9V4v+8eYHxeGGFygsExbeD4kJSzGiVGV0JDRzlxhHGtXB/pfyKuTDQJbszpX47B76ziV0XSvBsBXuqxDVqVqUY7Gd2n8xGw8/D4NvHwXTcxNkmr8lbckwC8olMySk5IzPCyTW5IbfkrvXH63ivvKNtqddqel6SHXhv/gLMbrqD</latexit><latexit sha1_base64="Wx3hIxObyD9jVmjGbChXYBtOR90=">AAACZXicbVBda9swFFXcfWTdliXZ6MseJhYGHYxgh8EyukBgL33sYFkLcRZk5boVlmUjXY8E4T/TX9PX9m3/YD9jsuOHJd0BicM59+rqniiXwqDv/255Bw8ePnrcfnL49Nnzzotur//DZIXmMOOZzPRFxAxIoWCGAiVc5BpYGkk4j5KvlX/+C7QRmfqOmxwWKbtUIhacoZOW3S+bY3xPwxM6qa4w1ozbUWnDXJQ0/EBDU6RLm0yC8qdq3NAIRRMsbVIuuwN/6Neg90nQkAFpcLbstfrhKuNFCgq5ZMbMAz/HhWUaBZdQHoaFgZzxhF3C3FHFUjALW69Z0ndOWdE40+4opLX6b4dlqTGbNHKVKcMrs+9V4v+8eYHxeGGFygsExbeD4kJSzGiVGV0JDRzlxhHGtXB/pfyKuTDQJbszpX47B76ziV0XSvBsBXuqxDVqVqUY7Gd2n8xGw8/D4NvHwXTcxNkmr8lbckwC8olMySk5IzPCyTW5IbfkrvXH63ivvKNtqddqel6SHXhv/gLMbrqD</latexit>

Euler’s identity

Euler’s identity. e ix = cos x + i sin x. Sinusoids. Sum of sine and cosines = sum of complex exponentials.

45

Time domain vs. frequency domain

Signal. [touch tone button 1]

Time domain.

Frequency domain.

46

€

y(t) = 12 sin(2π ⋅ 697 t) + 1

2 sin(2π ⋅ 1209 t)

Reference: Cleve Moler, Numerical Computing with MATLAB

soundpressure

0.5

amplitude

Time domain vs. frequency domain

Signal. [recording, 8192 samples per second]

Magnitude of discrete Fourier transform.

47

Reference: Cleve Moler, Numerical Computing with MATLAB

Fast Fourier transform

FFT. Fast way to convert between time domain and frequency domain.

Alternate viewpoint. Fast way to multiply and evaluate polynomials.

48

we take this approach

“ If you speed up any nontrivial algorithm by a factor of a million or so the world will beat a path towards finding useful applications for it. ” — Numerical Recipes

Fast Fourier transform: applications

Applications.

独Optics, acoustics, quantum physics, telecommunications, radar,control systems, signal processing, speech recognition, data

compression, image processing, seismology, mass spectrometry, …

独Digital media. [DVD, JPEG, MP3, H.264]

独Medical diagnostics. [MRI, CT, PET scans, ultrasound]

独Numerical solutions to Poisson’s equation.

独Integer and polynomial multiplication.

独Shor’s quantum factoring algorithm.

独…

49

“ The FFT is one of the truly great computational developments of [the 20th] century. It has changed the face of science and engineering so much that it is not an exaggeration to say that life as we know it would be very different without the FFT. ” — Charles van Loan

Fast Fourier transform: brief history

Gauss (1805, 1866). Analyzed periodic motion of asteroid Ceres.

Runge–König (1924). Laid theoretical groundwork.

Danielson–Lanczos (1942). Efficient algorithm, x-ray crystallography.

Cooley–Tukey (1965). Detect nuclear tests in Soviet Union and track

submarines. Rediscovered and popularized FFT.

Importance not fully realized until emergence of digital computers.

50

Polynomials: coefficient representation

Univariate polynomial. [ coefficient representation ]

Addition. O(n) arithmetic operations.

Evaluation. O(n) using Horner’s method.

Multiplication (linear convolution). O(n2) using brute force.

51

A(x) + B(x) = (a0 + b0) + (a1 + b1)x + . . . + (an�1 + bn�1)xn�1

<latexit sha1_base64="UKyo1cD/d0X9bu21oOzkYf1aTAQ=">AAACgHicbVBNTxsxEHWWllI+Axx7sRohJUKENUIqBVWCcuFIJUKQknTl9U7Awmuv7Nkq0So/rT+EM1f4D/VucmgCI9l+896Mx35xpqTDMHyqBUsfPi5/Wvm8ura+sblV3965dSa3AjrCKGPvYu5ASQ0dlKjgLrPA01hBN368LPXuH7BOGn2D4wwGKb/XcigFR09F9e5Fc9Si+/RnefTP6I9ya/Io9FwchaXkM1ZlrEVHHtC+Sgy6qVLoAzap1Ar5it8ViOqNsB1WQd8CNgMNMovraLu200+MyFPQKBR3rsfCDAcFtyiFgslqP3eQcfHI76HnoeYpuEFROTChe55J6NBYvzTSiv2/o+Cpc+M09pUpxwe3qJXke1ovx+HJoJA6yxG0mA4a5oqioaWdNJEWBKqxB1xY6d9KxQO3XKA3fW5KdXcGYu4nxSjXUpgEFliFI7S8dJEtevYWdI7a39vs13Hj/GRm5wr5Qr6SJmHkGzknV+SadIggf8kzeSGvwVLQCg4DNi0NarOeXTIXwek/t9G8Lg==</latexit><latexit sha1_base64="UKyo1cD/d0X9bu21oOzkYf1aTAQ=">AAACgHicbVBNTxsxEHWWllI+Axx7sRohJUKENUIqBVWCcuFIJUKQknTl9U7Awmuv7Nkq0So/rT+EM1f4D/VucmgCI9l+896Mx35xpqTDMHyqBUsfPi5/Wvm8ura+sblV3965dSa3AjrCKGPvYu5ASQ0dlKjgLrPA01hBN368LPXuH7BOGn2D4wwGKb/XcigFR09F9e5Fc9Si+/RnefTP6I9ya/Io9FwchaXkM1ZlrEVHHtC+Sgy6qVLoAzap1Ar5it8ViOqNsB1WQd8CNgMNMovraLu200+MyFPQKBR3rsfCDAcFtyiFgslqP3eQcfHI76HnoeYpuEFROTChe55J6NBYvzTSiv2/o+Cpc+M09pUpxwe3qJXke1ovx+HJoJA6yxG0mA4a5oqioaWdNJEWBKqxB1xY6d9KxQO3XKA3fW5KdXcGYu4nxSjXUpgEFliFI7S8dJEtevYWdI7a39vs13Hj/GRm5wr5Qr6SJmHkGzknV+SadIggf8kzeSGvwVLQCg4DNi0NarOeXTIXwek/t9G8Lg==</latexit><latexit sha1_base64="UKyo1cD/d0X9bu21oOzkYf1aTAQ=">AAACgHicbVBNTxsxEHWWllI+Axx7sRohJUKENUIqBVWCcuFIJUKQknTl9U7Awmuv7Nkq0So/rT+EM1f4D/VucmgCI9l+896Mx35xpqTDMHyqBUsfPi5/Wvm8ura+sblV3965dSa3AjrCKGPvYu5ASQ0dlKjgLrPA01hBN368LPXuH7BOGn2D4wwGKb/XcigFR09F9e5Fc9Si+/RnefTP6I9ya/Io9FwchaXkM1ZlrEVHHtC+Sgy6qVLoAzap1Ar5it8ViOqNsB1WQd8CNgMNMovraLu200+MyFPQKBR3rsfCDAcFtyiFgslqP3eQcfHI76HnoeYpuEFROTChe55J6NBYvzTSiv2/o+Cpc+M09pUpxwe3qJXke1ovx+HJoJA6yxG0mA4a5oqioaWdNJEWBKqxB1xY6d9KxQO3XKA3fW5KdXcGYu4nxSjXUpgEFliFI7S8dJEtevYWdI7a39vs13Hj/GRm5wr5Qr6SJmHkGzknV+SadIggf8kzeSGvwVLQCg4DNi0NarOeXTIXwek/t9G8Lg==</latexit><latexit sha1_base64="UKyo1cD/d0X9bu21oOzkYf1aTAQ=">AAACgHicbVBNTxsxEHWWllI+Axx7sRohJUKENUIqBVWCcuFIJUKQknTl9U7Awmuv7Nkq0So/rT+EM1f4D/VucmgCI9l+896Mx35xpqTDMHyqBUsfPi5/Wvm8ura+sblV3965dSa3AjrCKGPvYu5ASQ0dlKjgLrPA01hBN368LPXuH7BOGn2D4wwGKb/XcigFR09F9e5Fc9Si+/RnefTP6I9ya/Io9FwchaXkM1ZlrEVHHtC+Sgy6qVLoAzap1Ar5it8ViOqNsB1WQd8CNgMNMovraLu200+MyFPQKBR3rsfCDAcFtyiFgslqP3eQcfHI76HnoeYpuEFROTChe55J6NBYvzTSiv2/o+Cpc+M09pUpxwe3qJXke1ovx+HJoJA6yxG0mA4a5oqioaWdNJEWBKqxB1xY6d9KxQO3XKA3fW5KdXcGYu4nxSjXUpgEFliFI7S8dJEtevYWdI7a39vs13Hj/GRm5wr5Qr6SJmHkGzknV+SadIggf8kzeSGvwVLQCg4DNi0NarOeXTIXwek/t9G8Lg==</latexit>

A(x) = a0 + (x(a1 + x(a2 + . . . + x(an�2 + x(an�1)) . . .))<latexit sha1_base64="ColXSYPNX4IS9T4mmJbmC5WWByg=">AAACdXicbVBdaxNBFJ2sH631K20fBRmMShYx3Q0FW0So+OKDDxWMLSRhuTt70w6dnVlm7paEZf+Uv0Yf9Vf46GyyoEm9MHMP59w7d+5JCyUdRdGPTnDr9p27W9v3du4/ePjocXd376szpRU4EkYZe56CQyU1jkiSwvPCIuSpwrP06kOjn12jddLoL7QocJrDhZYzKYA8lXQ/ve/PQz55y981FyQRf8X78z4ksQdNHvo8UZkh1xKVfj2s/+K4DsOVHoZJtxcNomXwmyBuQY+1cZrsdvYmmRFljpqEAufGcVTQtAJLUiisdyalwwLEFVzg2EMNObpptVy75i88k/GZsf5o4kv2344KcucWeeorc6BLt6k15P+0cUmzo2kldVESarEaNCsVJ8MbD3kmLQpSCw9AWOn/ysUlWBDknV6bsny7QLG2STUvtRQmww1W0Zws1N7FeNOzm2A0HBwP4s+HvZOj1s5t9oQ9Y30WszfshH1kp2zEBPvGvrOf7Ffnd/A0eB68XJUGnbZnn61FcPAHKrW7Vg==</latexit><latexit sha1_base64="ColXSYPNX4IS9T4mmJbmC5WWByg=">AAACdXicbVBdaxNBFJ2sH631K20fBRmMShYx3Q0FW0So+OKDDxWMLSRhuTt70w6dnVlm7paEZf+Uv0Yf9Vf46GyyoEm9MHMP59w7d+5JCyUdRdGPTnDr9p27W9v3du4/ePjocXd376szpRU4EkYZe56CQyU1jkiSwvPCIuSpwrP06kOjn12jddLoL7QocJrDhZYzKYA8lXQ/ve/PQz55y981FyQRf8X78z4ksQdNHvo8UZkh1xKVfj2s/+K4DsOVHoZJtxcNomXwmyBuQY+1cZrsdvYmmRFljpqEAufGcVTQtAJLUiisdyalwwLEFVzg2EMNObpptVy75i88k/GZsf5o4kv2344KcucWeeorc6BLt6k15P+0cUmzo2kldVESarEaNCsVJ8MbD3kmLQpSCw9AWOn/ysUlWBDknV6bsny7QLG2STUvtRQmww1W0Zws1N7FeNOzm2A0HBwP4s+HvZOj1s5t9oQ9Y30WszfshH1kp2zEBPvGvrOf7Ffnd/A0eB68XJUGnbZnn61FcPAHKrW7Vg==</latexit><latexit sha1_base64="ColXSYPNX4IS9T4mmJbmC5WWByg=">AAACdXicbVBdaxNBFJ2sH631K20fBRmMShYx3Q0FW0So+OKDDxWMLSRhuTt70w6dnVlm7paEZf+Uv0Yf9Vf46GyyoEm9MHMP59w7d+5JCyUdRdGPTnDr9p27W9v3du4/ePjocXd376szpRU4EkYZe56CQyU1jkiSwvPCIuSpwrP06kOjn12jddLoL7QocJrDhZYzKYA8lXQ/ve/PQz55y981FyQRf8X78z4ksQdNHvo8UZkh1xKVfj2s/+K4DsOVHoZJtxcNomXwmyBuQY+1cZrsdvYmmRFljpqEAufGcVTQtAJLUiisdyalwwLEFVzg2EMNObpptVy75i88k/GZsf5o4kv2344KcucWeeorc6BLt6k15P+0cUmzo2kldVESarEaNCsVJ8MbD3kmLQpSCw9AWOn/ysUlWBDknV6bsny7QLG2STUvtRQmww1W0Zws1N7FeNOzm2A0HBwP4s+HvZOj1s5t9oQ9Y30WszfshH1kp2zEBPvGvrOf7Ffnd/A0eB68XJUGnbZnn61FcPAHKrW7Vg==</latexit><latexit sha1_base64="ColXSYPNX4IS9T4mmJbmC5WWByg=">AAACdXicbVBdaxNBFJ2sH631K20fBRmMShYx3Q0FW0So+OKDDxWMLSRhuTt70w6dnVlm7paEZf+Uv0Yf9Vf46GyyoEm9MHMP59w7d+5JCyUdRdGPTnDr9p27W9v3du4/ePjocXd376szpRU4EkYZe56CQyU1jkiSwvPCIuSpwrP06kOjn12jddLoL7QocJrDhZYzKYA8lXQ/ve/PQz55y981FyQRf8X78z4ksQdNHvo8UZkh1xKVfj2s/+K4DsOVHoZJtxcNomXwmyBuQY+1cZrsdvYmmRFljpqEAufGcVTQtAJLUiisdyalwwLEFVzg2EMNObpptVy75i88k/GZsf5o4kv2344KcucWeeorc6BLt6k15P+0cUmzo2kldVESarEaNCsVJ8MbD3kmLQpSCw9AWOn/ysUlWBDknV6bsny7QLG2STUvtRQmww1W0Zws1N7FeNOzm2A0HBwP4s+HvZOj1s5t9oQ9Y30WszfshH1kp2zEBPvGvrOf7Ffnd/A0eB68XJUGnbZnn61FcPAHKrW7Vg==</latexit>

A(x) � B(x) =

2n�2�

i=0

cixi r?2`2 ci =

i�

j=0

ajbi�j

<latexit sha1_base64="Xv1zuH5iops0CdrBmgSTFSlB4qU=">AAACkXicbVFdT9swFHUD2/jYRmG87cVahcQeqJIKCQaqVOBlEi9MWgGpLZHj3lIXx4nsG5Yqyl/knf/BKwgnzcNauJKto3Puh+9xEEth0HUfa87S8oePn1ZW19Y/f/m6Ud/cujRRojl0eSQjfR0wA1Io6KJACdexBhYGEq6Cu7NCv7oHbUSk/uI0hkHIbpUYCc7QUn59fLKb/qR9FCEYelriY9ourr5JQj8TbTe/yVpqr5VT7gua3ggrFjJCitm/MWjIZ0wht6uySVEmKPMnNLBN9ia5X2+4TbcM+hZ4FWiQKi78zdpWfxjxJASFXDJjep4b4yBjGgWXkK/1EwMx43fsFnoWKmZXGGSlJTndscyQjiJtj0Jasv9XZCw0ZhoGNjNkODaLWkG+p/USHB0OMqHiBEHx2aBRIilGtPCXDoUGjnJqAeNa2LdSPmaacbS/MDel7B0Dn9skSxMleDSEBVZiipoVLnqLnr0F3VbzV9P7s9/oHFZ2rpDv5AfZJR45IB3ym1yQLuHkgTyRZ/LibDtHTsc5naU6tarmG5kL5/wVpenIUw==</latexit><latexit sha1_base64="Xv1zuH5iops0CdrBmgSTFSlB4qU=">AAACkXicbVFdT9swFHUD2/jYRmG87cVahcQeqJIKCQaqVOBlEi9MWgGpLZHj3lIXx4nsG5Yqyl/knf/BKwgnzcNauJKto3Puh+9xEEth0HUfa87S8oePn1ZW19Y/f/m6Ud/cujRRojl0eSQjfR0wA1Io6KJACdexBhYGEq6Cu7NCv7oHbUSk/uI0hkHIbpUYCc7QUn59fLKb/qR9FCEYelriY9ourr5JQj8TbTe/yVpqr5VT7gua3ggrFjJCitm/MWjIZ0wht6uySVEmKPMnNLBN9ia5X2+4TbcM+hZ4FWiQKi78zdpWfxjxJASFXDJjep4b4yBjGgWXkK/1EwMx43fsFnoWKmZXGGSlJTndscyQjiJtj0Jasv9XZCw0ZhoGNjNkODaLWkG+p/USHB0OMqHiBEHx2aBRIilGtPCXDoUGjnJqAeNa2LdSPmaacbS/MDel7B0Dn9skSxMleDSEBVZiipoVLnqLnr0F3VbzV9P7s9/oHFZ2rpDv5AfZJR45IB3ym1yQLuHkgTyRZ/LibDtHTsc5naU6tarmG5kL5/wVpenIUw==</latexit><latexit sha1_base64="Xv1zuH5iops0CdrBmgSTFSlB4qU=">AAACkXicbVFdT9swFHUD2/jYRmG87cVahcQeqJIKCQaqVOBlEi9MWgGpLZHj3lIXx4nsG5Yqyl/knf/BKwgnzcNauJKto3Puh+9xEEth0HUfa87S8oePn1ZW19Y/f/m6Ud/cujRRojl0eSQjfR0wA1Io6KJACdexBhYGEq6Cu7NCv7oHbUSk/uI0hkHIbpUYCc7QUn59fLKb/qR9FCEYelriY9ourr5JQj8TbTe/yVpqr5VT7gua3ggrFjJCitm/MWjIZ0wht6uySVEmKPMnNLBN9ia5X2+4TbcM+hZ4FWiQKi78zdpWfxjxJASFXDJjep4b4yBjGgWXkK/1EwMx43fsFnoWKmZXGGSlJTndscyQjiJtj0Jasv9XZCw0ZhoGNjNkODaLWkG+p/USHB0OMqHiBEHx2aBRIilGtPCXDoUGjnJqAeNa2LdSPmaacbS/MDel7B0Dn9skSxMleDSEBVZiipoVLnqLnr0F3VbzV9P7s9/oHFZ2rpDv5AfZJR45IB3ym1yQLuHkgTyRZ/LibDtHTsc5naU6tarmG5kL5/wVpenIUw==</latexit><latexit sha1_base64="Xv1zuH5iops0CdrBmgSTFSlB4qU=">AAACkXicbVFdT9swFHUD2/jYRmG87cVahcQeqJIKCQaqVOBlEi9MWgGpLZHj3lIXx4nsG5Yqyl/knf/BKwgnzcNauJKto3Puh+9xEEth0HUfa87S8oePn1ZW19Y/f/m6Ud/cujRRojl0eSQjfR0wA1Io6KJACdexBhYGEq6Cu7NCv7oHbUSk/uI0hkHIbpUYCc7QUn59fLKb/qR9FCEYelriY9ourr5JQj8TbTe/yVpqr5VT7gua3ggrFjJCitm/MWjIZ0wht6uySVEmKPMnNLBN9ia5X2+4TbcM+hZ4FWiQKi78zdpWfxjxJASFXDJjep4b4yBjGgWXkK/1EwMx43fsFnoWKmZXGGSlJTndscyQjiJtj0Jasv9XZCw0ZhoGNjNkODaLWkG+p/USHB0OMqHiBEHx2aBRIilGtPCXDoUGjnJqAeNa2LdSPmaacbS/MDel7B0Dn9skSxMleDSEBVZiipoVLnqLnr0F3VbzV9P7s9/oHFZ2rpDv5AfZJR45IB3ym1yQLuHkgTyRZ/LibDtHTsc5naU6tarmG5kL5/wVpenIUw==</latexit>

A(x) = a0 + a1x + a2x2 + . . . + an�1x

n�1

B(x) = b0 + b1x + b2x2 + . . . + bn�1x

n�1<latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit><latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit><latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit><latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit>

double val = 0.0; for (int j = n-1; j >= 0; j--) val = a[j] + (x * val);

What was the subject of Gauss’ Ph.D thesis?

A. Gaussian elimination.

B. Fast Fourier transform.

C. Prime number theorem.

D. Cauchy integral theorem.

E. Fundamental theorem of algebra.

F. Angle-preserving maps.

G. Method of least squares.

H. Non-Euclidean geometry.

I. Constructing a regular heptadecagon with straightedge and compass.

52


A modest Ph.D. dissertation title

53

DEMONSTRATIO NOVATHEOREMATIS

OMNEM FVNCTIONEM ALGEBRAICAMRATIONALEM INTEGRAM

VNIVS VARIABILISIN FACTORES REALES PRIMI VEL SECUNDI GRADVS

RESOLVI POSSE

AVCTORECAROLO FRIDERICO GAVSS

HELMSTADIIAPVD C. G. FLECKEISEN. 1799

1.

Quaelibet aequatio algebraica determinata reduci potest ad

formam xm+Axm-1+Bxm-2+ etc. +M=0, ita vt m sit numerusinteger positiuus. Si partem primam huius aequationis per Xdenotamus, aequationique X=0 per plures valores inaequalesipsius x satisfieri supponimus, puta ponendo x=α, x=β, x=γ etc.functio X per productum e factoribus x-α, x-β, x-γ etc. diuisibiliserit. Vice versa, si productum e pluribus factoribus simplicibus x-α, x-β, x-γ etc. functionem X metitur: aequationi X=0 satisfiet,aequando ipsam x cuicunque quantitatum α, β, γ etc. Denique siX producto ex m factoribus talibus simplicibus aequalis est (siueomnes diuersi sint, siue quidam ex ipsis identici): alii factoressimplices praeter hos functionem X metiri non poterunt.

Quamobrem aequatio mti gradus plures quam m radices habere

nequit; simul vero patet, aequationem mti gradus paucioresradices habere posse, etsi X in m factores simplices resolubilis sit:

“ New proof of the theorem that every algebraic rational integral function in one variable can be resolved into real factors of the first or the second degree. ”

Polynomials: point-value representation

Fundamental theorem of algebra. A degree n univariate polynomial with

complex coefficients has exactly n complex roots.

Corollary. A degree n – 1 univariate polynomial A(x) is uniquely specified by

its evaluation at n distinct values of x.

54

x

y

xj

yj = A(xj )

Polynomials: point-value representation

Univariate polynomial. [ point-value representation ]

Addition. O(n) arithmetic operations.

Multiplication. O(n), but represent A(x) and B(x) using 2n points.

Evaluation. O(n2) using Lagrange’s formula.

55

A(x): (x0, y0), …, (xn−1, yn−1)B(x): (x0, z0), …, (xn−1, zn−1)

A(x) + B(x): (x0, y0 + z0), …, (xn−1, yn−1 + zn−1)

A(x) ! B(x): (x0, y0 ! z0), …, (x2n−1, y2n−1 ! z2n−1)

A(x) =

n�1�

k=0

yk

�j �=k(x � xj)�j �=k(xk � xj)

<latexit sha1_base64="3AlzPB2t5kSlKP9lDSHJ8RvBpoQ=">AAAClnicbVFNa9wwEJWdfqTp1yY9lV5El8LmkMUOhaYspQmltMctdJvAamtkeZwo1ocrycWL8a/MKT8lt8obH7qbDGh4em9GIz2lpeDWRdF1EG49ePjo8faTnafPnr94Odjd+2V1ZRjMmBbanKXUguAKZo47AWelASpTAadp8aXTT/+CsVyrn25ZwkLSc8VzzqjzVDKQJ6N6H5MJ/tQlYiuZNIXfRO3vRh3Ebccuk8JnTHJDWUNKo7OkucREwR9ctHhU4wNcJ5f7ZEIm7X16UvQVbTIYRuNoFfguiHswRH1Mk91gj2SaVRKUY4JaO4+j0i0aahxnAtodUlkoKSvoOcw9VFSCXTQrX1r8zjMZzrXxSzm8Yv/vaKi0dilTXympu7CbWkfep80rlx8tGq7KyoFit4PySmCncWcyzrgB5sTSA8oM93fF7IJ6+5z/irUpq7NLYGsvaepKcaYz2GCFq52hnYvxpmd3wexw/HEc/3g/PD7q7dxGb9BbNEIx+oCO0Xc0RTPE0BW6CcJgK3wdfg6/ht9uS8Og73mF1iKc/gMJQ8fb</latexit><latexit sha1_base64="3AlzPB2t5kSlKP9lDSHJ8RvBpoQ=">AAAClnicbVFNa9wwEJWdfqTp1yY9lV5El8LmkMUOhaYspQmltMctdJvAamtkeZwo1ocrycWL8a/MKT8lt8obH7qbDGh4em9GIz2lpeDWRdF1EG49ePjo8faTnafPnr94Odjd+2V1ZRjMmBbanKXUguAKZo47AWelASpTAadp8aXTT/+CsVyrn25ZwkLSc8VzzqjzVDKQJ6N6H5MJ/tQlYiuZNIXfRO3vRh3Ebccuk8JnTHJDWUNKo7OkucREwR9ctHhU4wNcJ5f7ZEIm7X16UvQVbTIYRuNoFfguiHswRH1Mk91gj2SaVRKUY4JaO4+j0i0aahxnAtodUlkoKSvoOcw9VFSCXTQrX1r8zjMZzrXxSzm8Yv/vaKi0dilTXympu7CbWkfep80rlx8tGq7KyoFit4PySmCncWcyzrgB5sTSA8oM93fF7IJ6+5z/irUpq7NLYGsvaepKcaYz2GCFq52hnYvxpmd3wexw/HEc/3g/PD7q7dxGb9BbNEIx+oCO0Xc0RTPE0BW6CcJgK3wdfg6/ht9uS8Og73mF1iKc/gMJQ8fb</latexit><latexit sha1_base64="3AlzPB2t5kSlKP9lDSHJ8RvBpoQ=">AAAClnicbVFNa9wwEJWdfqTp1yY9lV5El8LmkMUOhaYspQmltMctdJvAamtkeZwo1ocrycWL8a/MKT8lt8obH7qbDGh4em9GIz2lpeDWRdF1EG49ePjo8faTnafPnr94Odjd+2V1ZRjMmBbanKXUguAKZo47AWelASpTAadp8aXTT/+CsVyrn25ZwkLSc8VzzqjzVDKQJ6N6H5MJ/tQlYiuZNIXfRO3vRh3Ebccuk8JnTHJDWUNKo7OkucREwR9ctHhU4wNcJ5f7ZEIm7X16UvQVbTIYRuNoFfguiHswRH1Mk91gj2SaVRKUY4JaO4+j0i0aahxnAtodUlkoKSvoOcw9VFSCXTQrX1r8zjMZzrXxSzm8Yv/vaKi0dilTXympu7CbWkfep80rlx8tGq7KyoFit4PySmCncWcyzrgB5sTSA8oM93fF7IJ6+5z/irUpq7NLYGsvaepKcaYz2GCFq52hnYvxpmd3wexw/HEc/3g/PD7q7dxGb9BbNEIx+oCO0Xc0RTPE0BW6CcJgK3wdfg6/ht9uS8Og73mF1iKc/gMJQ8fb</latexit><latexit sha1_base64="3AlzPB2t5kSlKP9lDSHJ8RvBpoQ=">AAAClnicbVFNa9wwEJWdfqTp1yY9lV5El8LmkMUOhaYspQmltMctdJvAamtkeZwo1ocrycWL8a/MKT8lt8obH7qbDGh4em9GIz2lpeDWRdF1EG49ePjo8faTnafPnr94Odjd+2V1ZRjMmBbanKXUguAKZo47AWelASpTAadp8aXTT/+CsVyrn25ZwkLSc8VzzqjzVDKQJ6N6H5MJ/tQlYiuZNIXfRO3vRh3Ebccuk8JnTHJDWUNKo7OkucREwR9ctHhU4wNcJ5f7ZEIm7X16UvQVbTIYRuNoFfguiHswRH1Mk91gj2SaVRKUY4JaO4+j0i0aahxnAtodUlkoKSvoOcw9VFSCXTQrX1r8zjMZzrXxSzm8Yv/vaKi0dilTXympu7CbWkfep80rlx8tGq7KyoFit4PySmCncWcyzrgB5sTSA8oM93fF7IJ6+5z/irUpq7NLYGsvaepKcaYz2GCFq52hnYvxpmd3wexw/HEc/3g/PD7q7dxGb9BbNEIx+oCO0Xc0RTPE0BW6CcJgK3wdfg6/ht9uS8Og73mF1iKc/gMJQ8fb</latexit>

not used

Converting between two representations

Tradeoff. Either fast evaluation or fast multiplication. We want both!

Goal. Efficient conversion between two representations ⇒ all ops fast.

56

representation multiply evaluate

coefficient O(n2) O(n)

point-value O(n) O(n2)

point-value representationcoefficient representation

a0, a1, . . . , an�1

b0, b1, . . . , bn�1<latexit sha1_base64="XhUwKPPXUpHs4BCTkzKNKRK4h28=">AAACjHicbVFNaxsxEJW3H0ndprGTYy6iptCDY3ZDQhxKIRAIOaZQNwGvWWa141hEKy3SbIhZ/AP7E/orem1v0dp7iJ0OiHm8mTcjPaWFko7C8HcrePX6zdut7Xft9x92Pu52uns/nSmtwJEwytjbFBwqqXFEkhTeFhYhTxXepPcXdf3mAa2TRv+geYGTHO60nEoB5KmkIyAJ+3GfQxLVKVaZIedzH5JKH0YLHsftdNUSz1wBAqtBGB1JveBpI9mgn81IVzOSTi8chMvgL0HUgB5r4jrptvbizIgyR01CgXPjKCxoUoElKRQu2nHp0C+9hzsce6ghRzeplm4s+GfPZHxqrD+a+JJ9rqggd26ep74zB5q5zVpN/q82Lmk6nFRSFyWhFqtF01JxMry2lmfSoiA19wCElf6uXMzAgiD/AWtblrMLFGsvqR5LLYXJcINV9EgWahejTc9egtHR4GwQfT/unQ8bO7fZAfvEvrCInbJzdsWu2YgJ9ov9YX/Zv2A3OAm+Bt9WrUGr0eyztQgunwAsEcTQ</latexit><latexit sha1_base64="XhUwKPPXUpHs4BCTkzKNKRK4h28=">AAACjHicbVFNaxsxEJW3H0ndprGTYy6iptCDY3ZDQhxKIRAIOaZQNwGvWWa141hEKy3SbIhZ/AP7E/orem1v0dp7iJ0OiHm8mTcjPaWFko7C8HcrePX6zdut7Xft9x92Pu52uns/nSmtwJEwytjbFBwqqXFEkhTeFhYhTxXepPcXdf3mAa2TRv+geYGTHO60nEoB5KmkIyAJ+3GfQxLVKVaZIedzH5JKH0YLHsftdNUSz1wBAqtBGB1JveBpI9mgn81IVzOSTi8chMvgL0HUgB5r4jrptvbizIgyR01CgXPjKCxoUoElKRQu2nHp0C+9hzsce6ghRzeplm4s+GfPZHxqrD+a+JJ9rqggd26ep74zB5q5zVpN/q82Lmk6nFRSFyWhFqtF01JxMry2lmfSoiA19wCElf6uXMzAgiD/AWtblrMLFGsvqR5LLYXJcINV9EgWahejTc9egtHR4GwQfT/unQ8bO7fZAfvEvrCInbJzdsWu2YgJ9ov9YX/Zv2A3OAm+Bt9WrUGr0eyztQgunwAsEcTQ</latexit><latexit sha1_base64="XhUwKPPXUpHs4BCTkzKNKRK4h28=">AAACjHicbVFNaxsxEJW3H0ndprGTYy6iptCDY3ZDQhxKIRAIOaZQNwGvWWa141hEKy3SbIhZ/AP7E/orem1v0dp7iJ0OiHm8mTcjPaWFko7C8HcrePX6zdut7Xft9x92Pu52uns/nSmtwJEwytjbFBwqqXFEkhTeFhYhTxXepPcXdf3mAa2TRv+geYGTHO60nEoB5KmkIyAJ+3GfQxLVKVaZIedzH5JKH0YLHsftdNUSz1wBAqtBGB1JveBpI9mgn81IVzOSTi8chMvgL0HUgB5r4jrptvbizIgyR01CgXPjKCxoUoElKRQu2nHp0C+9hzsce6ghRzeplm4s+GfPZHxqrD+a+JJ9rqggd26ep74zB5q5zVpN/q82Lmk6nFRSFyWhFqtF01JxMry2lmfSoiA19wCElf6uXMzAgiD/AWtblrMLFGsvqR5LLYXJcINV9EgWahejTc9egtHR4GwQfT/unQ8bO7fZAfvEvrCInbJzdsWu2YgJ9ov9YX/Zv2A3OAm+Bt9WrUGr0eyztQgunwAsEcTQ</latexit><latexit sha1_base64="XhUwKPPXUpHs4BCTkzKNKRK4h28=">AAACjHicbVFNaxsxEJW3H0ndprGTYy6iptCDY3ZDQhxKIRAIOaZQNwGvWWa141hEKy3SbIhZ/AP7E/orem1v0dp7iJ0OiHm8mTcjPaWFko7C8HcrePX6zdut7Xft9x92Pu52uns/nSmtwJEwytjbFBwqqXFEkhTeFhYhTxXepPcXdf3mAa2TRv+geYGTHO60nEoB5KmkIyAJ+3GfQxLVKVaZIedzH5JKH0YLHsftdNUSz1wBAqtBGB1JveBpI9mgn81IVzOSTi8chMvgL0HUgB5r4jrptvbizIgyR01CgXPjKCxoUoElKRQu2nHp0C+9hzsce6ghRzeplm4s+GfPZHxqrD+a+JJ9rqggd26ep74zB5q5zVpN/q82Lmk6nFRSFyWhFqtF01JxMry2lmfSoiA19wCElf6uXMzAgiD/AWtblrMLFGsvqR5LLYXJcINV9EgWahejTc9egtHR4GwQfT/unQ8bO7fZAfvEvrCInbJzdsWu2YgJ9ov9YX/Zv2A3OAm+Bt9WrUGr0eyztQgunwAsEcTQ</latexit>

(x0, y0), . . . , (xn�1, yn�1)<latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit>

Converting between two representations

Application. Polynomial multiplication (coefficient representation).

57

coefficient representation

a0, a1, . . . , an�1


point-value multiplication

O(n)


c0, c1, . . . , c2n�2<latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit><latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit><latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit><latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit>

point value representation

(x0, y0), . . . , (x2n�1, y2n�1)

(x0, z0), . . . , (x2n�1, z2n�1)<latexit sha1_base64="nRFru5kGNQuXBAb7W+fNyoLkgcc=">AAACh3icfVBdT9swFHUztrHsq7DHvVhUSK3EqgRNG3sD8cIjSAsgNV3kOLetVceO7JupJeq/25/YX+CV/QCcNkhrQVzZ0tE599xrn7SQwmIQ/G15L7Zevnq9/cZ/++79h4/tnd1Lq0vDIeJaanOdMgtSKIhQoITrwgDLUwlX6fS01q9+g7FCq584L2CYs7ESI8EZOipp/+rOkuCAzpOgd0Bjd2Sm0dbQCdWh+hIuanWFejSO/ZXh5lnDzYMhaXeCfrAs+hiEDeiQps6TndZunGle5qCQS2btIAwKHFbMoOASFn5cWigYn7IxDBxULAc7rJZBLOi+YzI60sZdhXTJ/u+oWG7tPE9dZ85wYje1mnxKG5Q4OhpWQhUlguKrRaNSUtS0TpVmwgBHOXeAcSPcWymfMMM4uuzXtixnF8DXflLNSiW4zmCDlThDwxYuxXAzs8cgOuz/6IcXXzvHR02c2+Qz2SNdEpLv5JickXMSEU7+kFtyR/55vhd437ym12s1nk9krbyTe1JCwRM=</latexit><latexit sha1_base64="nRFru5kGNQuXBAb7W+fNyoLkgcc=">AAACh3icfVBdT9swFHUztrHsq7DHvVhUSK3EqgRNG3sD8cIjSAsgNV3kOLetVceO7JupJeq/25/YX+CV/QCcNkhrQVzZ0tE599xrn7SQwmIQ/G15L7Zevnq9/cZ/++79h4/tnd1Lq0vDIeJaanOdMgtSKIhQoITrwgDLUwlX6fS01q9+g7FCq584L2CYs7ESI8EZOipp/+rOkuCAzpOgd0Bjd2Sm0dbQCdWh+hIuanWFejSO/ZXh5lnDzYMhaXeCfrAs+hiEDeiQps6TndZunGle5qCQS2btIAwKHFbMoOASFn5cWigYn7IxDBxULAc7rJZBLOi+YzI60sZdhXTJ/u+oWG7tPE9dZ85wYje1mnxKG5Q4OhpWQhUlguKrRaNSUtS0TpVmwgBHOXeAcSPcWymfMMM4uuzXtixnF8DXflLNSiW4zmCDlThDwxYuxXAzs8cgOuz/6IcXXzvHR02c2+Qz2SNdEpLv5JickXMSEU7+kFtyR/55vhd437ym12s1nk9krbyTe1JCwRM=</latexit><latexit sha1_base64="nRFru5kGNQuXBAb7W+fNyoLkgcc=">AAACh3icfVBdT9swFHUztrHsq7DHvVhUSK3EqgRNG3sD8cIjSAsgNV3kOLetVceO7JupJeq/25/YX+CV/QCcNkhrQVzZ0tE599xrn7SQwmIQ/G15L7Zevnq9/cZ/++79h4/tnd1Lq0vDIeJaanOdMgtSKIhQoITrwgDLUwlX6fS01q9+g7FCq584L2CYs7ESI8EZOipp/+rOkuCAzpOgd0Bjd2Sm0dbQCdWh+hIuanWFejSO/ZXh5lnDzYMhaXeCfrAs+hiEDeiQps6TndZunGle5qCQS2btIAwKHFbMoOASFn5cWigYn7IxDBxULAc7rJZBLOi+YzI60sZdhXTJ/u+oWG7tPE9dZ85wYje1mnxKG5Q4OhpWQhUlguKrRaNSUtS0TpVmwgBHOXeAcSPcWymfMMM4uuzXtixnF8DXflLNSiW4zmCDlThDwxYuxXAzs8cgOuz/6IcXXzvHR02c2+Qz2SNdEpLv5JickXMSEU7+kFtyR/55vhd437ym12s1nk9krbyTe1JCwRM=</latexit><latexit sha1_base64="nRFru5kGNQuXBAb7W+fNyoLkgcc=">AAACh3icfVBdT9swFHUztrHsq7DHvVhUSK3EqgRNG3sD8cIjSAsgNV3kOLetVceO7JupJeq/25/YX+CV/QCcNkhrQVzZ0tE599xrn7SQwmIQ/G15L7Zevnq9/cZ/++79h4/tnd1Lq0vDIeJaanOdMgtSKIhQoITrwgDLUwlX6fS01q9+g7FCq584L2CYs7ESI8EZOipp/+rOkuCAzpOgd0Bjd2Sm0dbQCdWh+hIuanWFejSO/ZXh5lnDzYMhaXeCfrAs+hiEDeiQps6TndZunGle5qCQS2btIAwKHFbMoOASFn5cWigYn7IxDBxULAc7rJZBLOi+YzI60sZdhXTJ/u+oWG7tPE9dZ85wYje1mnxKG5Q4OhpWQhUlguKrRaNSUtS0TpVmwgBHOXeAcSPcWymfMMM4uuzXtixnF8DXflLNSiW4zmCDlThDwxYuxXAzs8cgOuz/6IcXXzvHR02c2+Qz2SNdEpLv5JickXMSEU7+kFtyR/55vhd437ym12s1nk9krbyTe1JCwRM=</latexit>

point value representation

(x0, y0 � z0), . . . , (x2n�1, y2n�1 � z2n�1)<latexit sha1_base64="X9PWDeWVMU1vYgTT4C3dKuNx4+E=">AAACcXicbVBNSwMxEE3Xr/pd600vwSIoatktgnoTvHhUsCq0ZclmpxrMJksyK61L/5K/xosH/RX+ALNtQVsdEnh582Ym86JUCou+/17yZmbn5hfKi0vLK6tr65WN6q3VmeHQ5Fpqcx8xC1IoaKJACfepAZZEEu6ip4sif/cMxgqtbrCfQidhD0p0BWfoqLByudcL/UPaD33aRpGApS+hv39I2+7IWKMtoNPkDXUUDArhCP2oR+/9sFLz6/4w6F8QjEGNjOMq3ChV27HmWQIKuWTWtgI/xU7ODAouYbDUziykjD+xB2g5qJib18mHKw/ormNi2tXGXYV0yP6uyFlibT+JnDJh+GincwX5X66VYfe0kwuVZgiKjwZ1M0lR08I/GgsDHGXfAcaNcH+l/JEZxtG5PDFl2DsFPrFJ3suU4DqGKVZiDw0bOBeDac/+gmajflYPro9r56djO8tkm+yQPRKQE3JOLskVaRJOXskb+SCfpS9vy6PezkjqlcY1m2QivINvAdS8eA==</latexit><latexit sha1_base64="X9PWDeWVMU1vYgTT4C3dKuNx4+E=">AAACcXicbVBNSwMxEE3Xr/pd600vwSIoatktgnoTvHhUsCq0ZclmpxrMJksyK61L/5K/xosH/RX+ALNtQVsdEnh582Ym86JUCou+/17yZmbn5hfKi0vLK6tr65WN6q3VmeHQ5Fpqcx8xC1IoaKJACfepAZZEEu6ip4sif/cMxgqtbrCfQidhD0p0BWfoqLByudcL/UPaD33aRpGApS+hv39I2+7IWKMtoNPkDXUUDArhCP2oR+/9sFLz6/4w6F8QjEGNjOMq3ChV27HmWQIKuWTWtgI/xU7ODAouYbDUziykjD+xB2g5qJib18mHKw/ormNi2tXGXYV0yP6uyFlibT+JnDJh+GincwX5X66VYfe0kwuVZgiKjwZ1M0lR08I/GgsDHGXfAcaNcH+l/JEZxtG5PDFl2DsFPrFJ3suU4DqGKVZiDw0bOBeDac/+gmajflYPro9r56djO8tkm+yQPRKQE3JOLskVaRJOXskb+SCfpS9vy6PezkjqlcY1m2QivINvAdS8eA==</latexit><latexit sha1_base64="X9PWDeWVMU1vYgTT4C3dKuNx4+E=">AAACcXicbVBNSwMxEE3Xr/pd600vwSIoatktgnoTvHhUsCq0ZclmpxrMJksyK61L/5K/xosH/RX+ALNtQVsdEnh582Ym86JUCou+/17yZmbn5hfKi0vLK6tr65WN6q3VmeHQ5Fpqcx8xC1IoaKJACfepAZZEEu6ip4sif/cMxgqtbrCfQidhD0p0BWfoqLByudcL/UPaD33aRpGApS+hv39I2+7IWKMtoNPkDXUUDArhCP2oR+/9sFLz6/4w6F8QjEGNjOMq3ChV27HmWQIKuWTWtgI/xU7ODAouYbDUziykjD+xB2g5qJib18mHKw/ormNi2tXGXYV0yP6uyFlibT+JnDJh+GincwX5X66VYfe0kwuVZgiKjwZ1M0lR08I/GgsDHGXfAcaNcH+l/JEZxtG5PDFl2DsFPrFJ3suU4DqGKVZiDw0bOBeDac/+gmajflYPro9r56djO8tkm+yQPRKQE3JOLskVaRJOXskb+SCfpS9vy6PezkjqlcY1m2QivINvAdS8eA==</latexit><latexit sha1_base64="X9PWDeWVMU1vYgTT4C3dKuNx4+E=">AAACcXicbVBNSwMxEE3Xr/pd600vwSIoatktgnoTvHhUsCq0ZclmpxrMJksyK61L/5K/xosH/RX+ALNtQVsdEnh582Ym86JUCou+/17yZmbn5hfKi0vLK6tr65WN6q3VmeHQ5Fpqcx8xC1IoaKJACfepAZZEEu6ip4sif/cMxgqtbrCfQidhD0p0BWfoqLByudcL/UPaD33aRpGApS+hv39I2+7IWKMtoNPkDXUUDArhCP2oR+/9sFLz6/4w6F8QjEGNjOMq3ChV27HmWQIKuWTWtgI/xU7ODAouYbDUziykjD+xB2g5qJib18mHKw/ormNi2tXGXYV0yP6uyFlibT+JnDJh+GincwX5X66VYfe0kwuVZgiKjwZ1M0lR08I/GgsDHGXfAcaNcH+l/JEZxtG5PDFl2DsFPrFJ3suU4DqGKVZiDw0bOBeDac/+gmajflYPro9r56djO8tkm+yQPRKQE3JOLskVaRJOXskb+SCfpS9vy6PezkjqlcY1m2QivINvAdS8eA==</latexit>

FFTO(n log n)

inverse FFT

O(n log n)

Converting between two representations: brute force

Coefficient ⇒ point-value. Given a polynomial A(x) = a0 + a1 x + ... + an–1 xn–1,

evaluate it at n distinct points x0 , ..., xn–1.

Running time. O(n2) via matrix–vector multiply (or n Horner’s).

58

€

y0

y1

y2

!yn−1

#

$

% % % % % %

&

'

( ( ( ( ( (

=

1 x0 x02 " x0

n−1

1 x1 x12 " x1

n−1

1 x2 x22 " x2

n−1

! ! ! # !1 xn−1 xn−1

2 " xn−1n−1

#

$

% % % % % %

&

'

( ( ( ( ( (

a0

a1

a2

! an−1

#

$

% % % % % %

&

'

( ( ( ( ( (

Converting between two representations: brute force

Point-value ⇒ coefficient. Given n distinct points x0, ... , xn–1 and values y0, ... , yn–1, find unique polynomial A(x) = a0 + a1 x + ... + an–1 xn–1

, that hasgiven values at given points.

Running time. O(n3) via Gaussian elimination.

59

€

y0

y1

y2

!yn−1

#

$

% % % % % %

&

'

( ( ( ( ( (

=

1 x0 x02 " x0

n−1

1 x1 x12 " x1

n−1

1 x2 x22 " x2

n−1

! ! ! # !1 xn−1 xn−1

2 " xn−1n−1

#

$

% % % % % %

&

'

( ( ( ( ( (

a0

a1

a2

! an−1

#

$

% % % % % %

&

'

( ( ( ( ( (

Vandermonde matrix is invertible iff xi distinct

or O(n2.38) via fast matrix multiplication

Which divide-and-conquer approach to use to multiply polynomials?

A. Divide polynomial into low- and high-degree terms.

B. Divide polynomial into even- and odd-degree terms.

C. Either A or B.

D. Neither A nor B.

60


A(x) = a0 + a1 x + a2 x2 + a3 x3 + a4 x4 + a5 x5 + a6 x6 + a7 x7.

Alow(x) = a0 + a1 x + a2 x2 + a3 x3.Ahigh (x) = a4 + a5 x + a6 x2 + a7 x3.

Aeven(x) = a0 + a2 x + a4 x2 + a6 x3.Aodd (x) = a1 + a3 x + a5 x2 + a7 x3.

Divide-and-conquer

Decimation in time. Divide into even- and odd- degree terms.

独A(x) = a0 + a1 x + a2 x2 + a3 x3 + a4 x4 + a5 x5 + a6 x6 + a7 x7.

独Aeven(x) = a0 + a2 x + a4 x2 + a6 x3.

独Aodd (x) = a1 + a3 x + a5 x2 + a7 x3.

独A(x) = Aeven(x2) + x Aodd(x2).

Decimation in frequency. Divide into low- and high-degree terms.


独Alow(x) = a0 + a1 x + a2 x2 + a3 x3.

独Ahigh (x) = a4 + a5 x + a6 x2 + a7 x3.

独A(x) = Alow(x) + x4 Ahigh(x).

61

Sande–Tukey radix 2 FFT

Cooley–Tukey radix 2 FFT

Coefficient to point-value representation: intuition



Divide. Break up polynomial into even- and odd-degree terms.


独Aeven(x) = a0 + a2 x + a4 x2 + a6 x3.

独Aodd (x) = a1 + a3 x + a5 x2 + a7 x3.


独A(−x) = Aeven(x2) – x Aodd(x2).Intuition. Choose two points to be ±1.

独A( 1) = Aeven(1) + 1 Aodd(1).

独A(−1) = Aeven(1) – 1 Aodd(1).

62

Can evaluate polynomial of degree n-1at 2 points by evaluating two polynomials of degree ½n - 1 at only 1 point.

we get to choose which ones!

Coefficient to point-value representation: intuition





独Aeven(x) = a0 + a2 x + a4 x2 + a6 x3.

独Aodd (x) = a1 + a3 x + a5 x2 + a7 x3.


独A(−x) = Aeven(x2) – x Aodd(x2).Intuition. Choose four complex points to be ±1, ±i.

独A( 1) = Aeven(1) + 1 Aodd(1).

独A(−1) = Aeven(1) – 1 Aodd(1).

独A( i ) = Aeven(−1) + i Aodd(−1).

独A( −i ) = Aeven(−1) – i Aodd(−1).

63

Can evaluate polynomial of degree n-1at 4 points by evaluating two polynomials of degree ½n - 1 at only 2 points.


Discrete Fourier transform



Key idea. Choose xk = ωk where ω is principal nth root of unity.

64

€

y0

y1

y2

y3

!yn−1

#

$

% % % % % % %

&

'

( ( ( ( ( ( (

=

1 1 1 1 " 11 ω1 ω2 ω3 " ωn−1

1 ω2 ω4 ω6 " ω2(n−1)

1 ω3 ω6 ω9 " ω3(n−1)

! ! ! ! # !1 ωn−1 ω2(n−1) ω3(n−1) " ω(n−1)(n−1)

#

$

% % % % % % %

&

'

( ( ( ( ( ( (

a0

a1

a2

a3

!an−1

#

$

% % % % % % %

&

'

( ( ( ( ( ( (

DFT Fourier matrix Fn

yk = A(ωk)


Roots of unity

Def. An nth root of unity is a complex number x such that xn = 1.

Fact. The nth roots of unity are: ω0, ω1, …, ωn–1

where ω = e2π i / n.

Pf. (ωk) n = (e2π i k / n) n = (eπ i ) 2k = (−1)2k = 1.

Fact. The ½nth roots of unity are: ν0, ν1, …, νn/2–1

where ν = ω2 = e4π i / n.

65

ω0 = ν0 = 1

ω1

ω2 = ν1

= i

ω3

ω4 = ν2 = −1

ω5

ω6 = ν3 = −i

ω7

n = 8

Fast Fourier transform

Goal. Evaluate a degree n – 1 polynomial A(x) = a0 + ... + an–1 xn–1 at its

nth roots of unity: ω0, ω1, …, ω n–1.


独Aeven(x) = a0 + a2 x + a4 x2 + … + an−2 x n/2–1.

独Aodd (x) = a1 + a3 x + a5 x2 + … + an−1 x n/2–1.

独A(x) = Aeven(x2) + x Aodd(x2).Conquer. Evaluate Aeven(x) and Aodd(x) at the ½nth roots of unity: ν0, ν1, …, νn/2–1

.

Combine.

独yk = A(ω k) = Aeven(ν k) + ω k Aodd (ν k), 0 ≤ k < n/2.

独yk+ ½n = A(ω k+ ½n) = Aeven(ν k) – ω k Aodd (ν k), 0 ≤ k < n/2.

66

ωk+ ½n = −ωk

νk = (ωk)2

νk = (ωk + ½n)2

FFT: implementation

Goal. Evaluate a degree n – 1 polynomial A(x) = a0 + ... + an–1 xn–1 at its

nth roots of unity: ω0, ω1, …, ω n–1.

独yk = A(ω k) = Aeven(ν k) + ω k Aodd (ν k), 0 ≤ k < n/2.

独yk+ ½n = A(ω k+ ½n) = Aeven(ν k) – ω k Aodd (ν k), 0 ≤ k < n/2.

67

FFT(n, a0, a1, a2, …, an–1) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (n = 1) RETURN a0.

(e0, e1, …, en/2–1) ← FFT(n / 2, a0, a2, a4, …, an–2).

(d0, d1, …, dn/2–1) ← FFT(n / 2, a1, a3, a5, …, an–1).

FOR k = 0 TO n / 2 – 1.

ωk ← e2π i k /n. yk ← ek + ωk dk.

yk + n/2 ← ek – ωk dk .

RETURN (y0, y1, y2, …, yn–1)._________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

2 T(n / 2)

Θ(n)

FFT: summary

Theorem. The FFT algorithm evaluates a degree n – 1 polynomial at each of

the nth roots of unity in O(n log n) arithmetic operations and O(n) extra space.

Pf.

68

assumes n is a power of 2

T (n) =

��

�(1) B7 n = 1

2T (n/2) + �(n) B7 n > 1<latexit sha1_base64="Ka5DcFJNlpZ3teGCEeCmHcRJgDk=">AAACr3icbVFda9swFJXdfXTZV5o+dg9i6UbKILPDoC2lo7CXPXaQrIHIy2T5OhGVZSNdlwST1/3H/Yf+iMmpB0uyCzKHc+85VzqOCyUtBsFvz9979PjJ0/1nrecvXr563T7ofLd5aQSMRK5yM465BSU1jFCignFhgGexgpv49kvdv7kDY2Wuh7gsIMr4TMtUCo6OmrZ/DXv6hLILell/WiyGmdSVcI521WIXbDgH5L3whL6nDGGBFZUpPdZuPDymK8rYpH8KWeRGB87p46D2+uCcaKPUO8rPf5UtBjppVk3b3aAfrIvugrABXdLU9fTA67AkF2UGGoXi1k7CoMCo4galUODuXloouLjlM5g4qHkGNqrWga3oO8ckNM2NOxrpmv1XUfHM2mUWu8mM49xu92ryf71JielZVEldlAhaPCxKS0Uxp3X6NJEGBKqlA1wY6e5KxZwbLtD9o40ta+8CxMZLqkWppcgT2GIVLtDwOsVwO7NdMBr0z/vht0/dq7Mmzn1yRN6SHgnJKbkiX8k1GRFB7r2Od+S98Qf+2P/h/3wY9b1Gc0g2ypd/AF7fy8Q=</latexit><latexit sha1_base64="Ka5DcFJNlpZ3teGCEeCmHcRJgDk=">AAACr3icbVFda9swFJXdfXTZV5o+dg9i6UbKILPDoC2lo7CXPXaQrIHIy2T5OhGVZSNdlwST1/3H/Yf+iMmpB0uyCzKHc+85VzqOCyUtBsFvz9979PjJ0/1nrecvXr563T7ofLd5aQSMRK5yM465BSU1jFCignFhgGexgpv49kvdv7kDY2Wuh7gsIMr4TMtUCo6OmrZ/DXv6hLILell/WiyGmdSVcI521WIXbDgH5L3whL6nDGGBFZUpPdZuPDymK8rYpH8KWeRGB87p46D2+uCcaKPUO8rPf5UtBjppVk3b3aAfrIvugrABXdLU9fTA67AkF2UGGoXi1k7CoMCo4galUODuXloouLjlM5g4qHkGNqrWga3oO8ckNM2NOxrpmv1XUfHM2mUWu8mM49xu92ryf71JielZVEldlAhaPCxKS0Uxp3X6NJEGBKqlA1wY6e5KxZwbLtD9o40ta+8CxMZLqkWppcgT2GIVLtDwOsVwO7NdMBr0z/vht0/dq7Mmzn1yRN6SHgnJKbkiX8k1GRFB7r2Od+S98Qf+2P/h/3wY9b1Gc0g2ypd/AF7fy8Q=</latexit><latexit sha1_base64="Ka5DcFJNlpZ3teGCEeCmHcRJgDk=">AAACr3icbVFda9swFJXdfXTZV5o+dg9i6UbKILPDoC2lo7CXPXaQrIHIy2T5OhGVZSNdlwST1/3H/Yf+iMmpB0uyCzKHc+85VzqOCyUtBsFvz9979PjJ0/1nrecvXr563T7ofLd5aQSMRK5yM465BSU1jFCignFhgGexgpv49kvdv7kDY2Wuh7gsIMr4TMtUCo6OmrZ/DXv6hLILell/WiyGmdSVcI521WIXbDgH5L3whL6nDGGBFZUpPdZuPDymK8rYpH8KWeRGB87p46D2+uCcaKPUO8rPf5UtBjppVk3b3aAfrIvugrABXdLU9fTA67AkF2UGGoXi1k7CoMCo4galUODuXloouLjlM5g4qHkGNqrWga3oO8ckNM2NOxrpmv1XUfHM2mUWu8mM49xu92ryf71JielZVEldlAhaPCxKS0Uxp3X6NJEGBKqlA1wY6e5KxZwbLtD9o40ta+8CxMZLqkWppcgT2GIVLtDwOsVwO7NdMBr0z/vht0/dq7Mmzn1yRN6SHgnJKbkiX8k1GRFB7r2Od+S98Qf+2P/h/3wY9b1Gc0g2ypd/AF7fy8Q=</latexit><latexit sha1_base64="Ka5DcFJNlpZ3teGCEeCmHcRJgDk=">AAACr3icbVFda9swFJXdfXTZV5o+dg9i6UbKILPDoC2lo7CXPXaQrIHIy2T5OhGVZSNdlwST1/3H/Yf+iMmpB0uyCzKHc+85VzqOCyUtBsFvz9979PjJ0/1nrecvXr563T7ofLd5aQSMRK5yM465BSU1jFCignFhgGexgpv49kvdv7kDY2Wuh7gsIMr4TMtUCo6OmrZ/DXv6hLILell/WiyGmdSVcI521WIXbDgH5L3whL6nDGGBFZUpPdZuPDymK8rYpH8KWeRGB87p46D2+uCcaKPUO8rPf5UtBjppVk3b3aAfrIvugrABXdLU9fTA67AkF2UGGoXi1k7CoMCo4galUODuXloouLjlM5g4qHkGNqrWga3oO8ckNM2NOxrpmv1XUfHM2mUWu8mM49xu92ryf71JielZVEldlAhaPCxKS0Uxp3X6NJEGBKqlA1wY6e5KxZwbLtD9o40ta+8CxMZLqkWppcgT2GIVLtDwOsVwO7NdMBr0z/vht0/dq7Mmzn1yRN6SHgnJKbkiX8k1GRFB7r2Od+S98Qf+2P/h/3wY9b1Gc0g2ypd/AF7fy8Q=</latexit>


a0, a1, . . . , an�1


O(n log n)FFT

???(x0, y0), . . . , (xn�1, yn�1)

<latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit>

When computing the FFT of (a0, a1, a2, …, a7), which are the first two coefficients involved in an arithmetic operation?

A. a0 and a1.

B. a0 and a2.

C. a0 and a4.

D. a0 and a7.

E. None of the above.

69

Divide-and-conquer II: quiz 8 FFT: recursion tree

70

a0, a1, a2, a3, a4, a5, a6, a7

a1, a3, a5, a7a0, a2, a4, a6

a3, a7a1, a5a0, a4 a2, a6

a0 a4 a2 a6 a1 a5 a3 a7

“bit-reversed” order

000 100 010 110 001 101 011 111

inverseperfect shuffle

FFT: Fourier matrix decomposition

Alternative viewpoint. FFT is a recursive decomposition of Fourier matrix.

71

€

Fn =

1 1 1 1 ! 11 ω1 ω2 ω3 ! ωn−1

1 ω2 ω4 ω6 ! ω2(n−1)

1 ω3 ω6 ω9 ! ω3(n−1)

" " " " # "1 ωn−1 ω2(n−1) ω3(n−1) ! ω(n−1)(n−1)

$

%

& & & & & & &

'

(

) ) ) ) ) ) )

Dn =

��

�0 0 0 . . . 00 �1 0 . . . 00 0 �2 . . . 0XXX

XXXXXX X X X XXX

0 0 0 . . . �n�1

��

<latexit sha1_base64="tX7Y8OLNVIIGfdLBMKQsVCHpwWI=">AAADHHicfZLNahsxEMe127RN3C8nOfYiYhp6qdkNhaaHQqA95JhCnQ+8rtFqx46IVlqk2WCzLL22T9Gn6SnkWuhz5AUqy0sS26UDWv7856cZabRpIYXFKPoThA/WHj56vL7RevL02fMX7c2tY6tLw6HHtdTmNGUWpFDQQ4ESTgsDLE8lnKQXH2f5k0swVmj1BacFDHI2VmIkOENnDds3n4aKfqCtJIWxUFWaMzRiUreoi0TnMGZfI0p3afPZpUmm0XqdJJ7y5pyM6X+pe+SeJ1eo5LJxVoWDb5X3FgovtW26VOpNXM9qJ6Cyu7sN252oG/mgqyJuRIc0cTTcDLZcZV7moJBLZm0/jgocVMyg4BLqVlJaKBi/YGPoO6lYDnZQ+dep6SvnZHSkjVsKqXfv76hYbu00Tx3pTnhul3Mz81+5fomj/UElVFEiKD5vNColRU1nT00zYYCjnDrBuBHurJSfM8M4uh9ioYuvXQBfuEk1KZXgOoMlV+IEDavdFOPlma2K3l73fTf+/LZzsN+Mc528JDvkNYnJO3JADskR6REenAXfgu/Bj/Bn+Cu8Cq/naBg0e7bJQoS//wLYQumu</latexit><latexit sha1_base64="tX7Y8OLNVIIGfdLBMKQsVCHpwWI=">AAADHHicfZLNahsxEMe127RN3C8nOfYiYhp6qdkNhaaHQqA95JhCnQ+8rtFqx46IVlqk2WCzLL22T9Gn6SnkWuhz5AUqy0sS26UDWv7856cZabRpIYXFKPoThA/WHj56vL7RevL02fMX7c2tY6tLw6HHtdTmNGUWpFDQQ4ESTgsDLE8lnKQXH2f5k0swVmj1BacFDHI2VmIkOENnDds3n4aKfqCtJIWxUFWaMzRiUreoi0TnMGZfI0p3afPZpUmm0XqdJJ7y5pyM6X+pe+SeJ1eo5LJxVoWDb5X3FgovtW26VOpNXM9qJ6Cyu7sN252oG/mgqyJuRIc0cTTcDLZcZV7moJBLZm0/jgocVMyg4BLqVlJaKBi/YGPoO6lYDnZQ+dep6SvnZHSkjVsKqXfv76hYbu00Tx3pTnhul3Mz81+5fomj/UElVFEiKD5vNColRU1nT00zYYCjnDrBuBHurJSfM8M4uh9ioYuvXQBfuEk1KZXgOoMlV+IEDavdFOPlma2K3l73fTf+/LZzsN+Mc528JDvkNYnJO3JADskR6REenAXfgu/Bj/Bn+Cu8Cq/naBg0e7bJQoS//wLYQumu</latexit><latexit sha1_base64="tX7Y8OLNVIIGfdLBMKQsVCHpwWI=">AAADHHicfZLNahsxEMe127RN3C8nOfYiYhp6qdkNhaaHQqA95JhCnQ+8rtFqx46IVlqk2WCzLL22T9Gn6SnkWuhz5AUqy0sS26UDWv7856cZabRpIYXFKPoThA/WHj56vL7RevL02fMX7c2tY6tLw6HHtdTmNGUWpFDQQ4ESTgsDLE8lnKQXH2f5k0swVmj1BacFDHI2VmIkOENnDds3n4aKfqCtJIWxUFWaMzRiUreoi0TnMGZfI0p3afPZpUmm0XqdJJ7y5pyM6X+pe+SeJ1eo5LJxVoWDb5X3FgovtW26VOpNXM9qJ6Cyu7sN252oG/mgqyJuRIc0cTTcDLZcZV7moJBLZm0/jgocVMyg4BLqVlJaKBi/YGPoO6lYDnZQ+dep6SvnZHSkjVsKqXfv76hYbu00Tx3pTnhul3Mz81+5fomj/UElVFEiKD5vNColRU1nT00zYYCjnDrBuBHurJSfM8M4uh9ioYuvXQBfuEk1KZXgOoMlV+IEDavdFOPlma2K3l73fTf+/LZzsN+Mc528JDvkNYnJO3JADskR6REenAXfgu/Bj/Bn+Cu8Cq/naBg0e7bJQoS//wLYQumu</latexit><latexit sha1_base64="tX7Y8OLNVIIGfdLBMKQsVCHpwWI=">AAADHHicfZLNahsxEMe127RN3C8nOfYiYhp6qdkNhaaHQqA95JhCnQ+8rtFqx46IVlqk2WCzLL22T9Gn6SnkWuhz5AUqy0sS26UDWv7856cZabRpIYXFKPoThA/WHj56vL7RevL02fMX7c2tY6tLw6HHtdTmNGUWpFDQQ4ESTgsDLE8lnKQXH2f5k0swVmj1BacFDHI2VmIkOENnDds3n4aKfqCtJIWxUFWaMzRiUreoi0TnMGZfI0p3afPZpUmm0XqdJJ7y5pyM6X+pe+SeJ1eo5LJxVoWDb5X3FgovtW26VOpNXM9qJ6Cyu7sN252oG/mgqyJuRIc0cTTcDLZcZV7moJBLZm0/jgocVMyg4BLqVlJaKBi/YGPoO6lYDnZQ+dep6SvnZHSkjVsKqXfv76hYbu00Tx3pTnhul3Mz81+5fomj/UElVFEiKD5vNColRU1nT00zYYCjnDrBuBHurJSfM8M4uh9ioYuvXQBfuEk1KZXgOoMlV+IEDavdFOPlma2K3l73fTf+/LZzsN+Mc528JDvkNYnJO3JADskR6REenAXfgu/Bj/Bn+Cu8Cq/naBg0e7bJQoS//wLYQumu</latexit>

In =

��

1 0 0 . . . 00 1 0 . . . 00 0 1 . . . 0XXX

XXXXXX X X X XXX

0 0 0 . . . 1

��

<latexit sha1_base64="4P72qi/5MSTFQ1NgES4W9V5utwM=">AAAC9nicfVFNj9MwEHUCLN3CLm05crGoQJyqZBeJckCqxAVuRdrQrpqqcpxpa63jRPakahX1t3BCe90/wg/g3+Ck4aMt2pFGenpv/GY8E2VSGPS8n4774OGjk8eN0+aTp2fnz1rtzleT5ppDwFOZ6nHEDEihIECBEsaZBpZEEkbRzcdSH61AG5GqK9xkME3YQom54AwtNWv9+DxT9ANthhEshCqihKEW622T2vDpa+rVGcYpmgqHYSWWpH+f+LvgSAxXNXMMbPEfVHF7fgfd/NIwBBX/nXrW6no9rwp6DPwadEkdw1nb6Vg7niegkEtmzMT3MpwWTKPgErbNMDeQMX7DFjCxULEEzLSo9r6lrywT03mqbSqkFfvvi4IlxmySyFbaCZfmUCvJ/2mTHOf9aSFUliMovms0zyXFlJZHpLHQwFFuLGBcCzsr5UumGUd76r0ulXcGfO8nxTpXgqcxHLAS16jZ1m7RP9zZMQgueu97/pe33UG/XmeDvCAvyRvik3dkQD6RIQkIdy6daydyuLtxv7nf3dtdqevUb56TvXDvfgGMxtrY</latexit><latexit sha1_base64="4P72qi/5MSTFQ1NgES4W9V5utwM=">AAAC9nicfVFNj9MwEHUCLN3CLm05crGoQJyqZBeJckCqxAVuRdrQrpqqcpxpa63jRPakahX1t3BCe90/wg/g3+Ck4aMt2pFGenpv/GY8E2VSGPS8n4774OGjk8eN0+aTp2fnz1rtzleT5ppDwFOZ6nHEDEihIECBEsaZBpZEEkbRzcdSH61AG5GqK9xkME3YQom54AwtNWv9+DxT9ANthhEshCqihKEW622T2vDpa+rVGcYpmgqHYSWWpH+f+LvgSAxXNXMMbPEfVHF7fgfd/NIwBBX/nXrW6no9rwp6DPwadEkdw1nb6Vg7niegkEtmzMT3MpwWTKPgErbNMDeQMX7DFjCxULEEzLSo9r6lrywT03mqbSqkFfvvi4IlxmySyFbaCZfmUCvJ/2mTHOf9aSFUliMovms0zyXFlJZHpLHQwFFuLGBcCzsr5UumGUd76r0ulXcGfO8nxTpXgqcxHLAS16jZ1m7RP9zZMQgueu97/pe33UG/XmeDvCAvyRvik3dkQD6RIQkIdy6daydyuLtxv7nf3dtdqevUb56TvXDvfgGMxtrY</latexit><latexit sha1_base64="4P72qi/5MSTFQ1NgES4W9V5utwM=">AAAC9nicfVFNj9MwEHUCLN3CLm05crGoQJyqZBeJckCqxAVuRdrQrpqqcpxpa63jRPakahX1t3BCe90/wg/g3+Ck4aMt2pFGenpv/GY8E2VSGPS8n4774OGjk8eN0+aTp2fnz1rtzleT5ppDwFOZ6nHEDEihIECBEsaZBpZEEkbRzcdSH61AG5GqK9xkME3YQom54AwtNWv9+DxT9ANthhEshCqihKEW622T2vDpa+rVGcYpmgqHYSWWpH+f+LvgSAxXNXMMbPEfVHF7fgfd/NIwBBX/nXrW6no9rwp6DPwadEkdw1nb6Vg7niegkEtmzMT3MpwWTKPgErbNMDeQMX7DFjCxULEEzLSo9r6lrywT03mqbSqkFfvvi4IlxmySyFbaCZfmUCvJ/2mTHOf9aSFUliMovms0zyXFlJZHpLHQwFFuLGBcCzsr5UumGUd76r0ulXcGfO8nxTpXgqcxHLAS16jZ1m7RP9zZMQgueu97/pe33UG/XmeDvCAvyRvik3dkQD6RIQkIdy6daydyuLtxv7nf3dtdqevUb56TvXDvfgGMxtrY</latexit><latexit sha1_base64="4P72qi/5MSTFQ1NgES4W9V5utwM=">AAAC9nicfVFNj9MwEHUCLN3CLm05crGoQJyqZBeJckCqxAVuRdrQrpqqcpxpa63jRPakahX1t3BCe90/wg/g3+Ck4aMt2pFGenpv/GY8E2VSGPS8n4774OGjk8eN0+aTp2fnz1rtzleT5ppDwFOZ6nHEDEihIECBEsaZBpZEEkbRzcdSH61AG5GqK9xkME3YQom54AwtNWv9+DxT9ANthhEshCqihKEW622T2vDpa+rVGcYpmgqHYSWWpH+f+LvgSAxXNXMMbPEfVHF7fgfd/NIwBBX/nXrW6no9rwp6DPwadEkdw1nb6Vg7niegkEtmzMT3MpwWTKPgErbNMDeQMX7DFjCxULEEzLSo9r6lrywT03mqbSqkFfvvi4IlxmySyFbaCZfmUCvJ/2mTHOf9aSFUliMovms0zyXFlJZHpLHQwFFuLGBcCzsr5UumGUd76r0ulXcGfO8nxTpXgqcxHLAS16jZ1m7RP9zZMQgueu97/pe33UG/XmeDvCAvyRvik3dkQD6RIQkIdy6daydyuLtxv7nf3dtdqevUb56TvXDvfgGMxtrY</latexit>

a =

��

a0

a1

a2

XXXan�1

��

<latexit sha1_base64="BtFIwEetBLCGkmmzIaI2pwmYZuI=">AAACinicbVBdSxtBFJ2stdVUa9RHX4aGgi+GXVtQKQWhPvioYKqQDeHu7E0cnJ1ZZu5KwrK/r7/BH+GrfXUSV2JiDwycOWfux5wkV9JRGD40gpUPqx8/ra03P29sftlqbe/8caawArvCKGNvEnCopMYuSVJ4k1uELFF4ndz9nvrX92idNPqKJjn2MxhpOZQCyEuDFgD/xZtxgiOpyyQDsnJcNbkHDEIexzWN5vTwlcb3qSE3N0p9EFXTa4w6nfcatNphJ5yBvydRTdqsxsVgu7ETp0YUGWoSCpzrRWFO/RIsSaGwasaFwxzEHYyw56mGDF2/nGVR8W9eSfnQWH808Zn6tqKEzLlJlviXfsNbt+xNxf95vYKGx/1S6rwg1OJl0LBQnAyfBstTaVGQmngCwkq/Kxe3YEGQj39hyqx3jmLhJ+W40FKYFJdURWOyUPkUo+XM3pPuYeekE13+aJ8e13GusT32le2ziB2xU3bOLliXCfaXPbIn9i/YDL4HJ8HPl6dBo67ZZQsIzp4BhJXBjA==</latexit><latexit sha1_base64="BtFIwEetBLCGkmmzIaI2pwmYZuI=">AAACinicbVBdSxtBFJ2stdVUa9RHX4aGgi+GXVtQKQWhPvioYKqQDeHu7E0cnJ1ZZu5KwrK/r7/BH+GrfXUSV2JiDwycOWfux5wkV9JRGD40gpUPqx8/ra03P29sftlqbe/8caawArvCKGNvEnCopMYuSVJ4k1uELFF4ndz9nvrX92idNPqKJjn2MxhpOZQCyEuDFgD/xZtxgiOpyyQDsnJcNbkHDEIexzWN5vTwlcb3qSE3N0p9EFXTa4w6nfcatNphJ5yBvydRTdqsxsVgu7ETp0YUGWoSCpzrRWFO/RIsSaGwasaFwxzEHYyw56mGDF2/nGVR8W9eSfnQWH808Zn6tqKEzLlJlviXfsNbt+xNxf95vYKGx/1S6rwg1OJl0LBQnAyfBstTaVGQmngCwkq/Kxe3YEGQj39hyqx3jmLhJ+W40FKYFJdURWOyUPkUo+XM3pPuYeekE13+aJ8e13GusT32le2ziB2xU3bOLliXCfaXPbIn9i/YDL4HJ8HPl6dBo67ZZQsIzp4BhJXBjA==</latexit><latexit sha1_base64="BtFIwEetBLCGkmmzIaI2pwmYZuI=">AAACinicbVBdSxtBFJ2stdVUa9RHX4aGgi+GXVtQKQWhPvioYKqQDeHu7E0cnJ1ZZu5KwrK/r7/BH+GrfXUSV2JiDwycOWfux5wkV9JRGD40gpUPqx8/ra03P29sftlqbe/8caawArvCKGNvEnCopMYuSVJ4k1uELFF4ndz9nvrX92idNPqKJjn2MxhpOZQCyEuDFgD/xZtxgiOpyyQDsnJcNbkHDEIexzWN5vTwlcb3qSE3N0p9EFXTa4w6nfcatNphJ5yBvydRTdqsxsVgu7ETp0YUGWoSCpzrRWFO/RIsSaGwasaFwxzEHYyw56mGDF2/nGVR8W9eSfnQWH808Zn6tqKEzLlJlviXfsNbt+xNxf95vYKGx/1S6rwg1OJl0LBQnAyfBstTaVGQmngCwkq/Kxe3YEGQj39hyqx3jmLhJ+W40FKYFJdURWOyUPkUo+XM3pPuYeekE13+aJ8e13GusT32le2ziB2xU3bOLliXCfaXPbIn9i/YDL4HJ8HPl6dBo67ZZQsIzp4BhJXBjA==</latexit><latexit sha1_base64="BtFIwEetBLCGkmmzIaI2pwmYZuI=">AAACinicbVBdSxtBFJ2stdVUa9RHX4aGgi+GXVtQKQWhPvioYKqQDeHu7E0cnJ1ZZu5KwrK/r7/BH+GrfXUSV2JiDwycOWfux5wkV9JRGD40gpUPqx8/ra03P29sftlqbe/8caawArvCKGNvEnCopMYuSVJ4k1uELFF4ndz9nvrX92idNPqKJjn2MxhpOZQCyEuDFgD/xZtxgiOpyyQDsnJcNbkHDEIexzWN5vTwlcb3qSE3N0p9EFXTa4w6nfcatNphJ5yBvydRTdqsxsVgu7ETp0YUGWoSCpzrRWFO/RIsSaGwasaFwxzEHYyw56mGDF2/nGVR8W9eSfnQWH808Zn6tqKEzLlJlviXfsNbt+xNxf95vYKGx/1S6rwg1OJl0LBQnAyfBstTaVGQmngCwkq/Kxe3YEGQj39hyqx3jmLhJ+W40FKYFJdURWOyUPkUo+XM3pPuYeekE13+aJ8e13GusT32le2ziB2xU3bOLliXCfaXPbIn9i/YDL4HJ8HPl6dBo67ZZQsIzp4BhJXBjA==</latexit>

y = Fn a =

�In/2 Dn/2

In/2 �Dn/2

� �Fn/2 aeven

Fn/2 aodd

�

<latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit><latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit><latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit><latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit>

DFT

Fourier matrix

y = Fn a =

�In/2 Dn/2

In/2 �Dn/2

� �Fn/2 aeven

Fn/2 aodd

�

<latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit><latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit><latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit><latexit sha1_base64="R+HeDCq3YRZPvfc/Pd8U8dvZUXw=">AAAC6XicbZHdatswFMdl76vLPpp2l7sRCyu7WDO7DNYRBoWNssEuOpjbQhSMLJ8korJkJLkkCL/ErsZu9yJ7jb3NZNdlSdoDEn9+51NHWSm4sVH0Nwjv3L13/8HWw96jx0+ebvd3dk+NqjSDhCmh9HlGDQguIbHcCjgvNdAiE3CWXXxs/GeXoA1X8rtdljAp6EzyKWfUepT2/ywxGeEPzXWcSkxeY3oNeiSDGZcuK6jVfFH3vqROvjmo8R4m5ZxKqwq3X3/qICEr/v0VSkDm/2vcUvb4OnaEaergEmSbt45Vntdk5PVmxbQ/iIZRa/imiDsxQJ2dpDvBLskVqwqQlglqzDiOSjtxVFvOBPgRKwMlZRd0BmMvJS3ATFy76hq/9CTHU6X9kRa3dDXD0cKYZZH5SD/g3Gz6Gnibb1zZ6eHEcVlWFiS7ajStBLYKN/+Gc66BWbH0gjLN/ayYzammzPrfXevS1i6Brb3ELSrJmcphgwq7sJo2W4w3d3ZTJAfD98P429vB0WG3zi30HL1Ar1CM3qEj9BmdoASxYC/4GiTBaSjCH+HP8NdVaBh0Oc/QmoW//wFSjOU3</latexit>

Inverse discrete Fourier transform

Point-value ⇒ coefficient. Given n distinct points x0, ... , xn–1 and values y0, ... , yn–1, find unique polynomial a0 + a1x + ... + an–1 xn–1

,that has given values at given points.

72

Inverse DFT

€

a0

a1

a2

a3

!an−1

#

$

% % % % % % %

&

'

( ( ( ( ( ( (

=

1 1 1 1 " 11 ω1 ω2 ω3 " ωn−1

1 ω2 ω4 ω6 " ω2(n−1)

1 ω3 ω6 ω9 " ω3(n−1)

! ! ! ! # !1 ωn−1 ω2(n−1) ω3(n−1) " ω(n−1)(n−1)

#

$

% % % % % % %

&

'

( ( ( ( ( ( (

−1

y0

y1y2

y3

!yn−1

#

$

% % % % % % %

&

'

( ( ( ( ( ( (

Fourier matrix inverse (Fn)−1

Inverse discrete Fourier transform

Claim. Inverse of Fourier matrix Fn is given by following formula:

Consequence. To compute the inverse FFT, apply the same algorithmbut use ω–1

= e –2π i / n as principal nth root of unity (and divide the result by n).

€

Gn =1n

1 1 1 1 ! 11 ω−1 ω−2 ω−3 ! ω−(n−1)

1 ω−2 ω−4 ω−6 ! ω−2(n−1)

1 ω−3 ω−6 ω−9 ! ω−3(n−1)

" " " " # "1 ω−(n−1) ω−2(n−1) ω−3(n−1) ! ω−(n−1)(n−1)

$

%

& & & & & & &

'

(

) ) ) ) ) ) )

73

Fn / √n is a unitary matrix

Inverse FFT: proof of correctness

Claim. Fn and Gn are inverses.

Pf.

Summation lemma. Let ω be a principal nth root of unity. Then

Pf.

独If k is a multiple of n, then ωk = 1 ⇒ series sums to n.

独Each nth root of unity ωk is a root of xn – 1 = (x – 1) (1 + x + x2 + ... + xn−1).

独if ωk ≠ 1, then 1 + ωk + ωk(2) + … + ωk(n−1) = 0 ⇒ series sums to 0. ▪

74

summation lemma (below)

(FnGn)kk� =1

n

n�1�

j=0

�kj��jk�=

1

n

n�1�

j=0

�(k�k�)j =

�1 B7 k = k�

0 Qi?2`rBb2<latexit sha1_base64="CafxWfe1s7GPmJcicwse+7gvYVQ=">AAADDnicjVLditNAFJ7Ev7X+dddLbwaLtgu2JCK4IoUFQQVvVrDuQtMNk+lJO00yiTMn2jrkIXwar8RbX8EH8D2cdMNqu1544MDH953vnJkzExWp0Oh5Px330uUrV6/tXG/duHnr9p327t57nZeKw4jnaa5OIqYhFRJGKDCFk0IBy6IUjqPkRa0ffwSlRS7f4aqAScZmUsSCM7RU2P4VpBAj7dGXoaSvbAZKzOa4H5ok6VY0eD602QpixbjxKyMtpcssNIuhV50a2fctkWcwY6cmWfzB/UXj/l9zL+kn3f3FuSeCmZCG25vpqhU8oj59SAOEJRoR10U0oUOadGkQ1Kp3ruY4B/VJaLAukNOmQ9jueANvHfQi8BvQIU0chbvOXjDNeZmBRJ4yrce+V+DEMIWCp3XzUkPBeMJmMLZQsgz0xKzfo6IPLDOlca5sSqRr9m+HYZnWqyyylRnDud7WavJf2rjE+GBihCxKBMnPBsVlSjGn9ePSqVDAMV1ZwLgS9qyUz5ldP9ovsDFl3bsAvnETsyyl4PkUttgUl6hYvUV/e2cXwejx4NnAf/ukc3jQrHOH3CP3SY/45Ck5JK/JERkR7rxxPjifHeN+cb+639zvZ6Wu03juko1wf/wGpJL1Lg==</latexit><latexit sha1_base64="CafxWfe1s7GPmJcicwse+7gvYVQ=">AAADDnicjVLditNAFJ7Ev7X+dddLbwaLtgu2JCK4IoUFQQVvVrDuQtMNk+lJO00yiTMn2jrkIXwar8RbX8EH8D2cdMNqu1544MDH953vnJkzExWp0Oh5Px330uUrV6/tXG/duHnr9p327t57nZeKw4jnaa5OIqYhFRJGKDCFk0IBy6IUjqPkRa0ffwSlRS7f4aqAScZmUsSCM7RU2P4VpBAj7dGXoaSvbAZKzOa4H5ok6VY0eD602QpixbjxKyMtpcssNIuhV50a2fctkWcwY6cmWfzB/UXj/l9zL+kn3f3FuSeCmZCG25vpqhU8oj59SAOEJRoR10U0oUOadGkQ1Kp3ruY4B/VJaLAukNOmQ9jueANvHfQi8BvQIU0chbvOXjDNeZmBRJ4yrce+V+DEMIWCp3XzUkPBeMJmMLZQsgz0xKzfo6IPLDOlca5sSqRr9m+HYZnWqyyylRnDud7WavJf2rjE+GBihCxKBMnPBsVlSjGn9ePSqVDAMV1ZwLgS9qyUz5ldP9ovsDFl3bsAvnETsyyl4PkUttgUl6hYvUV/e2cXwejx4NnAf/ukc3jQrHOH3CP3SY/45Ck5JK/JERkR7rxxPjifHeN+cb+639zvZ6Wu03juko1wf/wGpJL1Lg==</latexit><latexit sha1_base64="CafxWfe1s7GPmJcicwse+7gvYVQ=">AAADDnicjVLditNAFJ7Ev7X+dddLbwaLtgu2JCK4IoUFQQVvVrDuQtMNk+lJO00yiTMn2jrkIXwar8RbX8EH8D2cdMNqu1544MDH953vnJkzExWp0Oh5Px330uUrV6/tXG/duHnr9p327t57nZeKw4jnaa5OIqYhFRJGKDCFk0IBy6IUjqPkRa0ffwSlRS7f4aqAScZmUsSCM7RU2P4VpBAj7dGXoaSvbAZKzOa4H5ok6VY0eD602QpixbjxKyMtpcssNIuhV50a2fctkWcwY6cmWfzB/UXj/l9zL+kn3f3FuSeCmZCG25vpqhU8oj59SAOEJRoR10U0oUOadGkQ1Kp3ruY4B/VJaLAukNOmQ9jueANvHfQi8BvQIU0chbvOXjDNeZmBRJ4yrce+V+DEMIWCp3XzUkPBeMJmMLZQsgz0xKzfo6IPLDOlca5sSqRr9m+HYZnWqyyylRnDud7WavJf2rjE+GBihCxKBMnPBsVlSjGn9ePSqVDAMV1ZwLgS9qyUz5ldP9ovsDFl3bsAvnETsyyl4PkUttgUl6hYvUV/e2cXwejx4NnAf/ukc3jQrHOH3CP3SY/45Ck5JK/JERkR7rxxPjifHeN+cb+639zvZ6Wu03juko1wf/wGpJL1Lg==</latexit><latexit sha1_base64="CafxWfe1s7GPmJcicwse+7gvYVQ=">AAADDnicjVLditNAFJ7Ev7X+dddLbwaLtgu2JCK4IoUFQQVvVrDuQtMNk+lJO00yiTMn2jrkIXwar8RbX8EH8D2cdMNqu1544MDH953vnJkzExWp0Oh5Px330uUrV6/tXG/duHnr9p327t57nZeKw4jnaa5OIqYhFRJGKDCFk0IBy6IUjqPkRa0ffwSlRS7f4aqAScZmUsSCM7RU2P4VpBAj7dGXoaSvbAZKzOa4H5ok6VY0eD602QpixbjxKyMtpcssNIuhV50a2fctkWcwY6cmWfzB/UXj/l9zL+kn3f3FuSeCmZCG25vpqhU8oj59SAOEJRoR10U0oUOadGkQ1Kp3ruY4B/VJaLAukNOmQ9jueANvHfQi8BvQIU0chbvOXjDNeZmBRJ4yrce+V+DEMIWCp3XzUkPBeMJmMLZQsgz0xKzfo6IPLDOlca5sSqRr9m+HYZnWqyyylRnDud7WavJf2rjE+GBihCxKBMnPBsVlSjGn9ePSqVDAMV1ZwLgS9qyUz5ldP9ovsDFl3bsAvnETsyyl4PkUttgUl6hYvUV/e2cXwejx4NnAf/ukc3jQrHOH3CP3SY/45Ck5JK/JERkR7rxxPjifHeN+cb+639zvZ6Wu03juko1wf/wGpJL1Lg==</latexit>

n�1�

j=0

�kj =

�n B7 k � 0 (mod n)

0 Qi?2`rBb2<latexit sha1_base64="SrB/TWKiFoQ4PYxVkTe6X/0N4DI=">AAACp3icbVFdixMxFM2MX2v92O6ub74EiyKiZUYEV5aFBV98rGB3C023ZDK3bbaZZEzurC1h/J+++0PMdAexXS8EDueee25ykpVKOkySX1F85+69+w/2HnYePX7ydL97cHjuTGUFDIVRxo4y7kBJDUOUqGBUWuBFpuAiW35u+hfXYJ00+huuS5gUfK7lTAqOgZp2fzJXFVN/dZrUl16/S2vKTAFzfumXVwGfnLKTDstgLrUXYY2rO+wt1fQVZQgr9HLWiOiSMvheyWuaUFYWJve6ZqxRJn+VBhdgf0gHwQF03rpNu72kn2yK3gZpC3qkrcH0IDpkuRFVARqF4s6N06TEiecWpVCNeeWg5GLJ5zAOUPMC3MRvgqrpy8DkdGZsOBrphv13wvPCuXWRBWXBceF2ew35v964wtnxxEtdVgha3CyaVYqioU3qNJcWBKp1AFxYGe5KxYJbLjD8zdaWjXcJYuslflVpKUwOO6zCFVrepJjuZnYbDN/3P/XTrx96Z8dtnHvkOXlBXpOUfCRn5AsZkCER5HfUiY6iZ/GbeBCfx6MbaRy1M0dkq2L+B+kA0JM=</latexit><latexit sha1_base64="SrB/TWKiFoQ4PYxVkTe6X/0N4DI=">AAACp3icbVFdixMxFM2MX2v92O6ub74EiyKiZUYEV5aFBV98rGB3C023ZDK3bbaZZEzurC1h/J+++0PMdAexXS8EDueee25ykpVKOkySX1F85+69+w/2HnYePX7ydL97cHjuTGUFDIVRxo4y7kBJDUOUqGBUWuBFpuAiW35u+hfXYJ00+huuS5gUfK7lTAqOgZp2fzJXFVN/dZrUl16/S2vKTAFzfumXVwGfnLKTDstgLrUXYY2rO+wt1fQVZQgr9HLWiOiSMvheyWuaUFYWJve6ZqxRJn+VBhdgf0gHwQF03rpNu72kn2yK3gZpC3qkrcH0IDpkuRFVARqF4s6N06TEiecWpVCNeeWg5GLJ5zAOUPMC3MRvgqrpy8DkdGZsOBrphv13wvPCuXWRBWXBceF2ew35v964wtnxxEtdVgha3CyaVYqioU3qNJcWBKp1AFxYGe5KxYJbLjD8zdaWjXcJYuslflVpKUwOO6zCFVrepJjuZnYbDN/3P/XTrx96Z8dtnHvkOXlBXpOUfCRn5AsZkCER5HfUiY6iZ/GbeBCfx6MbaRy1M0dkq2L+B+kA0JM=</latexit><latexit sha1_base64="SrB/TWKiFoQ4PYxVkTe6X/0N4DI=">AAACp3icbVFdixMxFM2MX2v92O6ub74EiyKiZUYEV5aFBV98rGB3C023ZDK3bbaZZEzurC1h/J+++0PMdAexXS8EDueee25ykpVKOkySX1F85+69+w/2HnYePX7ydL97cHjuTGUFDIVRxo4y7kBJDUOUqGBUWuBFpuAiW35u+hfXYJ00+huuS5gUfK7lTAqOgZp2fzJXFVN/dZrUl16/S2vKTAFzfumXVwGfnLKTDstgLrUXYY2rO+wt1fQVZQgr9HLWiOiSMvheyWuaUFYWJve6ZqxRJn+VBhdgf0gHwQF03rpNu72kn2yK3gZpC3qkrcH0IDpkuRFVARqF4s6N06TEiecWpVCNeeWg5GLJ5zAOUPMC3MRvgqrpy8DkdGZsOBrphv13wvPCuXWRBWXBceF2ew35v964wtnxxEtdVgha3CyaVYqioU3qNJcWBKp1AFxYGe5KxYJbLjD8zdaWjXcJYuslflVpKUwOO6zCFVrepJjuZnYbDN/3P/XTrx96Z8dtnHvkOXlBXpOUfCRn5AsZkCER5HfUiY6iZ/GbeBCfx6MbaRy1M0dkq2L+B+kA0JM=</latexit><latexit sha1_base64="SrB/TWKiFoQ4PYxVkTe6X/0N4DI=">AAACp3icbVFdixMxFM2MX2v92O6ub74EiyKiZUYEV5aFBV98rGB3C023ZDK3bbaZZEzurC1h/J+++0PMdAexXS8EDueee25ykpVKOkySX1F85+69+w/2HnYePX7ydL97cHjuTGUFDIVRxo4y7kBJDUOUqGBUWuBFpuAiW35u+hfXYJ00+huuS5gUfK7lTAqOgZp2fzJXFVN/dZrUl16/S2vKTAFzfumXVwGfnLKTDstgLrUXYY2rO+wt1fQVZQgr9HLWiOiSMvheyWuaUFYWJve6ZqxRJn+VBhdgf0gHwQF03rpNu72kn2yK3gZpC3qkrcH0IDpkuRFVARqF4s6N06TEiecWpVCNeeWg5GLJ5zAOUPMC3MRvgqrpy8DkdGZsOBrphv13wvPCuXWRBWXBceF2ew35v964wtnxxEtdVgha3CyaVYqioU3qNJcWBKp1AFxYGe5KxYJbLjD8zdaWjXcJYuslflVpKUwOO6zCFVrepJjuZnYbDN/3P/XTrx96Z8dtnHvkOXlBXpOUfCRn5AsZkCER5HfUiY6iZ/GbeBCfx6MbaRy1M0dkq2L+B+kA0JM=</latexit>

(FnGn)kk� =1

n

n�1�

j=0

�kj��jk�=

1

n

n�1�

j=0

�(k�k�)j =

�1 B7 k = k�


(FnGn)kk� =1

n

n�1�

j=0

�kj��jk�=

1

n

n�1�

j=0

�(k�k�)j =

�1 B7 k = k�


Inverse FFT: implementation

Note. Need to divide result by n.

75

INVERSE-FFT(n, y0, y1, y2, …, yn–1) __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

IF (n = 1) RETURN y0.

(e0, e1, …, en/2–1) ← INVERSE-FFT(n / 2, y0, y2, y4, …, yn–2).

(d0, d1, …, dn/2–1) ← INVERSE-FFT(n / 2, y1, y3, y5, …, yn–1).

FOR k = 0 TO n / 2 – 1.

ωk ← e–2π i k / n.

ak ← ek + ωk dk.

ak + n/2 ← ek – ωk dk.

RETURN (a0, a1, a2, …, an–1)._______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

switch roles of ai and yi

Inverse FFT: summary

Theorem. The inverse FFT algorithm interpolates a degree n – 1 polynomial

at each of the nth roots of unity in O(n log n) arithmetic operations.

Corollary. Can convert between coefficient and point-value representations

in O(n log n) arithmetic operations.

76

assumes n is a power of 2


a0, a1, . . . , an�1


O(n log n)FFT

inverse FFTO(n log n)

(x0, y0), . . . , (xn�1, yn�1)<latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit><latexit sha1_base64="13wQhaZoRAHHzqQQKZgkdsRzkWQ=">AAACVHicbVDLSgMxFM2Mr/quunQTLIKilhkR1F3BjUsFq0JbhkzmVoOZZEjuSMvQj/Br3OpXiB8jmBm7sNVDAodz7s3NPXEmhcUg+PT8mdm5+YXa4tLyyuraen1j89bq3HBocy21uY+ZBSkUtFGghPvMAEtjCXfx00Xp3z2DsUKrGxxm0EvZgxJ9wRk6Kaof7A2i4JAOo2D/kHbdkYlGW1JnFOooHJVmRfajeiNoBhXoXxKOSYOMcRVteJvdRPM8BYVcMms7YZBhr2AGBZcwWurmFjLGn9gDdBxVLAXbK6qtRnTXKQnta+OuQlqpvzsKllo7TGNXmTJ8tNNeKf7ndXLsn/UKobIcQfGfQf1cUtS0jIgmwgBHOXSEcSPcXyl/ZIZxdEFOTKnezoBPbFIMciW4TmBKlThAw0YuxXA6s7+kfdw8b4bXJ43W2TjOGtkmO2SPhOSUtMgluSJtwskLeSVv5N378L78GX/up9T3xj1bZAL+2jesjLKd</latexit>

Theorem. Given two polynomials A(x) = a0 + a1 x + … + an−1 xn−1

and B(x) = b0 + b1 x + … + bn−1 bn−1 of degree n – 1, can multiply

them in O(n log n) arithmetic operations.

Pf.

Polynomial multiplication

77

pad with 0s to make

n a power of 2

inverse FFTO(n log n)


a0, a1, . . . , an�1



two FFTsO(n log n)

point-value multiplication

O(n)

A(�0), . . . , A(�2n�1)

B(�0), . . . , B(�2n�1)

C(�0), . . . , C(�2n�1)<latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit><latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit><latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit><latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit>

A(�0), . . . , A(�2n�1)

B(�0), . . . , B(�2n�1)

C(�0), . . . , C(�2n�1)<latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit><latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit><latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit><latexit sha1_base64="je6OjH/w85vdqvC7MaeUC89Eaps=">AAACpnicbVFda9swFJXdrWvTrzR97ItYKDTQBjsMlr11zcvelsK8FOI0k+WbVFSWjHQ9Gox/aJ/3R6akoTQfF4QO55yrKx0luRQWg+DF83c+fNz9tLdfOzg8Oj6pnzZ+W10YDhHXUpv7hFmQQkGEAiXc5wZYlkgYJE+9uT74C8YKrX7hLIdRxqZKTARn6Khxvfp+GesMpuwhaF3RWKYarduv3uiyo67DqkXjuHa73Xq7xdrbbu2tWcf1ZtAOFkU3QbgETbKs/vjUa8Sp5kUGCrlk1g7DIMdRyQwKLqGqxYWFnPEnNoWhg4plYEflIqeKXjgmpRNt3FJIF+z7jpJl1s6yxDkzho92XZuT27RhgZPuqBQqLxAUfx00KSRFTeeh01QY4ChnDjBuhLsr5Y/MMI7ua1amLM7Oga+8pHwulOA6hTVW4jMaVrkUw/XMNkHUaX9rh3dfmjfdZZx75Jx8JpckJF/JDflB+iQinPzz9r2Gd+a3/J9+5A9erb637DkjK+X/+Q/+S8ss</latexit>

c0, c1, . . . , c2n�2<latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit><latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit><latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit><latexit sha1_base64="aF3CWRb6xDS5I1NCeso8q/oCVe8=">AAACa3icbVBNT9tAEN2YftBAS/g4tRxWREg9pJEdVQJuqFw4UqkBpDiy1uMJWbHetXbHiMjy/+HXcIX+CX4Dm5ADCYy00tN7M/N2Xloo6SgM/zeClQ8fP31e/dJcW//6baO1uXXuTGkB+2CUsZepcKikxj5JUnhZWBR5qvAivT6Z6hc3aJ00+h9NChzm4krLkQRBnkpafyAJO3GHx2NXCMCqG0Y9qWsOSfQezWOVGXJe6EBS9fSvXt1MWu2wG86KvwXRHLTZvM6SzcZWnBkoc9QESjg3iMKChpWwJEFh3YxLh972WlzhwEMtcnTDanZszfc9k/GRsf5p4jP29UQlcucmeeo7c0Fjt6xNyfe0QUmjw2EldVESangxGpWKk+HT5HgmLQKpiQcCrPR/5TAWVgD5fBdcZrsLhIVLqttSSzAZLrGKbsmK2qcYLWf2FvR73aNu9Pd3+/hwHucq+8H22E8WsQN2zE7ZGeszYHfsnj2wx8ZTsBN8D3ZfWoPGfGabLVSw/wypk7qA</latexit>

point-value representation

point-value representation

FFT in practice ?

78

Fastest Fourier transform in the West. [Frigo–Johnson]

独Optimized C library.

独Features: DFT, DCT, real, complex, any size, any dimension.

独Won 1999 Wilkinson Prize for Numerical Software.

独Portable, competitive with vendor-tuned code.

Implementation details.

独Core algorithm is an in-place, nonrecursive version of Cooley–Tukey.

独Instead of executing a fixed algorithm, it evaluates the hardware and uses a special-purpose compiler to generate an optimizedalgorithm catered to “shape” of the problem.

独Runs in O(n log n) time, even when n is prime.

独Multidimensional FFTs.

独Parallelism.

FFT in practice

79

http://www.fftw.org

Top 10 algorithms of the 20th century

80

JANUARY/FEBRUARY 2000 23

methods for this problem.Introducing the decompositional approach to ma-

trix computations revolutionized the field. G.W. Stew-art describes the history leading up to the decomposi-tional approach and presents a brief tour of the sixcentral decompositions that have evolved and are inuse today in many areas of scientific computation.

David Padua argues that the Fortran I compiler,with its parsing, analysis, and code-optimization tech-niques, qualifies as one of the top 10 “algorithms.” Thearticle describes the language, compiler, and opti-mization techniques that the first compiler had.

The QR Algorithm for computing eigenvalues of amatrix has transformed the approach to computing thespectrum of a matrix. Beresford Parlett takes usthrough the history of early eigenvalue computationsand the discovery of the family of algorithms referredto as the QR Algorithm.

Sorting is a central problem in many areas of com-puting so it is no surprise to see an approach to solvingthe problem as one of the top 10. Joseph JaJa describesQuicksort as one of the best practical sorting algorithmfor general inputs. In addition, its complexity analysisand its structure have been a rich source of inspirationfor developing general algorithm techniques for vari-ous applications.

Daniel Rockmore describes the FFT as an algo-rithm “the whole family can use.” The FFT is per-haps the most ubiquitous algorithm in use today toanalyze and manipulate digital or discrete data. TheFFT takes the operation count for discrete Fouriertransform from O(N2) to O(N log N).

Some recently discovered integer relation detectionalgorithms have become a centerpiece of the emerg-ing discipline of “experimental mathematics”—the useof modern computer technology as an exploratory toolin mathematical research. David Bailey describes the

integer relation problem: given n real numbers x1, …,xn, find the n integers a1, … , an (if they exist) such thata1x1 + … + anxn = 0. Originally, the algorithm was usedto find the coefficients of the minimal integer polyno-mial an algebraic number satisfied. However, more re-cently, researchers have used them to discover un-known mathematical identities, as well as to identifysome constants that arise in quantum field theory interms of mathematical constants.

The Fast Multipole Algorithm was developed orig-inally to calculate gravitational and electrostatic po-tentials. The method utilizes techniques to quicklycompute and combine the pair-wise approximationin O(N) operations. This has led to a significant re-duction in the computational complexity from O(N2)to O(N log N) to O(N) in certain important cases.John Board and Klaus Schulten describe the ap-proach and its importance in the field.

Your thoughts?We have had fun putting together this issue, and we

assume that some of you will have strong feelingsabout our selection. Please let us know what you think.

Jack Dongarra is a professor of computer science in theComputer Science Department at the University of Ten-nessee and a scientist in the mathematical science sectionof Oak Ridge National Lab. He received his BS in mathe-matics from Chicago State University, his MS in computerscience from the Illinois Institute of Technology, and hisPhD in applied mathematics from the University of NewMexico. Contact him at [email protected]; www.cs.utk.edu/~dongarra.

Francis Sullivan’s biography appears in his article onpage 69.

2 COMPUTING IN SCIENCE& ENGINEERING

Computational algorithms are probably as old as civilization.Sumerian cuneiform, one of the most ancient written records,consists partly of algorithm descriptions for reckoning in base60. And I suppose we could claim that the Druid algorithm forestimating the start of summer is embodied in Stonehenge.(That’s really hard hardware!)

Like so many other things that technology affects, algo-rithms have advanced in startling and unexpected ways in the20th century—at least it looks that way to us now. The algo-rithms we chose for this issue have been essential for progressin communications, health care, manufacturing, economics,weather prediction, defense, and fundamental science. Con-versely, progress in these areas has stimulated the search forever-better algorithms. I recall one late-night bull session onthe Maryland Shore when someone asked, “Who first ate acrab? After all, they don’t look very appetizing.’’ After the usualspeculations about the observed behavior of sea gulls, someonegave what must be the right answer—namely, “A very hungryperson first ate a crab.”

The flip side to “necessity is the mother of invention’’ is “in-vention creates its own necessity.’’ Our need for powerful ma-chines always exceeds their availability. Each significant com-putation brings insights that suggest the next, usually muchlarger, computation to be done. New algorithms are an attemptto bridge the gap between the demand for cycles and the avail-able supply of them. We’ve become accustomed to gaining theMoore’s Law factor of two every 18 months. In effect, Moore’sLaw changes the constant in front of the estimate of runningtime as a function of problem size. Important new algorithmsdo not come along every 1.5 years, but when they do, they canchange the exponent of the complexity!

For me, great algorithms are the poetry of computation.Just like verse, they can be terse, allusive, dense, and even

mysterious. But once unlocked, they cast a brilliant new lighton some aspect of computing. A colleague recently claimedthat he’d done only 15 minutes of productive work in hiswhole life. He wasn’t joking, because he was referring to the15 minutes during which he’d sketched out a fundamental op-timization algorithm. He regarded the previous years ofthought and investigation as a sunk cost that might or mightnot have paid off.

Researchers have cracked many hard problems since 1 Jan-uary 1900, but we are passing some even harder ones on to thenext century. In spite of a lot of good work, the question ofhow to extract information from extremely large masses ofdata is still almost untouched. There are still very big chal-lenges coming from more “traditional” tasks, too. For exam-ple, we need efficient methods to tell when the result of a largefloating-point calculation is likely to be correct. Think of theway that check sums function. The added computational costis very small, but the added confidence in the answer is large.Is there an analog for things such as huge, multidisciplinaryoptimizations? At an even deeper level is the issue of reason-able methods for solving specific cases of “impossible’’ prob-lems. Instances of NP-complete problems crop up in at-tempting to answer many practical questions. Are thereefficient ways to attack them?

I suspect that in the 21st century, things will be ripe for an-other revolution in our understanding of the foundations ofcomputational theory. Questions already arising from quan-tum computing and problems associated with the generationof random numbers seem to require that we somehow tie to-gether theories of computing, logic, and the nature of thephysical world.

The new century is not going to be very restful for us, but itis not going to be dull either!

THEJOY OF ALGORITHMS

Francis Sullivan, Associate Editor-in-Chief

THE THEME OF THIS FIRST-OF-THE-CENTURY ISSUE OF COMPUTING IN

SCIENCE & ENGINEERING IS ALGORITHMS. IN FACT, WE WERE BOLD

ENOUGH—AND PERHAPS FOOLISH ENOUGH—TO CALL THE 10 EXAMPLES WE’VE SE-

LECTED “THE TOP 10 ALGORITHMS OF THE CENTURY.”

F R O M T H EED I T O R S

Integer multiplication, redux

Integer multiplication. Given two n-bit integers a = an–1 … a1a0 and b = bn–1 … b1b0, compute their product a ⋅ b.

Convolution algorithm.

独Form two polynomials.

独Note: a = A(2), b = B(2).

独Compute C(x) = A(x) ⋅ B(x).

独Evaluate C(2) = a ⋅ b.

独Running time: O(n log n) floating-point operations.

Theory. [Schönhage–Strassen 1971]

独O(n log2 n) bit operations.

独O(n log n ⋅ log log n) bit operations.

Practice. [GNU Multiple Precision Arithmetic Library]Switches to FFT-based algorithm when n is large (≥ 5–10K).

81

A(x) = a0 + a1x + a2x2 + . . . + an�1x

n�1

B(x) = b0 + b1x + b2x2 + . . . + bn�1x

n�1<latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit><latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit><latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit><latexit sha1_base64="PziBkiliYlomVCoz/GH8C/tIHNY=">AAAC13icbVFba9swFJa9W5vukrZ764tY2OgYC3YotFsZ9LKHPXawtGVxGmT5pBGVJSMdDwcT2NPoa//I/s/+zWQnjDnpAZ3z8Z3LJx3FmRQWg+CP5z94+Ojxk7X11sbTZ89ftDe3zq3ODYc+11Kby5hZkEJBHwVKuMwMsDSWcBHfnFb5ix9grNDqG04zGKbsWomx4AwdNWr/Pt4t3tLokL75VHk2Cug750Na1LFHi6ueQ5FMNNqaKtX7cFZc1YFGUeukMSB2A6KJzRiHshuEPaFmriuuB97HzwVWM/8E44bgqN0JukFtdBWEC9AhCzsbbXpbUaJ5noJCLpm1gzDIcFgyg4JLmLWi3ILTvmHXMHBQsRTssKwXO6OvHZPQsTbuKKQ1+39HyVJrp2nsKlOGE7ucq8j7coMcxwfDUqgsR1B8LjTOJUVNq1+iiTDAUU4dYNwId1fKJ8wwju4vGyr17Ax44yVlkSvBdQJLrMQCDau2GC7vbBX0e90P3fDrXufoYLHONbJDXpFdEpJ9ckS+kDPSJ9x76X30Tr3P/nf/p//Lv52X+t6iZ5s0zL/7C7hY2FI=</latexit>

FFT over complex numbers; need O(log n) bits of precision

FFT over ring of integers (modulo a Fermat number)

3-SUM (REVISITED)

3-SUM. Given three sets X, Y, and Z of n integers each, determine whether

there is a triple i ∈ X, j ∈ Y, k ∈ Z such that i + j = k. Assumption. All integers are between 0 and m.

Goal. O(m log m + n log n) time.

82

a yes instance(4 + 15 = 19)

m = 19, n = 3X = { 4, 7, 10 }Y = { 5, 8, 15 }Z = { 4, 13, 19 }

Date post:	26-Apr-2023
Category:	Documents
Upload:	khangminh22
View:	0 times
Download:	0 times

Divide and Conquer II

Documents