Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | amy-stokes |
View: | 215 times |
Download: | 2 times |
The Fast Fourier Transform and
Applications to Multiplication
Prepared by
John Reif, Ph.D.
Analysis of Algorithms
Topics and Readings:
- The Fast Fourier Transform
Advanced Material :
- Using FFT to solve other Multipoint Evaluation Problems
- Applications to Multiplication
• Reading Selection:• CLR, Chapter 30
Nth Roots of Unity
• Assume Commutative Ring (R,+,·, 0,1)
is principal nth root of unity if
i 1 for i = 1, …, n-1
n = 1, and
• Example: for complex numbers
n-1jp
j=0
0 for 1 p n
2 i/ne
Example of nth Root of Unity for Complex Numbers
is the 8th root of unity2 i/8e
Fourier Matrix
1
2 2( 1)n
1 ( 1)( 1)
ijij
0
n-1
1 1 1
1
M ( ) = 1
1
so M( ) = for 0 i, j<n
a
given a =
a
n
n
n n n
Discrete Fourier Transform
Input a column n-vector a = (a0, …, an-1)T
Output an n-vector which is the product of the Fourier matrix times the input vector
n
0
n-1
n-1ik
i kk=0
DFT (a) = M( ) x a
f
= where
f
f = a
Inverse Fourier Transform-1 -1n
-1 -ijij
-1
n-1 n-1ik -kj k(i-j)
k=0 k=0
DFT (a) = M( ) x a
1 M( ) =
n
We must show M( ) M( ) = I
1 1 =
n n
0 if i-j 0 =
1 if i-j = 0
using i
Theorem
proof
n-1kp
k=0
dentity 0, for 1 p < n
Fourier Transform is Polynomial Evaluation at the Roots of Unity Input a column n-vector a = (a0, …, an-1)T
Output an n-vector (f0, …, fn-1)T which are the
values polynomial f(x)at the n roots of unity
0
n
n-1
ii
n-1j
jj=0
f
DFT (a) = where
f
f = f( ) and
f(x) = a x
Fast Fourier Transform
• Viewed as Evaluation Problem: naïve algorithm takes n2 ops
• Divide and Conquer gives FFT with O(n log n) ops for n a power of 2
• Key Idea:• If is nth root of unity
then 2 is n/2th root of unity• So can reduce the problem to two
subproblems of size n/2
Algorithm FFTn
• Input a = (a0, …, an-1)T, n a power of 2
T
' '0 n 0 2 2
12 2
T
" "0 n 1 3 1
12 2
' "i i i
[1] If n=1 then ouput
[2] f ,..., f (( , ,..., ) )
f ,..., f (( , ,..., ) )
n[3] For i=0, ..., 1 do f f f
2
Tn n
Tn n
i
FFT a a a
FFT a a a
' "n i i
i+2
0 1 n-1
f f f
[4] Output (f , f , ..., f )
i
FFT Circuit (also known as Butterfly Network)
• Total Recursion depth = log n• Communication Distance 2d at depth d
i 2i (n-1)ii 0 1 2 n-1
' i "i i i
i(n-2)' 2 i 2 2i 2 2i 0 2 4 n-2
i(n-2)" 2 i 2 2i 1 3 n-1
0'0
22n
2'n
1 22
f = a + a + a +...+a
f = f + f where
f = a + a ( ) + a ( ) +...+ a ( )
f = a + a ( ) +...+ a ( )
f
M ( )
fn
a
a
a
0 2 2
2
1"0
32n 1 3 1
2 2"n
1 12
(( , ,..., ) )
f
M ( ) (( , ,..., ) )
f
Tn n
Tn n
n
DFT a a a
a
aDFT a a a
a
' ' " "n i n i
i i2 2
n1n 2 n 2 i 2i2
' i "i i i
ni +' "2
n i ii +
2
nNote: f = f , f f , i=0, ..., 1
2
But =1, so ( ) = ( )
n for i=0, ..., 1
2n
Thus, f = f + f for i=0, ..., 12
and f = f + f
' i "i i
n n2 n2 2
n = f - f for i=0, ..., 1
2
since ( ) 1, so = -1
Operation Counts for FFT Algorithm
• Assume n = 2k
• # additionsAdd(n) = 2· Add(n/2) + n = n log n
• # multiplicationsMult(n) = 2· Mult(n/2) + n/2 = ½ n log n
• Total Time O(n log n)• Note in complex FFT,
# real ops is 5 n log n
Multipoint Polynomial Evaluation
• Input polynomial
• Problem evaluate f(x) at x0, x1, …, xn-1
• Easy Cases:FFT Case xi = i
= principal root of unity
1
0
( )n
ii
i
f x a x
Multipoint Polynomial Evaluation (cont’d)
2
Summary of FFT:
( ) '( ) "( )
where
'( ), "( ) both degree halved
needed to only evaluate at half as many points
method f x f y x f y
y x
f x f x
Other Polynomial Evaluation Problems Solved by FFT
Each costs O(n log n) time
• Evaluate at points Xi = bai + d for i=0,…, n-1(Chirp Transfom)– Reduced to FFT
• Single point evaluation of all derivatives of a polynomial– Solve by reduction to above Chirp Transform of case 2)
• Evaluate at points Xi = b(ai)2+ cai + d for i=0,…, n-1 – Solve by divide and conquer similar to FFT
Single Point Evaluation of all Derivatives of Polynomial
• Input
and point x0
• output
1
0
( )n
ii
i
f x a x
0
( )( ) for 0,..., 1
kk d f xf x x x k n
dx
Single Point Evaluation of all Derivatives of Polynomial (cont’d)• Taylor Series Representation of
Then
reduces to case of evaluation at points
• Solve this Chirp Transform problem by reduction FFT
1
00
( ) ( )n
ii
i
f x c x x
( )0( ) !k
kf x k c
for 0,..., 1iix ab i n
Advanced Material: Further Applications of FFT
1) Convolution: Products and Powers of Polynomials• Used for for Integer Multiplication
Algorithms • Also used for Filtering on infinite
input streams2) Division and Inverse of Polynomials3) Multipoint Evaluation and
Interpolation
Advanced Material: Products and Powers of Polynomials
• Input vectors a = (a0, a1, …, an-1)T
b = (b0, b1, …, bn-1)T
• Definition of Convolution c = a b
Where for i=0, …, 2n-1
define ak = bk = 0 if k< 0 or kn
n-1
i j i-jj=0
c = a b
Products and Powers of Polynomials (cont’d)
• Convolution Theorem
• Application to Polynomial Products:n-1
ii
i=0
n-1j
jj=0
2n-2 n-1i
i i j i-ji=0 j=0
p(x) = a x
q(x) = b x
p(x) q(x) = c x where c = a b
-12n 2n 2na b = FFT FFT (a) FFT (b)
Products of m Polynomials
• Generalized Convolution Theorem
k
n-1i
k k,ii=0
m(n-1)m mi
k i i k,jki=0k=1 k=1
j =1
for k=1, ..., m let P (x) = a x
P (x) = c x , where c = a
1 2 m
-1n m n m 1 n m 2 n m m
a a ... a =
FFT FFT (a ) FFT (a ) ... FFT (a )
Wrapped Convolutions
• a = (a0, a1, …, an-1)T , b = (b0, b1, …, bn-1)T
• Positive wrapped convolution is c = (c0, c1, …, cn-1)T
• Negative wrapped convolution is d = (d0, d1, …, dn-1)T
i n-1
i j i-j j n+i-jj=0 j=i+1
c = a b + a b
i n-1
i j i-j j n+i-jj=0 j=i+1
d = a b - a b
Application of Wrapped Convolution to Modular Polynomial Products
n-1i
ii=0
n-1j
jj=0
n
n-1i n n
ii=0
p(x) = a x
q(x) = b x
p(x) q(x) mod(x +1)
= d x since x = -1 mod(x +1)
Computing Positive Wrapped Convolution
• Let = principal nth root of unity• Assume n has multiplicative inverse,
Theorem
is the positive wrapped convolution of n-vectors a and b.
-1n n nc = FFT FFT (a) FFT (b)
Computing Negative Wrapped Convolution• Also
is the negatively wrapped convolution of n-vectors a and b
where
and 2 = = principal nth root of unity
-1n n n
ˆ ˆˆd = FFT FFT (a) FFT (b)
Tn-10 1 n-1
Tn-10 1 n-1
a = a , a , ..., a
b = b , b , ..., b
Integer Multiplication by Polynomial Product (solved via FFT)• Input n bit integers a,b
define polynomials degree k = n/L
k-1i L
i ii=0
k-1i L
i ii=0
L L
a(x) = a x , 0 a 2
b(x) = b x , 0 2
so a = a(2 ), b = b(2 )
b
Integer Multiplication by Polynomial Product (cont’d)
• Idea1) Compute c(x) = a(x)· b(x)
by convolution
2) Evaluate c(2L) = a· b
Integer Multiplication Algorithms using Reduction to Polynomial Product
• Pollard Mult Algorithm
• Karp Mult Algorithm
• Schönage-Strassen Mult Algorithm
2O(n(logn) )(loglogn) use L = logn
2O(n(logn) ) use L = n
O(n(logn)(loglogn)) use L = n
and wrapped convolution
Pollard Multiplication Algorithm
• n = kL, L = 1 + log k1) Choose primes P1, P2, P3 where
2) Compute C(x) by convolution over finite field Zpi for i =1,2,3 (requires k mults on 2L bit integers)
31 2 3
Li i i
P P P 4 k
and P = α 2 + 1, α = O(1)
Pollard Multiplication Algorithm (cont’d)
3) Evaluate C(2L)
• Time Bounds
2
2
T(n) = 3kT(2L) + O(k logk) O(L)
= 3kT (2(1 + logk)) + O(k(log k) )
O(n(log n) (log log n) ) for any > 0
recursive mults FFT
Korp Multiplication Algorithm
1) Compute C(x) modulo k by convolution2) Compute C(x) modulo (22L+1) by
convolution3) Compute C(x) coefficients from C(x)
mod k, C(x) mod (22L+1) by Chinese remaindering
s
s
2
(s-1)
2
n = 2 = kL
2 if s evenk =
2 else
Korp Multiplication Algorithm (cont’d)
4) Compute C(2L)
• Time
2
T(n) = 2kT(2L) + O(k logk)O(L)
= 2 nT (2 n ) + O(n log n)
O(n(log n) )
recursive mults FFT
Schönage-Strassen Multiplication Algorithm
(2’) Compute C(x) mod (xk+1) modulo (22L+1) by wrapped convolution requires only k recursive mults on 2L bit numbers
• Time
T(n) = kT(2L) + O(k logk)O(L)
= nT (2 n ) + O(n log n)
O(n log n)(log log n)
recursive mults FFT
Still Open Problem: How Fast Can You Multiply Integers?
• Can you mult n bit integers in O(n log n) time?
The Fast Fourier Transform and
Applications to Multiplication
Prepared by
John Reif, Ph.D.
Analysis of Algorithms