Applied Symbolic Computation 1
Applied Symbolic Computation (CS 300)
Karatsuba’s Algorithm for Integer Multiplication
Jeremy R. Johnson
Applied Symbolic Computation 2
Introduction
• Objective: To derive a family of asymptotically fast integer multiplication algorithms using polynomial interpolation
– Karatsuba’s Algorithm– Polynomial algebra– Interpolation– Vandermonde Matrices– Toom-Cook algorithm– Polynomial multiplication using interpolation– Faster algorithms for integer multiplication
References: Lipson, Cormen et al.
Applied Symbolic Computation 3
Karatsuba’s Algorithm
• Using the classical pen and paper algorithm two n digit integers can be multiplied in O(n2) operations. Karatsuba came up with a faster algorithm.
• Let A and B be two integers with– A = A110k + A0, A0 < 10k
– B = B110k + B0, B0 < 10k
– C = A*B = (A110k + A0)(B110k + B0)
= A1B1102k + (A1B0 + A0 B1)10k + A0B0
Instead this can be computed with 3 multiplications
• T0 = A0B0
• T1 = (A1 + A0)(B1 + B0)
• T2 = A1B1
• C = T2102k + (T1 - T0 - T2)10k + T0
Applied Symbolic Computation 4
Complexity of Karatsuba’s Algorithm
• Let T(n) be the time to compute the product of two n-digit numbers using Karatsuba’s algorithm. Assume n = 2k. T(n) = (nlg(3)), lg(3) 1.58
• T(n) 3T(n/2) + cn
3(3T(n/4) + c(n/2)) + cn = 32T(n/22) + cn(3/2 + 1)
32(3T(n/23) + c(n/4)) + cn(3/2 + 1)
= 33T(n/23) + cn(32/22 + 3/2 + 1)
…
3iT(n/2i) + cn(3i-1/2i-1 + … + 3/2 + 1)
...
cn[((3/2)k - 1)/(3/2 -1)] --- Assuming T(1) c
2c(3k - 2k) 2c3lg(n) = 2cnlg(3)
Applied Symbolic Computation 5
Divide & Conquer Recurrence
Assume T(n) = aT(n/b) + (n)
• T(n) = (n) [a < b]
• T(n) = (nlog(n)) [a = b]
• T(n) = (nlogb
(a)) [a > b]
Applied Symbolic Computation 6
Polynomial Algebra
• Let F[x] denote the set of polynomials in the variable x whose coefficients are in the field F.
• F[x] becomes an algebra where +, * are defined by polynomial addition and multiplication.
babac
xc
xbxa
ik
mk
nkiij
jikik
knm
kk
jn
jj
im
ii
xBxAxC
xBxA
),min(
),0max(
0
00
,)()()(
)(,)(
Applied Symbolic Computation 7
Interpolation
• A polynomial of degree n is uniquely determined by its value at (n+1) distinct points.
Theorem: Let A(x) and B(x) be polynomials of degree m. If A(i) = B(i) for i = 0,…,m, then A(x) = B(x).
Proof.
Recall that a polynomial of degree m has m roots.
A(x) = Q(x)(x- ) + A(), if A() = 0, A(x) = Q(x)(x- ), and deg(Q) = m-1
Consider the polynomial C(x) = A(x) - B(x). Since C(i) = A(i) - B(i) = 0, for m+1 points, C(x) = 0, and A(x) must equal B(x).
Applied Symbolic Computation 8
Lagrange Interpolation Formula
• Find a polynomial of degree m given its value at (m+1) distinct points. Assume A(i) = yi
• Observe that
yi
m
iij ji
jx
xA
0 )(
)()(
yyki
m
iij ji
jk
kA
0 )(
)()(
Applied Symbolic Computation 9
Matrix Version of Polynomial Evaluation
• Let A(x) = a3x3 + a2x2 + a1x + a0
• Evaluation at the points , , , is obtained from the following matrix-vector product
aaaa
A
A
A
A
3
2
1
0
321
321
321
321
1
1
1
1
)(
)(
)(
)(
Applied Symbolic Computation 10
Matrix Interpretation of Interpolation
• Let A(x) = anxn + … + a1x +a0 be a polynomial of degree n. The problem of determining the (n+1) coefficients an,…,a1,a0 from the (n+1) values A(0),…,A(n) is equivalent to solving the linear system
a
aa
n
nn
n
n
nA
A
A
3
1
0
1
1
1
1
0
1
0
1
0
...
...1
............
...1
...1
)(
...
)(
)(
Applied Symbolic Computation 11
Vandermonde Matrix
n
nn
n
n
nV
...1
............
...1
...1
),...,(
1
1
1
1
0
1
0
0
)()det( iji
jV
V(0,…, n) is non-singular when 0,…, n are distinct.
Applied Symbolic Computation 12
Polynomial Multiplication using Interpolation
• Compute C(x) = A(x)B(x), where degree(A(x)) = m, and degree(B(x)) = n. Degree(C(x)) = m+n, and C(x) is uniquely determined by its value at m+n+1 distinct points.
• [Evaluation] Compute A(i) and B(i) for distinct i, i=0,…,m+n.
• [Pointwise Product] Compute C(i) = A(i)*B(i) for i=0,…,m+n.
• [Interpolation] Compute the coefficients of C(x) = cnxm+n + … + c1x +c0 from the points C(i) = A(i)*B(i) for i=0,…,m+n.
Applied Symbolic Computation 13
Interpolation and Karatsuba’s Algorithm
• Let A(x) = A1x + A0, B(x) = B1x + B0, C(x) = A(x)B(x) = C2x2 + C1x + C0
• Then A(10k) = A, B(10k) = B, and C = C(10k) = A(10k)B(10k) = AB
• Use interpolation based algorithm:
– Evaluate A(), A(), A() and B(), B(), B() for = 0, = 1, and =.– Compute C() = A()B(), C() = A() B(), C() = A()B()
– Interpolate the coefficients C2, C1, and C0
– Compute C = C2102k + C110k + C0
Applied Symbolic Computation 14
Matrix Equation for Karatsuba’s Algorithm
• Modified Vandermonde Matrix
• Interpolation
ccc
C
C
C
2
1
0
100
111
001
)(
)1(
)0(
BABBAA
BA
ccc
11
1010
00
2
1
0
100
111
001
Applied Symbolic Computation 15
Integer Multiplication Splitting the Inputs into 3 Parts
• Instead of breaking up the inputs into 2 equal parts as is done for Karatsuba’s algorithm, we can split the inputs into three equal parts.
• This algorithm is based on an interpolation based polynomial product of two quadratic polynomials.
• Let A(x) = A2x2 + A1x + A0, B(x) = B2x2 + B1x + B, C(x) = A(x)B(x) = C4x4 + C3x3 + C2x2 + C1x + C0
• Thus there are 5 products. The divide and conquer part still takes time = O(n). Therefore the total computing time T(n) = 5T(n/3) + O(n) = (nlog
3(5)), log3(5) 1.46
Applied Symbolic Computation 16
Asymptotically Fast Integer Multiplication
• We can obtain a sequence of asymptotically faster multiplication algorithms by splitting the inputs into more and more pieces.
• If we split A and B into k equal parts, then the corresponding multiplication algorithm is obtained from an interpolation based polynomial multiplication algorithm of two degree (k-1) polynomials.
• Since the product polynomial is of degree 2(k-1), we need to evaluate at 2k-1 points. Thus there are (2k-1) products. The divide and conquer part still takes time = O(n). Therefore the total computing time T(n) = (2k-1)T(n/k) + O(n) = (nlog
k(2k-1)).
Applied Symbolic Computation 17
Asymptotically Fast Integer Multiplication
• Using the previous construction we can find an algorithm to multiply two n digit integers in time (n1+ ) for any positive .
– logk(2k-1) = logk(k(2-1/k)) = 1 + logk(2-1/k)
– logk(2-1/k) logk(2) = ln(2)/ln(k) 0.
• Can we do better?
• The answer is yes. There is a faster algorithm, with computing time (nlog(n)loglog(n)), based on the fast Fourier transform (FFT). This algorithm is also based on interpolation and the polynomial version of the CRT.