ADVANCED ALGORITHMS LECTURE 3: DIVIDE & CONQUER
�1
LECTURE 3
ANNOUNCEMENTS
▸ Homework 0 grading
▸ Homework 1 posted; due on Monday Sep 10, 11:59 PM
▸ Start early! (Post comments/questions on canvas…)
▸ Unless immediate, argue why algorithm is correct
▸ Three steps — algorithm description, proof of correctness, complexity analysis
�2
LECTURE 3
LAST CLASS
�3
▸ Basics of data structures
▸ What data? What operations supported? Time/space complexity
▸ Prefix trees (aka tries)
▸ Connectivity in graphs via depth-first-search (DFS)
▸ Proved using induction on the length of the path
LECTURE 3
TODAY’S CLASS
▸ Divide and conquer
▸ Analysis: recurrences and how to solve them
�4
LECTURE 3
DIVIDE AND CONQUER
�5
LECTURE 3
DIVIDE AND CONQUER
�6
PROBLEM INSTANCE
Divide step
Solve sub-problems
FULL SOLUTIONCombine/“conquer”
LECTURE 3
WHEN AND WHY?
�7
▸ Problem “cleanly” divides into not-too-many sub-problems
▸ Solutions easily (efficiently) combined
▸ Leads to simple, efficient solutions
▸ Analysis easier — correctness by induction, complexity by recurrences
Sometimes difficult to see how D&C can be used…
LECTURE 3
BASIC: SORTING
�8
Basic question: given unsorted array A[0], …, A[n-1], re-order elements in increasing order (produce new array)
▸ Divide step?
▸ Conquer step
35 4 7 98 5 6 10
Find smallestelement
IMAM Aln DT
t 717 Fasort recursively
LECTURE 3
MERGING SORTED LISTS
�9
CyBY's BYE Bz
qfaoi
j is the smallest ett inan
B E Bik's EB the entire arrayu
Shift the array 134 tothe left repeated
qeeach taking time
7 8 9 nk log k5 6 10
LECTURE 3
OVERALL ALGORITHM — MERGE SORT
�10
▸ Divide A[] into k pieces each of size [n/k]
▸ Sort each one recursively
▸ Use merge procedure to create new sorted list
What is the overall time complexity?
Tfn k TLEdnt
rk arrays each takesFCI to sor
01
Let Tcm denote runningtime of procedureon
an instance of size m
LECTURE 3
BASIC “RECURRENCE”
�11
T(n) = k ⋅ T(n /k) + kn + n
T(n) = 2T(n /2) + 3nSetting k=2
Minor: is n divisible by k?
tlnl 4 TT t5n
Goyal come up witha closed form solution
I expticitfnotnik
THE n
LECTURE 3
ASIDE: HOW TO SOLVE RECURRENCES?
�12
▸ No silver bullet!
▸ Guess, prove by induction — sometimes proof gives “feedback”
▸ “Plug and chug” [Lehman, Leighton]
▸ Recurrence treekeep pluggingback
into equation see a
pattern pronebyinduction
LECTURE 3
PLUG AND CHUG
�13
T(n) = 2T(n /2) + 3n
TH I
Ent 2T ET n 3 n t 213 ng t 2T F 3nt3n t 4517
3nt3nt 4 3 1 2T Ig nt3nt3y1851
r steps
r 3N IT Ir 7 3mlogin n 1
Base case Lr t rzlogen
LECTURE 3
RECURRENCE TREE
�14
T nl Lt3n1 n
eternity tinitiation we do
Tt HzTs T u totwafaexha.sn
IT 11 c 3 nn
terms I3N logan n th
LECTURE 3
EXAMPLE 2: MULTIPLYING INTEGERS
�15
Question: given two n digit numbers, how efficiently can we multiply them?
A Az anb
gRftifunenians HTIsee.ee eini I i t
LECTURE 3
DIVIDE AND CONQUER
�16BA
a.az anxbibnLbn500 5 102 X Y Z W
A X 0000,20 4 X ion Y
B Z lo w
A B x t x.I.LII.IoIi 7Oln 0cm
conquer stepis 0h Tfn _4TH101in
LECTURE 3
RUNNING TIME?
�17
TIM 4 TIE c n
11hTfn Acn 4T I Hai
Cnt 41C Z 4T fD Cn then 47T
Cnt 2cn t life It 4T 8i
I negynentientta cn 4ttfITH i He
LECTURE 3
CAN WE DO BETTER?
�18
Reason to hope: we only need to find three quantities:
XZ, (XW + YZ), YW
21
LECTURE 3
CAN WE DO BETTER?
�19
Reason to hope: we only need to find three quantities:
XZ, (XW + YZ), YW
Kartsuba’s observation:
XW + YZ = (X + Y )(W + Z) − XZ − YW
LECTURE 3
OVERALL RUNNING TIME
�20
LECTURE 3
EVEN BETTER?
�21
▸ Can we do linear time?
▸ HW1 — will see something approaching linear
LECTURE 3
COMPUTING THE MEDIAN
�22
Question: given an unsorted list A[0], A[1], …, A[n-1], find the (n/2)th smallest element
Divide and conquer?
LECTURE 3
KEY INSIGHT: “APPROXIMATE MEDIAN” HELPS!
�23
LECTURE 3
MEDIAN OF MEDIANS IS APPROXIMATE MEDIAN!
�24