7/27/2019 Randomized Algorithms-2013
1/30
Introduction to Randomized
Algorithms
Srikrishnan Divakaran
DA-IICT
7/27/2019 Randomized Algorithms-2013
2/30
Lecture Outline
Preliminaries and Motivation
Analysis of the Randomized hiring problem
Analysis of Randomized Quick Sort
References
7/27/2019 Randomized Algorithms-2013
3/30
Preliminaries and Motivation
7/27/2019 Randomized Algorithms-2013
4/30
Quick Sort
Select: pick an arbitrary element xin S to be the pivot.
Partition: rearrange elements so
that elements with value less than xgo to List L to the left of x andelements with value greater than xgo to the List R to the right of x.
Recursion: recursively sort the listsL and R.
7/27/2019 Randomized Algorithms-2013
5/30
Worst Case Partitioning of
Quick Sort
7/27/2019 Randomized Algorithms-2013
6/30
Best Case Partitioning of Quick
Sort
7/27/2019 Randomized Algorithms-2013
7/30
Average Case of Quick Sort
7/27/2019 Randomized Algorithms-2013
8/30
Randomized Quick Sort
Randomized-Partition(A , p, r)1. iRandom(p, r)
2. exchangeA[r]A[i]
3. returnPartition(A,p, r)
Randomized-Quicksort(A , p, r)1. ifp < r
2. thenqRandomized-Partition(A,p, r)
3. Randomized-Quicksort(A,p , q-1)
4. Randomized-Quicksort(A, q+1, r)
7/27/2019 Randomized Algorithms-2013
9/30
Randomized Quick Sort
Exchange A[r] with an element chosen at random from A[pr] inPartition.
The pivot element is equally likely to be any of input elements.
For any given input, the behavior of Randomized Quick Sort isdetermined not only by the input but also by the random choices ofthe pivot.
We add randomization to Quick Sort to obtain for any input theexpected performance of the algorithm to be good.
7/27/2019 Randomized Algorithms-2013
10/30
Deterministic Algorithms
Goal: Prove for all input instances the algorithm solves theproblem correctly and the number of steps is bounded by a
polynomial in the size of the input.
ALGORITHMINPUT OUTPUT
7/27/2019 Randomized Algorithms-2013
11/30
Randomized Algorithms
In addition to input, algorithm takes a source of random numbers
and makes random choices during execution;
Behavior can vary even on a fixed input;
ALGORITHMINPUT OUTPUT
RANDOM NUMBERS
7/27/2019 Randomized Algorithms-2013
12/30
Las Vegas Randomized
Algorithms
Goal: Prove that for all input instances the algorithm solves theproblem correctly and the expected number of steps is bounded by
a polynomial in the input size.
Note: The expectation is over the random choices made by the
algorithm.
ALGORITHMINPUT OUTPUT
RANDOM NUMBERS
7/27/2019 Randomized Algorithms-2013
13/30
Probabilistic Analysis of
Algorithms
Input is assumed to be from a probability distribution.
Goal:Show that for all inputs the algorithm works correctly and formost inputs the number of steps is bounded by a polynomial in thesize of the input.
ALGORITHMRANDOM
INPUT
OUTPUT
DISTRIBUTION
7/27/2019 Randomized Algorithms-2013
14/30
Monte Carlo Randomized
Algorithms
Goal:Prove that the algorithm with high probability solves the problem correctly;
for every input the expected number of steps is bounded by apolynomial in the input size.
Note: The expectation is over the random choices made by the
algorithm.
ALGORITHMINPUT OUTPUT
RANDOM NUMBERS
7/27/2019 Randomized Algorithms-2013
15/30
Monte Carlo versus Las Vegas
A Monte Carlo algorithm runs produces an answer that is correct
with non-zero probability, whereas a Las Vegas algorithm always
produces the correct answer.
The running time of both types of randomized algorithms is arandom variable whose expectation is bounded say by a polynomial
in terms of input size.
These expectations are only over the random choices made by the
algorithm independent of the input. Thus independent repetitions ofMonte Carlo algorithms drive down the failure probability
exponentially.
7/27/2019 Randomized Algorithms-2013
16/30
Motivation for Randomized
Algorithms
Simplicity;
Performance;
Reflects reality better (Online Algorithms);
For many hard problems helps obtain better complexity bounds
when compared to deterministic approaches;
7/27/2019 Randomized Algorithms-2013
17/30
Linearity of Expectation
If X1, X2, , Xn are random variables, then
n
i
XiEn
i
XiE
1
][
1
7/27/2019 Randomized Algorithms-2013
18/30
Hiring Problem
7/27/2019 Randomized Algorithms-2013
19/30
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
A Deterministic Algorithm for
hiring
7/27/2019 Randomized Algorithms-2013
20/30
Analysis of the Deterministic Hiring
Algorithm
Ci cost for interviewing a candidate
(typically low)
Ch
cost for hiring a candidate (typically
high)
M is the number of people hired
O(ci * h + ch * m) is the total costassociated with the deterministic hiring
algorithm.
7/27/2019 Randomized Algorithms-2013
21/30
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
A Randomized Algorithm for
hiring
7/27/2019 Randomized Algorithms-2013
22/30
Analysis of the Randomized
Hiring AlgorithmLet X denote the number of times we hire an office assistant.
E[X] = x = 1 to n x * Pr { X = x}
Let Xi
= I {candidate i is hired}
= 1 if the candidate i is hired and 0 otherwise
Notice X = X1 + X2+ + Xn
E[Xi] = 1/i
By linearity of expectation, we get
E[X] = E[X1 + X2+ + Xn ] = i = 1 to n E [Xi] = i = 1 to n 1/I = ln n.
7/27/2019 Randomized Algorithms-2013
23/30
Analysis of Randomized QuickSort
7/27/2019 Randomized Algorithms-2013
24/30
Notation
Rename the elements of A as z1, z2, . . . , zn, with zi being the ith
smallest element (Rank i).
Define the set Zij = {zi , zi+1, . . . , zj } be the set of elements between
zi and zj, inclusive.
106145389 72
z1z2 z9 z8 z5z3 z4 z6 z10 z7
7/27/2019 Randomized Algorithms-2013
25/30
Expected Number of Total
Comparisons in PARTITION
Let Xij = I {zi is compared to zj }
Let X be the total number of comparisons performed by the
algorithm. Then
][XE
by linearity
of expectation
1
1 1
n
i
n
ijijXE
1
1 1
n
i
n
ijijXE
1
1 1
}Pr{n
i
n
ij
ji ztocomparedisz
indicatorrandom variable
1
1 1
n
i
n
ij
ijXX
The expected number of comparisons performed by the algorithm is
7/27/2019 Randomized Algorithms-2013
26/30
Comparisons in PARTITION
Observation 1: Each pair of elements is compared at most once
during the entire execution of the algorithm
Elements are compared only to the pivot point!
Pivot point is excluded from future calls to PARTITION
Observation 2: Only the pivot is compared with elements in bothpartitions
z1z2 z9 z8 z5z3 z4 z6 z10 z7
106145389 72
Z1,6= {1, 2, 3, 4, 5, 6} Z8,9 = {8, 9, 10}{7}
pivot
Elements between different partitions are never compared
7/27/2019 Randomized Algorithms-2013
27/30
Comparisons in PARTITION
Case 1: pivot chosen such as: zi < x < zj
ziand zj will never be compared
Case 2: zi orzj is the pivot
zi and zjwill be compared
only if one of them is chosen as pivot before any other element
in range zito zj
106145389 72
z1z2 z9 z8 z5z3 z4 z6 z10 z7
Z1,6= {1, 2, 3, 4, 5, 6} Z8,9 = {8, 9, 10}{7}
Pr{ }?i jz is compared to z
7/27/2019 Randomized Algorithms-2013
28/30
Expected Number of Comparisons
in PARTITION
1
1 1
}Pr{][n
i
n
ij
ji ztocomparediszXE
Pr {Zi is compared with Zj}
= Pr{Zi or Zj is chosen as pivot before other elements in Zi,j} = 2 / (j-i+1)
1 1 1 1
1 1 1 1 1 1 1
2 2 2[ ] (lg )
1 1
n n n n i n n n
i j i i k i k i
E X O nj i k k
= O(nlgn)
7/27/2019 Randomized Algorithms-2013
29/30
1. Amihood Amir, Kargers Min-cut Algorithm, Bar-Ilan University, 2009.
2. George Bebis, Randomizing Quick Sort, Lecture Notes of CS 477/677:
Analysis of Algorithms, University of Nevada.
3. Avrim Blum and Amit Gupta, Lecture Notes on Randomized Algorithms, CMU,
2011.
4. Rada Mihalcea, Quick Sort, Lecture Notes of CSCE3110: Data Structures,
University of North Texas, http://www.cs.unt.edu/~rada/CSCE3110.
5. Rajeev Motwani and Prabhakar Raghavan, Randomized Algorithms,
Cambridge University Press, 1995.
6. Prabhakar Raghavan, AMS talk on Randomized Algorithms, Stanford
University.
References
1000 Wh t Wh t N t
http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE3110http://www.cs.unt.edu/~rada/CSCE31107/27/2019 Randomized Algorithms-2013
30/30
1000 Whats, What Nuts
&
Wall (Face Book) Nuts?DA-IICTs Wall Broken (i.e.,
No Boundaries)