03 Analysis of Algorithms: Probabilistic Analysis

Analysis of Algorithms
Probabilistic Analysis
Andres Mendez-Vazquez
March 16, 2015
1 / 19

Outline
1 The Hiring Problem
2 The Hiring Algorithm
3 Paradigm
4 Indicator Random Variable
5 The Randomized Hiring
6 Methods to Enforce Randomization
Permute By Sorting
Permute in Place
2 / 19

A Random Process
First
In order to exemplify the usage of the probabilistic analysis, we will use the
Hiring Problem.
Why?
It is clear that hiring a person is a random process.
An Example
Many possible process involving a Expected count in the number of steps
of them.
3 / 19

The Hiring Problem
Suppose the following
You are using an employment agency to hire a new oce assistant.
You interview the candidate and must immediately decide whether or
not to hire that person.
Cost
Cost to interview is ci per candidate.
Cost to hire is ch per candidate.
Assume that ch ci .
Observation!
You must have somebody working all the time!
You will always hire the best candidate that you interview!
4 / 19

Hiring Algorithm
Hire-Assistant(n)
b e s t = 0 // c a n d i d a t e dummy
f o r i = 1 t o n
i n t e r v i e w i
i f c a n d i d a t e i i s b e t t e r t h a n c a n d i d a t e b e s t
t h e n b e s t = i
h i r e c a n d i d a t e i
COST Given n candidates and we hire m of them, O(nci + mch)
5 / 19

Paradigm
We have that
We need to nd the maximum or minimum in a sequence by
examining each element and maintaining a current WINNER
The variable m denotes how many times we change our notion of
which element is currently winning.
Worst Case Analysis
You hire all of n
O(nci + nch) = O(nch)
Why? Because every time we hire somebody, we re somebody.
6 / 19

We want to avoid the worst case
How?
Because many times we do not get the worst case
Actually
Many times we get the Average input
This makes
The probability analysis a useful tool to analyze average complexities for
many algorithms
7 / 19

Probabilistic Analysis
Uniform Distribution Assumption
1 Assign a rank to each candidate i, rank : U → {1, 2, ..., n}
2 The possible number of permutations of individuals by the rank is n!
3 Therefore, if we assume that all individuals have the same probability
to have any ranking - Unifrom Distribution Assumption.
4 The input in the hiring problem comes from a uniform distribution.
Essentials of Probability Analysis
You assume a distribution over permutation of elements.
The expectation is over this distribution.
This technique requires that we can make a reasonable
characterization of the input distribution.
8 / 19

Indicator Random Variable
Indicator Random Variable
We require to use the indicator random variable to facilitate the use of
probabilistic analysis:
I {A} =
0 if A does not ocurr
1 if A does ocurr
.
Using this we have
Lemma 5.1
Given a sample space S and an event A in the sample space S, let
XA = I {A}. Then E [XA] = Pr {A}.
9 / 19

Analyzing Hiring By Indicator Variable
Given X
Assume a X, the random variable of the number of times we hire a
person. Then E [X] = n
x=1 xPr {X = x}
We could analyze the hiring problem by using the indicator function:
Xi = I { candidate i is hired} =
1 if candidate i is hired
0 if candidate i is not hired
.
In addition
X = X1 + X2 + ... + Xn
An because the candidates come randomly (Uniform Assumption), we
have that E [Xi ] = Pr { cantidate i is hired} = 1
i
After all each time, we get a new i candidate is better than the
previous 1 to i − 1.
10 / 19

Analyzing Hiring By Indicator Variable
Finally...
We can calculate
E [X] = E
n
i=1
Xi =
n
i=1
E [X] =
n
i=1
1
i
= ln n + O (1) .
Final hiring cost is O (ch ln n).
11 / 19

Hiring Problem
What if
The employment agency sends us a list of all n candidates in advance.
On each day, we randomly choose a candidate from the list to
interview.
Instead of relaying on the candidate being presented to us in a random
order, we take control of the process and enforce a random order.
What makes an algorithm randomized?
An algorithm is randomized if its behavior is determined in part by
values produced by a random-number generator.
A random-number generator is implemented by a
pseudorandom-number generator, which is a deterministic method
returning numbers that look random and can pass certain statistical
tests.
12 / 19

Hiring Algorithm
Randomized-Hire-Assistant(n)
Randomly Permute t h e l i s t o f c a n d i d a t e s
b e s t = 0 // c a n d i d a t e dummy
f o r i = 1 t o n
i n t e r v i e w i
i f c a n d i d a t e i i s b e t t e r t h a n c a n d i d a t e b e s t
t h e n b e s t = i
h i r e c a n d i d a t e i
Lemma (5.3)
The expected hiring cost of the procedure Randomized-Hiring-Assistant is
O(ch ln n)
Proof.
Go to the Blackboard.
13 / 19

Methods to Enforce Randomization
Randomly Permute Arrays
1 A = 1, 2, 3, 4
2 P = 36, 3, 97, 19
Then, sort B = 2, 4, 1, 3
Permute-By-Sorting(A)
n = l e n g h t [ A ]
f o r i = 1 t o n
do P [ i ]=RANDOM( 1 , n ^3)
s o r t A , u s i n g P a s s o r t k e y s
r e t u r n A
15 / 19

Proving Correctness of Permute-By-Sorting
Lemma 5.4
Procedure Permute-By-Sorting produces a uniform random permutation of
the input, assuming that all priorities are distinct.
Proof: Go to the Blackboard
16 / 19

Permute in Place
Randomize-In-Place(A)
n = l e n g h t [ A ]
f o r i = 1 t o n
do swap A [ i ] − A [RANDOM( i , n ) ]
Lemma (5.5)
Procedure Randomize-In-Place computes a unifrom random permutation.
Proof.
Go to the Blackboard.
18 / 19

Exercises
5.1-1
5.1-2
5.2-1
5.2-3
5.2-5
5.3-1
5.3-3
5.3-4
19 / 19

03 Analysis of Algorithms: Probabilistic Analysis

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to 03 Analysis of Algorithms: Probabilistic Analysis (20)

More from Andres Mendez-Vazquez (20)

Recently uploaded (20)

03 Analysis of Algorithms: Probabilistic Analysis