0 ratings0% found this document useful (0 votes) 632 views6 pagesSubstitution Method
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content,
claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
43 The substitution method for solving recurrences 3
426
How quickly can you multiply a kn xn matrix by an. xk matrix, using Strassen’s,
algorithm as a subroutine? Answer the same question with the order of the input
matrices reversed.
42-7
Show how to multiply the complex numbers a + bi and ¢ + di using only three
multiplications of real numbers. The algorithm should take a, b, ¢, and d as input
and produce the real component ac — bd and the imaginary component ad + be
separately.
4.3 The substitution method for solving recurrences
of divide-
tart in this
Now that we have seen how recurrences characterize the running tim«
and-conquer algorithms, we will learn how to solve recurrences. W
section with the “substitution” method.
‘The substitution method for solving recurren
We substitute the guessed solution for the function when applying the inductive
hypothesis to smaller values; hence the name “substitution method.” This method
comprises two steps
is powerful, but we must be able fo guess the form of the answer in order to apply it.
We can use the substitution method to establish either upper or lower bounds on
a recurrence. As an example, let us determine an upper bound on the recurrence
Ten) =27((n/2)) 4m, a9
urenees (4.3) and (44). We guess that the Solution is
substitution method requires us to prove that T(n) =
“enlgn for an appropriate choice of the constant ¢ > 0. We start by assuming
that this bound holds forall positive m 3, the
recurrence does not depend directly on T(1). Thus, we can replace T(1) by T(2)
and T(3) as the base cases in the inductive proof, letting ng = 2. Note that we
‘make a distinction between the base case of the recurrence (n — 1) and the base
‘cases of the inductive proof ( = 2andm = 3). With T(1) = 1, we derive from
the recurrence that T(2) = 4 and T(3) = 5. Now we can complete the inductive
proof that T(n) < emlgn for some constant ¢ = 1 by choosing ¢ lange enough
so that (2) = e2lg2 and T(3) = e3lg3. As it tums out, any choice of ¢ = 2
suffices for the base cases of = 2and n = 3 to hold. For most of the recurrences
‘we shall examine, it is straightforward to extend boundary conditions to make the
inductive assumption work for small 1, and we shall not always explicitly work out
the details.
Making a good guess
s the correct solutions to recurreneé
Unfortunately, there is no general way to gui
Guessing a solution takes experience and, occasionally, creativity. Fortunately,
though, you can use some heuristics to help you become a good guesser. You
can also use recursion trees, which we shall see in Section 4.4, to generate good
guesse
Ifa recurrence is similar to one you have seen before, then guessing a similar
solution is reasonable. As an example, consider the recurrence
Tn) = 2T(\n/2| +17) +n,
which looks difficult because of the added “17” in the argument to T’on the right-
hand side. Intuitively, however, this additional term cannot substantially affect the 43° The substitution method for solving recurrences 8s
solution to the recurrence. When 1 is large, the difference between [71/2| and
n/2| 4 17 is not that large: both cut 1 nearly evenly in half. Consequently, we
make the guess that 7(n) = O(n Ign), which you can verify as comect by using
the substitution method (see Exercise 4.3-6).
Another way to make a good guess is to prove loose upper and lower bounds on
the recurrence and then reduce the range of uncertainty. For example, we might
start with a lower bound of 7(n) = Qin) for the recurrence (4.19), since we
have the term m in the recurrence, and we can prove an initial upper bound of
T(n) = O(n). ‘Then, we can gradually lower the upper bound and raise the
lower bound until we converge on the correct, asymptotically tight solution of
T(n) = O(nlgn),
Subtleties
Sometimes you might correctly gi
ess an asymptotic bound on the solution of a
recurrence, but somehow the math fails to work out in the induction. The problem
frequently tums out to be that the inductive assumption is not strong enough to
prove the detailed bound, If you revise the guess by subtracting a lower-order term
When you hit such a snag, the math often goes through,
Consider the recurrence
Tn) = T((n/2)) + T((n/2)) +1
“We guess that the solution is 7(m) = O(), and we try to show that T(n) = en for
aan appropriate choice of the constant ¢. Substituting our guess in the recurrence,
we obtain
Tin) = eln/2
= en+1
“which does not imply 7(n) < en for any choice of c. We might be tempted to try
a larger guess, say T(n) = O(n?). Although we can make this larger guess work,
our original guess of T(n) = O(n) is correct. In order to show that it is correct,
however, we must make a stronger inductive hypothesis.
Intuitively, our guess is nearly right’ we are off only by the constant 1, a
lower-order term. Nevertheless, mathematical induction does not work unl
prove the exact form of the inductive hypothesis. We overcome our difficulty
by subiracting a lowerorder term from our previous guess. Our mew guess is
‘T(al) = en —d, where d » Os a constant, We now have
Tin) = (€|m/2|—d) + [n/2| a) +1
en—2d 41
end,
fe [n/2] 41 86
Chaprer 4 Divide-and-Conguer
as long as d = 1. Asbefore, we must choose the constant ¢ large enough to handle
the boundary conditions.
You might find the idea of subtracting a lower-order term counterintuitive. Af-
ter all, if the math does not work out, we should increase our guess, right?
Not necessarily! When proving an upper bound by induction, it may actually be
‘more diflicult to prove that a weaker upper bound holds, because in order to prove
the weaker bound, we must use the same weaker bound inductively in the proof.
In our current example, when the recurrence has more than one recursive term, we
get to subtract out the lower-order term of the proposed bound once per recursive
term. In the above example, we subtracted out the constant d twice, once for the
T({n/2|) term and once for the T({n/2|) term. We ended up with the inequality
Tin) < en —2d + 1, and it was easy to find values of d to make en —2d + 1 be
less than or equal to en —d.
Avoiding pitfalls
It is easy (0 err in the use of asymptotic notation. For example, in the recur-
rence (4.19) we can falsely “prove” T(n) = O(n) by guessing T(n) <= en and
then arguing
Tn) = 2c\n/2)) +m
cntn
= 0m). <= wrong!!
since ¢ is a constant. The error is that we have not proved the exact form of the
inductive hypothesis, that is, that T(n) < cn. We therefore will explicitly prove
that T(n) = en when we want to show that T(n) = O(n)
Changing variables
Sometimes, a litte algebraic manipulation can make an unknown recurrence simi-
lar to one you have seen before. As an example, consider the recurrence
Te) = 27 (| va) | Ign
\hich looks difficult, We can simplify this recurrence, though, with a change of
variables. For convenience, we shall not worry about rounding off values, such
as it, t0 be integers. Renaming m = Ign yields
T(2") = 272") +m
‘We can now rename S(m
T (2) to produce the new recurrence
S(m) = 28(m/2) +m 43° The substitution method for solving recurrences 7
which
very much like recurrence (4.19). Indeed, this new recurrence has the
olution: $m) = O(mlgm). Changing back from S(m) to T(n), we obtain
T(n) =TQ")
(m) = O(migm) = Ollgnigign)
Exercises.
43-1
Show that the solution of T(n) = T(n —1) jn is O(n?).
32
Show that the
olution of T(n) = T([n/2}) +1 is Odgn).
43-3
We saw that the solution of T() = 27(|n/2|) 1-n is O(a Ign). Show that the so-
lution of this recurrence is also $2(71 Ig). Conclude that the solution is @(n Ig).
43-4
Show that by making a different inductive hypothesi
culty with the boundary condition T'(1) = | for recurrence (4.19) without adjusting
the boundary conditions for the inductive proof,
‘we can overcome the dif
43-5
Show that ©(7 Ig) is the solution to the “exact” recurrence (4.3) for merge sort.
43-6
Show that the solution to T(n) = 27(|n/2| | 17) + nis O(nign).
43-7
Using the master method in Section 4.5, you can show that the solution to the
recurrence T(n) = 47(n/3) +n is T(n) = @(n4), Show that a substitution
proof with the assumption T(m) = en fails. Then show how to subtract off a
lower-order term to make a substitution proof work.
43-8
Using the master method in Section 4.5, you can show that the solution to the
recurrence T(n) = 4T(n/2) | n? is T(n) = O(n). Show that a substitution
proof with the assumption T'(n) = en? fails. Then show how to subtract off a
lower-order term to make a substitution proof work. 88
Chaprer 4 Divide-and-Conguer
439
Solve the recurrence T(n) = 31) + logn by making a change of variables,
Your solution should be asymptotically tight. Do not worry about whether values
ave integral
method for
Although you can use the substitution method to provide a succinct proof that
a solution to a recurrence is correct, you might have trouble coming up with
good guess. Drawing out a recursion tree, as We did in our analysis of the merge
sort recurrence in Section 2.3.2, serves as a straightforward way to devise a good
ss. In a recursion free, each node represents the cost of a single subproblem
ve function invocations. We sum the costs within
to obtain a set of per-level costs, and then we sum all the
per-level costs to determine the total cost of all levels of the recursion.
A recursion tree is best used to generate a good guess, which you can then verify
by the substitution method. When using a recursion tree to generate a good guess,
you can ofien tolerate a small amount of “sloppiness,” since you will be verifying
your guess later on. If you are very careful when drawing out a recursion tree and
summing the costs, however, you can use a recursion tree as a direct proof of a
solution to a recurrence. In this section, we will use recursion trees to generate
good guesses, and in Section 4.6, we will use recursion trees directly to prove the
theorem that forms the basis of the master method.
For example, let us see how a recursion tree would provide a good guess for
the recurrence T(n) = 37(|n/4|) | @(n2). We start by focusing on finding an
upper bound for the solution. Because we know that floors and ceilings usually do
not matter when solving recurrences (here’s an example of sloppiness that we can
tolerate), We create a recursion tree for the recurrence T(n) = 31 (n/4) | en,
having written out the implied constant coefficient ¢ > 0.
Figure 4.5 shows how we derive the recursion tree for T'(n) = 37 (n/4) + cn?
For convenience, we assume that n is an exact power of 4 (another example of
tolerable sloppiness) so that all subproblem sizes are integers. Part (a) of the figure
shows T(mn), which we expand in part (b) into an equivalent tree representing the
recurrence. The cn? term at the root represents the cost at the top level of recursion,
and the three subtrees of the root represent the costs incurred by the subproblems
of size n/4, Part (c) shows this process carried one step further by expanding each
node with cost T(n/4) from part (b). The cost for each of the three children of the
root is c(n/4)*. We continue expanding each node in the tree by breaking it into
its constituent parts as determined by the recurrence.
