![Introduction
Now suppose you want to write a function for comparing two strings that tests if they are
equal, ignoring case:
// equal-ignore-case -- returns true if s and t are equal, ignoring case
function equal-ignore-case(string s, string t): boolean
These ideas might come to mind:
1. Every character in strings s and t will have to be looked at
2. A single loop iterating through both might accomplish this
3. But such a loop should be careful that the strings are of equal length ïŹrst
4. If the strings aren't the same length, then they cannot be equal because the considera-
tion of ignoring case doesn't aïŹect how long the string is
5. A tolower subroutine for characters can be used again, and only the lowercase versions
will be compared
These ideas come from familiarity both with strings and with the looping and conditional
constructs in your language. The function you thought of may have looked something like this:
// equal-ignore-case -- returns true if s or t are equal, ignoring case
function equal-ignore-case(string s[1..n], string t[1..m]): boolean
if n != m:
return false if they aren't the same length, they aren't equal
fi
for i := 1 to n:
if tolower(s[i]) != tolower(t[i]):
return false
fi
repeat
return true
end
This code sample is also available in Adaa
a http://en.wikibooks.org/wiki/Ada_Programming%2FAlgorithms%23Equal_Ignore_Case
Or, if you thought of the problem in terms of functional decomposition instead of iterations,
you might have thought of a function more like this:
// equal-ignore-case -- returns true if s or t are equal, ignoring case
function equal-ignore-case(string s, string t): boolean
return tolower(s).equals(tolower(t))
end
Alternatively, you may feel neither of these solutions is eïŹcient enough, and you would
prefer an algorithm that only ever made one pass of s or t. The above two implementations
each require two-passes: the ïŹrst version computes the lengths and then compares each
character, while the second version computes the lowercase versions of the string and then
compares the results to each other. (Note that for a pair of strings, it is also possible to
have the length precomputed to avoid the second pass, but that can have its own drawbacks
6](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-9-320.jpg)
![Algorithm Analysis: Solving Recurrence Equations
Note that a function f is in o(g(n)) when for any coeïŹcient of g, g eventually gets larger
than f, while for O(g(n)), there only has to exist a single coeïŹcient for which g eventually
gets at least as big as f.
[TODO: deïŹne what T(n,m) = O(f(n,m)) means. That is, when the running time of an
algorithm has two dependent variables. Ex, a graph with n nodes and m edges. It's important
to get the quantiïŹers correct!]
2.2 Algorithm Analysis: Solving Recurrence Equations
Merge sort of n elements: T(n) = 2âT(n/2)+c(n) This describes one iteration of the merge
sort: the problem space n is reduced to two halves (2 â T(n/2)), and then merged back
together at the end of all the recursive calls (c(n)). This notation system is the bread and
butter of algorithm analysis, so get used to it.
There are some theorems you can use to estimate the big Oh time for a function if its
recurrence equation ïŹts a certain pattern.
[TODO: write this section]
2.2.1 Substitution method
Formulate a guess about the big Oh time of your equation. Then use proof by induction to
prove the guess is correct.
[TODO: write this section]
2.2.2 Summations
[TODO: show the closed forms of commonly needed summations and prove them]
2.2.3 Draw the Tree and Table
This is really just a way of getting an intelligent guess. You still have to go back to the
substitution method in order to prove the big Oh time.
[TODO: write this section]
2.2.4 The Master Theorem
Consider a recurrence equation that ïŹts the following formula:
T(n) = aT
n
b
+O(nk
)
15](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-18-320.jpg)
![Amortized Analysis
2.3 Amortized Analysis
[Start with an adjacency list representation of a graph and show two nested for loops: one
for each node n, and nested inside that one loop for each edge e. If there are n nodes and m
edges, this could lead you to say the loop takes O(nm) time. However, only once could the
innerloop take that long, and a tighter bound is O(n+m).]
17](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-20-320.jpg)
![3 Divide and Conquer
The ïŹrst major algorithmic technique we cover is divide and conquer. Part of the trick
of making a good divide and conquer algorithm is determining how a given problem could
be separated into two or more similar, but smaller, subproblems. More generally, when we
are creating a divide and conquer algorithm we will take the following steps:
Divide and Conquer Methodology
1. Given a problem, identify a small number of signiïŹcantly smaller subproblems of
the same type
2. Solve each subproblem recursively (the smallest possible size of a subproblem is
a base-case)
3. Combine these solutions into a solution for the main problem
The ïŹrst algorithm we'll present using this methodology is the merge sort.
3.1 Merge Sort
The problem that merge sort solves is general sorting: given an unordered array of elements
that have a total ordering, create an array that has the same elements sorted. More precisely,
for an array a with indexes 1 through n, if the condition
for all i, j such that 1 †i < j †n then a[i] †a[j]
holds, then a is said to be sorted. Here is the interface:
// sort -- returns a sorted copy of array a
function sort(array a): array
Following the divide and conquer methodology, how can a be broken up into smaller
subproblems? Because a is an array of n elements, we might want to start by breaking the
array into two arrays of size n/2 elements. These smaller arrays will also be unsorted and it
is meaningful to sort these smaller problems; thus we can consider these smaller arrays
"similar". Ignoring the base case for a moment, this reduces the problem into a diïŹerent
one: Given two sorted arrays, how can they be combined to form a single sorted array that
contains all the elements of both given arrays:
19](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-22-320.jpg)
![Divide and Conquer
// merge -- given a and b (assumed to be sorted) returns a merged array that
// preserves order
function merge(array a, array b): array
So far, following the methodology has led us to this point, but what about the base case?
The base case is the part of the algorithm concerned with what happens when the problem
cannot be broken into smaller subproblems. Here, the base case is when the array only has
one element. The following is a sorting algorithm that faithfully sorts arrays of only zero or
one elements:
// base-sort -- given an array of one element (or empty), return a copy of the
// array sorted
function base-sort(array a[1..n]): array
assert (n <= 1)
return a.copy()
end
Putting this together, here is what the methodology has told us to write so far:
// sort -- returns a sorted copy of array a
function sort(array a[1..n]): array
if n <= 1: return a.copy()
else:
let sub_size := n / 2
let first_half := sort(a[1,..,sub_size])
let second_half := sort(a[sub_size + 1,..,n])
return merge(first_half, second_half)
fi
end
And, other than the unimplemented merge subroutine, this sorting algorithm is done!
Before we cover how this algorithm works, here is how merge can be written:
// merge -- given a and b (assumed to be sorted) returns a merged array that
// preserves order
function merge(array a[1..n], array b[1..m]): array
let result := new array[n + m]
let i, j := 1
for k := 1 to n + m:
if i >= n: result[k] := b[j]; j += 1
else-if j >= m: result[k] := a[i]; i += 1
else:
if a[i] < b[j]:
result[k] := a[i]; i += 1
else:
result[k] := b[j]; j += 1
fi
fi
repeat
end
20](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-23-320.jpg)
![Merge Sort
[TODO: how it works; including correctness proof] This algorithm uses the fact that, given
two sorted arrays, the smallest element is always in one of two places. It's either at the head
of the ïŹrst array, or the head of the second.
3.1.1 Analysis
Let T(n) be the number of steps the algorithm takes to run on input of size n.
Merging takes linear time and we recurse each time on two sub-problems of half the original
size, so
T(n) = 2·T
n
2
+O(n).
By the master theorem, we see that this recurrence has a "steady state" tree. Thus, the
runtime is:
T(n) = O(n·logn).
3.1.2 Iterative Version
This merge sort algorithm can be turned into an iterative algorithm by iteratively merging
each subsequent pair, then each group of four, et cetera. Due to a lack of function overhead,
iterative algorithms tend to be faster in practice. However, because the recursive version's
call tree is logarithmically deep, it does not require much run-time stack space: Even
sorting 4 gigs of items would only require 32 call entries on the stack, a very modest amount
considering if even each call required 256 bytes on the stack, it would only require 8 kilobytes.
The iterative version of mergesort is a minor modiïŹcation to the recursive version - in
fact we can reuse the earlier merging function. The algorithm works by merging small,
sorted subsections of the original array to create larger subsections of the array which
are sorted. To accomplish this, we iterate through the array with successively larger "strides".
// sort -- returns a sorted copy of array a
function sort_iterative(array a[1..n]): array
let result := a.copy()
for power := 0 to log2(n-1)
let unit := 2Ëpower
for i := 1 to n by unit*2
let a1[1..unit] := result[i..i+unit-1]
let a2[1..unit] := result[i+unit..min(i+unit*2-1, n)]
result[i..i+unit*2-1] := merge(a1,a2)
repeat
repeat
return result
end
This works because each sublist of length 1 in the array is, by deïŹnition, sorted. Each
iteration through the array (using counting variable i) doubles the size of sorted sublists by
21](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-24-320.jpg)
![Divide and Conquer
merging adjacent sublists into sorted larger versions. The current size of sorted sublists in
the algorithm is represented by the unit variable.
3.2 Binary Search
Once an array is sorted, we can quickly locate items in the array by doing a binary search.
Binary search is diïŹerent from other divide and conquer algorithms in that it is mostly divide
based (nothing needs to be conquered). The concept behind binary search will be useful
for understanding the partition and quicksort algorithms, presented in the randomization
chapter.
Finding an item in an already sorted array is similar to ïŹnding a name in a phonebook: you
can start by ïŹipping the book open toward the middle. If the name you're looking for is
on that page, you stop. If you went too far, you can start the process again with the ïŹrst
half of the book. If the name you're searching for appears later than the page, you start
from the second half of the book instead. You repeat this process, narrowing down your
search space by half each time, until you ïŹnd what you were looking for (or, alternatively,
ïŹnd where what you were looking for would have been if it were present).
The following algorithm states this procedure precisely:
// binary-search -- returns the index of value in the given array, or
// -1 if value cannot be found. Assumes array is sorted in ascending order
function binary-search(value, array A[1..n]): integer
return search-inner(value, A, 1, n + 1)
end
// search-inner -- search subparts of the array; end is one past the
// last element
function search-inner(value, array A, start, end): integer
if start == end:
return -1 // not found
fi
let length := end - start
if length == 1:
if value == A[start]:
return start
else:
return -1
fi
fi
let mid := start + (length / 2)
if value == A[mid]:
return mid
else-if value > A[mid]:
return search-inner(value, A, mid + 1, end)
else:
return search-inner(value, A, start, mid)
fi
end
Note that all recursive calls made are tail-calls, and thus the algorithm is iterative. We
can explicitly remove the tail-calls if our programming language does not do that for us
already by turning the argument values passed to the recursive call into assignments, and
22](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-25-320.jpg)
![Binary Search
then looping to the top of the function body again:
// binary-search -- returns the index of value in the given array, or
// -1 if value cannot be found. Assumes array is sorted in ascending order
function binary-search(value, array A[1,..n]): integer
let start := 1
let end := n + 1
loop:
if start == end: return -1 fi // not found
let length := end - start
if length == 1:
if value == A[start]: return start
else: return -1 fi
fi
let mid := start + (length / 2)
if value == A[mid]:
return mid
else-if value > A[mid]:
start := mid + 1
else:
end := mid
fi
repeat
end
Even though we have an iterative algorithm, it's easier to reason about the recursive version.
If the number of steps the algorithm takes is T(n), then we have the following recurrence
that deïŹnes T(n):
T(n) = 1·T
n
2
+O(1).
The size of each recursive call made is on half of the input size (n), and there is a constant
amount of time spent outside of the recursion (i.e., computing length and mid will take the
same amount of time, regardless of how many elements are in the array). By the master
theorem, this recurrence has values a = 1,b = 2,k = 0, which is a "steady state" tree, and
thus we use the steady state case that tells us that
T(n) = Î(nk
·logn) = Î(logn).
Thus, this algorithm takes logarithmic time. Typically, even when n is large, it is safe to let
the stack grow by logn activation records through recursive calls.
diïŹculty in initially correct binary search implementations
The article on wikipedia on Binary Search also mentions the diïŹculty in writing a correct
binary search algorithm: for instance, the java Arrays.binarySearch(..) overloaded function
implementation does an interative binary search which didn't work when large integers
overïŹowed a simple expression of mid calculation mid = ( end + start) / 2 i.e. end + start
> max_positive_integer . Hence the above algorithm is more correct in using a length =
23](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-26-320.jpg)
![Integer Multiplication
easier in binary, because we only multiply by 1 or 0:
x6 x5 x4 x3 x2 x1 x0
Ă y6 y5 y4 y3 y2 y1 y0
-----------------------
x6 x5 x4 x3 x2 x1 x0 (when y0 is 1; 0 otherwise)
x6 x5 x4 x3 x2 x1 x0 0 (when y1 is 1; 0 otherwise)
x6 x5 x4 x3 x2 x1 x0 0 0 (when y2 is 1; 0 otherwise)
x6 x5 x4 x3 x2 x1 x0 0 0 0 (when y3 is 1; 0 otherwise)
... et cetera
As an algorithm, here's what multiplication would look like:
// multiply -- return the product of two binary integers, both of length n
function multiply(bitarray x[1,..n], bitarray y[1,..n]): bitarray
bitarray p = 0
for i:=1 to n:
if y[i] == 1:
p := add(p, x)
fi
x := pad(x, 0) // add another zero to the end of x
repeat
return p
end
The subroutine add adds two binary integers and returns the result, and the subroutine
pad adds an extra digit to the end of the number (padding on a zero is the same thing as
shifting the number to the left; which is the same as multiplying it by two). Here, we loop n
times, and in the worst-case, we make n calls to add. The numbers given to add will at
most be of length 2n. Further, we can expect that the add subroutine can be done in linear
time. Thus, if n calls to a O(n) subroutine are made, then the algorithm takes O(n2) time.
3.3.2 Divide and Conquer Multiplication
As you may have ïŹgured, this isn't the end of the story. We've presented the "obvious"
algorithm for multiplication; so let's see if a divide and conquer strategy can give us something
better. One route we might want to try is breaking the integers up into two parts. For
example, the integer x could be divided into two parts, xh and xl, for the high-order and
low-order halves of x. For example, if x has n bits, we have
x = xh ·2n/2
+xl
We could do the same for y:
y = yh ·2n/2
+yl
But from this division into smaller parts, it's not clear how we can multiply these parts such
that we can combine the results for the solution to the main problem. First, let's write out
xĂy would be in such a system:
25](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-28-320.jpg)
![Divide and Conquer
xĂy = xh Ăyh ·(2n/2
)2
+(xh Ăyl +xl Ăyh)·(2n/2
)+xl Ăyl
This comes from simply multiplying the new hi/lo representations of x and y together. The
multiplication of the smaller pieces are marked by the "Ă" symbol. Note that the multiplies
by 2n/2 and (2n/2)2 = 2n does not require a real multiplication: we can just pad on the right
number of zeros instead. This suggests the following divide and conquer algorithm:
// multiply -- return the product of two binary integers, both of length n
function multiply(bitarray x[1,..n], bitarray y[1,..n]): bitarray
if n == 1: return x[1] * y[1] fi // multiply single digits: O(1)
let xh := x[n/2 + 1, .., n] // array slicing, O(n)
let xl := x[0, .., n / 2] // array slicing, O(n)
let yh := y[n/2 + 1, .., n] // array slicing, O(n)
let yl := y[0, .., n / 2] // array slicing, O(n)
let a := multiply(xh, yh) // recursive call; T(n/2)
let b := multiply(xh, yl) // recursive call; T(n/2)
let c := multiply(xl, yh) // recursive call; T(n/2)
let d := multiply(xl, yl) // recursive call; T(n/2)
b := add(b, c) // regular addition; O(n)
a := shift(a, n) // pad on zeros; O(n)
b := shift(b, n/2) // pad on zeros; O(n)
return add(a, b, d) // regular addition; O(n)
end
We can use the master theorem to analyze the running time of this algorithm. Assuming
that the algorithm's running time is T(n), the comments show how much time each step
takes. Because there are four recursive calls, each with an input of size n/2, we have:
T(n) = 4T(n/2)+O(n)
Here, a = 4,b = 2,k = 1, and given that 4 > 21 we are in the "bottom heavy" case and thus
plugging in these values into the bottom heavy case of the master theorem gives us:
T(n) = O(nlog2 4
) = O(n2
).
Thus, after all of that hard work, we're still no better oïŹ than the grade school algorithm!
Luckily, numbers and polynomials are a data set we know additional information about. In
fact, we can reduce the running time by doing some mathematical tricks.
First, let's replace the 2n/2 with a variable, z:
xĂy = xh âyhz2
+(xh âyl +xl âyh)z +xl âyl
This appears to be a quadratic formula, and we know that you only need three co-eïŹcients
or points on a graph in order to uniquely describe a quadratic formula. However, in our
above algorithm we've been using four multiplications total. Let's try recasting x and y as
linear functions:
26](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-29-320.jpg)
![Base Conversion
Px(z) = xh ·z +xl
Py(z) = yh ·z +yl
Now, for xĂy we just need to compute (Px ·Py)(2n/2). We'll evaluate Px(z) and Py(z) at
three points. Three convenient points to evaluate the function will be at (Px ·Py)(1),(Px ·
Py)(0),(Px ·Py)(â1):
[TODO: show how to make the two-parts breaking more eïŹcient; then mention that the
best multiplication uses the FFT, but don't actually cover that topic (which is saved for the
advanced book)]
3.4 Base Conversion
[TODO: Convert numbers from decimal to binary quickly using DnC.]
Along with the binary, the science of computers employs bases 8 and 16 for it's very easy
to convert between the three while using bases 8 and 16 shortens considerably number
representations.
To represent 8 ïŹrst digits in the binary system we need 3 bits. Thus we have, 0=000, 1=001,
2=010, 3=011, 4=100, 5=101, 6=110, 7=111. Assume M=(2065)8. In order to obtain its
binary representation, replace each of the four digits with the corresponding triple of bits:
010 000 110 101. After removing the leading zeros, binary representation is immediate:
M=(10000110101)2. (For the hexadecimal system conversion is quite similar, except that
now one should use 4-bit representation of numbers below 16.) This fact follows from the
general conversion algorithm and the observation that 8=23 (and, of course, 16=24). Thus it
appears that the shortest way to convert numbers into the binary system is to ïŹrst convert
them into either octal or hexadecimal representation. Now let see how to implement the
general algorithm programmatically.
For the sake of reference, representation of a number in a system with base (radix) N may
only consist of digits that are less than N.
More accurately, if
(1)M = akNk
+akâ1Nkâ1
+...+a1N1
+a0
with 0 <= ai < N we have a representation of M in base N system and write
M = (akakâ1...a0)N
If we rewrite (1) as
(2)M = a0 +N â(a1 +N â(a2 +N â...))
27](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-30-320.jpg)
![Divide and Conquer
[TODO: explain the algorithm, and show the nË2 algorithm]
[TODO: write the algorithm, include intuition, proof of correctness, and runtime analysis]
Use this link for the original document.
http://www.cs.mcgill.ca/~cs251/ClosestPair/ClosestPairDQ.html
3.6 Closest Pair: A Divide-and-Conquer Approach
3.6.1 Introduction
The brute force approach to the closest pair problem (i.e. checking every possible pair of
points) takes quadratic time. We would now like to introduce a faster divide-and-conquer
algorithm for solving the closest pair problem. Given a set of points in the plane S, our
approach will be to split the set into two roughly equal halves (S1 and S2) for which we
already have the solutions, and then to merge the halves in linear time to yield an O(nlogn)
algorithm. However, the actual solution is far from obvious. It is possible that the desired
pair might have one point in S1 and one in S2, does this not force us once again to check
all possible pairs of points? The divide-and-conquer approach presented here generalizes
directly from the one dimensional algorithm we presented in the previous section.
3.6.2 Closest Pair in the Plane
Alright, we'll generalize our 1-D algorithm as directly as possible (see ïŹgure 3.2). Given a
set of points S in the plane, we partition it into two subsets S1 and S2 by a vertical line l
such that the points in S1 are to the left of l and those in S2 are to the right of l.
We now recursively solve the problem on these two sets obtaining minimum distances of d1
(for S1), and d2 (for S2). We let d be the minimum of these.
Now, identical to the 1-D case, if the closes pair of the whole set consists of one point from
each subset, then these two points must be within d of l. This area is represented as the two
strips P1 and P2 on either side of l
Up to now, we are completely in step with the 1-D case. At this point, however, the extra
dimension causes some problems. We wish to determine if some point in say P1 is less
than d away from another point in P2. However, in the plane, we don't have the luxury
that we had on the line when we observed that only one point in each set can be within
d of the median. In fact, in two dimensions, all of the points could be in the strip! This
is disastrous, because we would have to compare n2 pairs of points to merge the set, and
hence our divide-and-conquer algorithm wouldn't save us anything in terms of eïŹciency.
Thankfully, we can make another life saving observation at this point. For any particular
point p in one strip, only points that meet the following constraints in the other strip need
to be checked:
âą those points within d of p in the direction of the other strip
âą those within d of p in the positive and negative y directions
30](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-33-320.jpg)
![Divide and Conquer
T(n)O(nlogn)
Hence the merging of the sub-solutions is dominated by the sorting at step 4, and hence
takes O(nlogn) time.
This must be repeated once for each level of recursion in the divide-and-conquer algorithm,
hence the whole of algorithm ClosestPair takes O(logn*nlogn) = O(nlog2n) time.
3.6.4 Improving the Algorithm
We can improve on this algorithm slightly by reducing the time it takes to achieve the
y-coordinate sorting in Step 4. This is done by asking that the recursive solution computed
in Step 1 returns the points in sorted order by their y coordinates. This will yield two sorted
lists of points which need only be merged (a linear time operation) in Step 4 in order to yield
a complete sorted list. Hence the revised algorithm involves making the following changes:
Step 1: Divide the set into..., and recursively compute the distance in each part, returning
the points in each set in sorted order by y-coordinate. Step 4: Merge the two sorted lists
into one sorted list in O(n) time. Hence the merging process is now dominated by the linear
time steps thereby yielding an O(nlogn) algorithm for ïŹnding the closest pair of a set of
points in the plane.
3.7 Towers Of Hanoi Problem
[TODO: Write about the towers of hanoi algorithm and a program for it]
There are n distinct sized discs and three pegs such that discs are placed at the left peg in
the order of their sizes. The smallest one is at the top while the largest one is at the bottom.
This game is to move all the discs from the left peg
3.7.1 Rules
1) Only one disc can be moved in each step.
2) Only the disc at the top can be moved.
3) Any disc can only be placed on the top of a larger disc.
3.7.2 Solution
Intuitive Idea
In order to move the largest disc from the left peg to the middle peg, the smallest discs
must be moved to the right peg ïŹrst. After the largest one is moved. The smaller discs are
then moved from the right peg to the middle peg.
32](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-35-320.jpg)
![4 Randomization
As deterministic algorithms are driven to their limits when one tries to solve hard problems
with them, a useful technique to speed up the computation is randomization. In randomized
algorithms, the algorithm has access to a random source, which can be imagined as tossing
coins during the computation. Depending on the outcome of the toss, the algorithm may
split up its computation path.
There are two main types of randomized algorithms: Las Vegas algorithms and Monte-Carlo
algorithms. In Las Vegas algorithms, the algorithm may use the randomness to speed up
the computation, but the algorithm must always return the correct answer to the input.
Monte-Carlo algorithms do not have the former restriction, that is, they are allowed to give
wrong return values. However, returning a wrong return value must have a small probability,
otherwise that Monte-Carlo algorithm would not be of any use.
Many approximation algorithms use randomization.
4.1 Ordered Statistics
Before covering randomized techniques, we'll start with a deterministic problem that leads
to a problem that utilizes randomization. Suppose you have an unsorted array of values and
you want to ïŹnd
âą the maximum value,
âą the minimum value, and
âą the median value.
In the immortal words of one of our former computer science professors, "How can you do?"
4.1.1 ïŹnd-max
First, it's relatively straightforward to ïŹnd the largest element:
// find-max -- returns the maximum element
function find-max(array vals[1..n]): element
let result := vals[1]
for i from 2 to n:
result := max(result, vals[i])
repeat
return result
end
35](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-38-320.jpg)
![Randomization
An initial assignment of ââ to result would work as well, but this is a useless call to the
max function since the ïŹrst element compared gets set to result. By initializing result
as such the function only requires n-1 comparisons. (Moreover, in languages capable of
metaprogramming, the data type may not be strictly numerical and there might be no good
way of assigning ââ; using vals[1] is type-safe.)
A similar routine to ïŹnd the minimum element can be done by calling the min function
instead of the max function.
4.1.2 ïŹnd-min-max
But now suppose you want to ïŹnd the min and the max at the same time; here's one solution:
// find-min-max -- returns the minimum and maximum element of the given array
function find-min-max(array vals): pair
return pair {find-min(vals), find-max(vals)}
end
Because ïŹnd-max and ïŹnd-min both make n-1 calls to the max or min functions (when
vals has n elements), the total number of comparisons made in ïŹnd-min-max is 2nâ2.
However, some redundant comparisons are being made. These redundancies can be removed
by "weaving" together the min and max functions:
// find-min-max -- returns the minimum and maximum element of the given array
function find-min-max(array vals[1..n]): pair
let min := â
let max := ââ
if n is odd:
min := max := vals[1]
vals := vals[2,..,n] // we can now assume n is even
n := n - 1
fi
for i:=1 to n by 2: // consider pairs of values in vals
if vals[i] < vals[i + n by 2]:
let a := vals[i]
let b := vals[i + n by 2]
else:
let a := vals[i + n by 2]
let b := vals[i] // invariant: a <= b
fi
if a < min: min := a fi
if b > max: max := b fi
repeat
return pair {min, max}
end
Here, we only loop n/2 times instead of n times, but for each iteration we make three
comparisons. Thus, the number of comparisons made is (3/2)n = 1.5n, resulting in a 3/4
speed up over the original algorithm.
36](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-39-320.jpg)
![Ordered Statistics
Only three comparisons need to be made instead of four because, by construction, it's always
the case that a †b. (In the ïŹrst part of the "if", we actually know more speciïŹcally that
a < b, but under the else part, we can only conclude that a †b.) This property is utilized by
noting that a doesn't need to be compared with the current maximum, because b is already
greater than or equal to a, and similarly, b doesn't need to be compared with the current
minimum, because a is already less than or equal to b.
In software engineering, there is a struggle between using libraries versus writing customized
algorithms. In this case, the min and max functions weren't used in order to get a faster ïŹnd-
min-max routine. Such an operation would probably not be the bottleneck in a real-life
program: however, if testing reveals the routine should be faster, such an approach should be
taken. Typically, the solution that reuses libraries is better overall than writing customized
solutions. Techniques such as open implementation and aspect-oriented programming may
help manage this contention to get the best of both worlds, but regardless it's a useful
distinction to recognize.
4.1.3 ïŹnd-median
Finally, we need to consider how to ïŹnd the median value. One approach is to sort the
array then extract the median from the position vals[n/2]:
// find-median -- returns the median element of vals
function find-median(array vals[1..n]): element
assert (n > 0)
sort(vals)
return vals[n / 2]
end
If our values are not numbers close enough in value (or otherwise cannot be sorted by a
radix sort) the sort above is going to require O(nlogn) steps.
However, it is possible to extract the nth-ordered statistic in O(n) time. The key is
eliminating the sort: we don't actually require the entire array to be sorted in order to ïŹnd
the median, so there is some waste in sorting the entire array ïŹrst. One technique we'll use
to accomplish this is randomness.
Before presenting a non-sorting ïŹnd-median function, we introduce a divide and conquer-
style operation known as partitioning. What we want is a routine that ïŹnds a random
element in the array and then partitions the array into three parts:
1. elements that are less than or equal to the random element;
2. elements that are equal to the random element; and
3. elements that are greater than or equal to the random element.
These three sections are denoted by two integers: j and i. The partitioning is performed "in
place" in the array:
// partition -- break the array three partitions based on a randomly picked element
function partition(array vals): pair{j, i}
37](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-40-320.jpg)
![Randomization
Note that when the random element picked is actually represented three or more times in
the array it's possible for entries in all three partitions to have the same value as the random
element. While this operation may not sound very useful, it has a powerful property that
can be exploited: When the partition operation completes, the randomly picked element
will be in the same position in the array as it would be if the array were fully sorted!
This property might not sound so powerful, but recall the optimization for the ïŹnd-min-
max function: we noticed that by picking elements from the array in pairs and comparing
them to each other ïŹrst we could reduce the total number of comparisons needed (because
the current min and max values need to be compared with only one value each, and not
two). A similar concept is used here.
While the code for partition is not magical, it has some tricky boundary cases:
// partition -- break the array into three ordered partitions from a random element
function partition(array vals): pair{j, i}
let m := 0
let n := vals.length - 1
let irand := random(m, n) // returns any value from m to n
let x := vals[irand]
// values in vals[n..] are greater than x
// values in vals[0..m] are less than x
while (m <= n)
if vals[m] <= x
m++
else
swap(m,n) // exchange vals[n] and vals[m]
n--
endif
endwhile
// partition: [0..m-1] [] [n+1..] note that m=n+1
// if you need non empty sub-arrays:
swap(irand,n)
// partition: [0..n-1] [n..n] [n+1..]
end
We can use partition as a subroutine for a general ïŹnd operation:
// find -- moves elements in vals such that location k holds the value it would when sorted
function find(array vals, integer k)
assert (0 <= k < vals.length) // k it must be a valid index
if vals.length <= 1:
return
fi
let pair (j, i) := partition(vals)
if k <= i:
find(a[0,..,i], k)
else-if j <= k:
find(a[j,..,n], k - j)
fi
TODO: debug this!
end
Which leads us to the punch-line:
38](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-41-320.jpg)
![Quicksort
// find-median -- returns the median element of vals
function find-median(array vals): element
assert (vals.length > 0)
let median_index := vals.length / 2;
find(vals, median_index)
return vals[median_index]
end
One consideration that might cross your mind is "is the random call really necessary?"
For example, instead of picking a random pivot, we could always pick the middle element
instead. Given that our algorithm works with all possible arrays, we could conclude that the
running time on average for all of the possible inputs is the same as our analysis that used
the random function. The reasoning here is that under the set of all possible arrays, the
middle element is going to be just as "random" as picking anything else. But there's a pitfall
in this reasoning: Typically, the input to an algorithm in a program isn't random at all.
For example, the input has a higher probability of being sorted than just by chance alone.
Likewise, because it is real data from real programs, the data might have other patterns in
it that could lead to suboptimal results.
To put this another way: for the randomized median ïŹnding algorithm, there is a very
small probability it will run suboptimally, independent of what the input is; while for a
deterministic algorithm that just picks the middle element, there is a greater chance it will
run poorly on some of the most frequent input types it will receive. This leads us to the
following guideline:
Randomization Guideline:
If your algorithm depends upon randomness, be sure you introduce the randomness
yourself instead of depending upon the data to be random.
Note that there are "derandomization" techniques that can take an average-case fast algorithm
and turn it into a fully deterministic algorithm. Sometimes the overhead of derandomization
is so much that it requires very large datasets to get any gains. Nevertheless, derandomization
in itself has theoretical value.
The randomized ïŹnd algorithm was invented by C. A. R. "Tony" Hoare. While Hoare is
an important ïŹgure in computer science, he may be best known in general circles for his
quicksort algorithm, which we discuss in the next section.
4.2 Quicksort
The median-ïŹnding partitioning algorithm in the previous section is actually very close to
the implementation of a full blown sorting algorithm. Until this section is written, building
a Quicksort Algorithm is left as an exercise for the reader.
[TODO: Quicksort Algorithm]
39](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-42-320.jpg)
![Randomization
4.3 ShuïŹing an Array
This keeps data in during shuffle
temporaryArray = { }
This records if an item has been shuffled
usedItemArray = { }
Number of item in array
itemNum = 0
while ( itemNum != lengthOf( inputArray) ){
usedItemArray[ itemNum ] = false None of the items have been shuffled
itemNum = itemNum + 1
}
itemNum = 0 we'll use this again
itemPosition = randdomNumber( 0 --- (lengthOf(inputArray) - 1 ))
while( itemNum != lengthOf( inputArray ) ){
while( usedItemArray[ itemPosition ] != false ){
itemPosition = randdomNumber( 0 --- (lengthOf(inputArray)
- 1 ))
}
temporaryArray[ itemPosition ] = inputArray[ itemNum ]
}
inputArray = temporaryArray
4.4 Equal Multivariate Polynomials
[TODO: as of now, there is no known deterministic polynomial time solution, but there
is a randomized polytime solution. The canonical example used to be IsPrime, but a
deterministic, polytime solution has been found.]
4.5 Skip Lists
[TODO: Talk about skips lists. The point is to show how randomization can sometimes
make a structure easier to understand, compared to the complexity of balanced trees.] In
order to compare the complexity of implementation, it is a good idea to read about skip
lists and red-black trees ; the internet has the skip list's author paper spruiking skip lists in
a very informed and lucid way, whilst Sedgewick's graduated explanation of red-black trees
by describing them as models of 2-3 trees is also on the internet on the website promoting
his latest algorithms book.
4.5.1 Recap on red-black trees
What Sedgewick does is to diagrammatically show that the mechanisms maintaining 2-3
tree nodes as being either 2 nodes (a node with 2 children and 1 value) or 3 nodes (2 values
separating 3 children) by making sure 4 nodes are always split into three 2-nodes , and the
middle 2-node is passed up into the parent node, which may also split. 2-3 tree nodes are
really very small B-trees in behavior. The inductive proof might be that if one tries to load
everything into the left side of the tree by say a descending sequence of keys as input, the
tree still looks balanced after a height of 4 is reached, then it will looked balanced at any
height.
40](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-43-320.jpg)
![Randomization
the actual hardware itself! (e.g. sunspots can randomly ïŹip bits in hardware, causing failure,
which is a risk we take quite often)]
[Main idea: Look at all blocks of 5 elements, and pick the median (O(1) to pick), put all
medians into an array (O(n)), recursively pick the medians of that array, repeat until you
have < 5 elements in the array. This recursive median constructing of every ïŹve elements
takes time T(n)=T(n/5) + O(n), which by the master theorem is O(n). Thus, in O(n) we
can ïŹnd the right pivot. Need to show that this pivot is suïŹciently good so that we're still
O(n log n) no matter what the input is. This version of quicksort doesn't need rand, and it
never performs poorly. Still need to show that element picked out is suïŹciently good for a
pivot.]
4.8 Exercises
1. Write a ïŹnd-min function and run it on several diïŹerent inputs to demonstrate its
correctness.
44](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-47-320.jpg)
![Shortest Path Problem (exhaustive version)
The solution can be thought in terms of the back tracking methodology: Try it both ways
and see! Either way, the two sub-problems are manipulating smaller lists, so you know
that the recursion will eventually terminate. Whichever trial results in the longer common
subsequence is the winner.
Instead of "throwing it away" by deleting the item from the array we use array slices. For
example, the slice
a[1,..,5]
represents the elements
{a[1], a[2], a[3], a[4], a[5]}
of the array as an array itself. If your language doesn't support slices you'll have to pass
beginning and/or ending indices along with the full array. Here, the slices are only of the
form
a[1,..]
which, when using 0 as the index to the ïŹrst element in the array, results in an
array slice that doesn't have the 0th element. (Thus, a non-sliced version of this
algorithm would only need to pass the beginning valid index around instead, and that value
would have to be subtracted from the complete array's length to get the pseudo-slice's length.)
// lcs -- returns the longest common subsequence of a and b
function lcs(array a, array b): array
if a.length == 0 OR b.length == 0:
// if we're at the end of either list, then the lcs is empty
return new array {}
else-if a[0] == b[0]:
// if the start element is the same in both, then it is on the lcs,
// so we just recurse on the remainder of both lists.
return append(new array {a[0]}, lcs(a[1,..], b[1,..]))
else
// we don't know which list we should discard from. Try both ways,
// pick whichever is better.
let discard_a := lcs(a[1,..], b)
let discard_b := lcs(a, b[1,..])
if discard_a.length > discard_b.length:
let result := discard_a
else
let result := discard_b
fi
return result
fi
end
5.2 Shortest Path Problem (exhaustive version)
To be improved as Dijkstra's algorithm in a later section.
47](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-50-320.jpg)
![Fibonacci Numbers
(Note the algebraic manipulation used above to make the base of the exponent the number
2.) This means that there are far too many leaves, particularly considering the repeated
patterns found in the call tree above.
One optimization we can make is to save a result in a table once it's already been computed,
so that the same result needs to be computed only once. The optimization process is called
memoization and conforms to the following methodology:
Memoization Methodology
# Start with a backtracking algorithm# Look up the problem in a table; if there's a
valid entry for it, return that value# Otherwise, compute the problem recursively, and
then store the result in the table before returning the value
Consider the solution presented in the backtracking chapter for the Longest Common
Subsequence problem. In the execution of that algorithm, many common subproblems were
computed repeatedly. As an optimization, we can compute these subproblems once and
then store the result to read back later. A recursive memoization algorithm can be turned
"bottom-up" into an iterative algorithm that ïŹlls in a table of solutions to subproblems.
Some of the subproblems solved might not be needed by the end result (and that is where
dynamic programming diïŹers from memoization), but dynamic programming can be very
eïŹcient because the iterative version can better use the cache and have less call overhead.
Asymptotically, dynamic programming and memoization have the same complexity.
So how would a ïŹbonacci program using memoization work? Consider the following program
(f[n] contains the nth Fibonacci-number if has been calculated, -1 otherwise):
function fib(integer n): integer
if n == 0 or n == 1:
return n
else-if f[n] != -1:
return f[n]
else
f[n] = fib(n - 1) + fib(n - 2)
return f[n]
fi
end
This code sample is also available in Adaa
a http://en.wikibooks.org/wiki/Ada_Programming%2FAlgorithms%23Cached_Implementation
The code should be pretty obvious. If the value of ïŹb(n) already has been calculated it's
stored in f[n] and then returned instead of calculating it again. That means all the copies of
the sub-call trees are removed from the calculation.
51](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-54-320.jpg)
![Longest Common Subsequence (DP version)
We can modify the code to store the values in an array for subsequent calls, but the point is
that we don't have to. This method is typical for dynamic programming. First we identify
what subproblems need to be solved in order to solve the entire problem, and then we
calculate the values bottom-up using an iterative process.
6.2 Longest Common Subsequence (DP version)
This will remind us of the backtracking version and then improve it via memoization. Finally,
the recursive algorithm will be made iterative and be full-ïŹedged DP. [TODO: write this
section]
6.3 Matrix Chain Multiplication
Suppose that you need to multiply a series of n matrices M1,...,Mn together to form a
product matrix P:
P = M1 ·M2 ···Mnâ1 ·Mn
This will require nâ1 multiplications, but what is the fastest way we can form this product?
Matrix multiplication is associative, that is,
(A·B)·C = A·(B ·C)
for any A,B,C, and so we have some choice in what multiplication we perform ïŹrst. (Note
that matrix multiplication is not commutative, that is, it does not hold in general that
A·B = B ·A.)
Because you can only multiply two matrices at a time the product M1 ·M2 ·M3 ·M4 can be
paranthesized in these ways:
((M1M2)M3)M4
(M1(M2M3))M4
M1((M2M3)M4)
(M1M2)(M3M4)
M1(M2(M3M4))
53](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-56-320.jpg)
![Dynamic Programming
Two matrices M1 and M2 can be multiplied if the number of columns in M1 equals the
number of rows in M2. The number of rows in their product will equal the number rows
in M1 and the number of columns will equal the number of columns in M2. That is, if the
dimensions of M1 is aĂb and M2 has dimensions bĂc their product will have dimensions
aĂc.
To multiply two matrices with each other we use a function called matrix-multiply that
takes two matrices and returns their product. We will leave implementation of this function
alone for the moment as it is not the focus of this chapter (how to multiply two matrices in
the fastest way has been under intensive study for several years [TODO: propose this topic
for the Advanced book]). The time this function takes to multiply two matrices of size aĂb
and bĂc is proportional to the number of scalar multiplications, which is proportional to
abc. Thus, paranthezation matters: Say that we have three matrices M1, M2 and M3. M1
has dimensions 5Ă100, M2 has dimensions 100Ă100 and M3 has dimensions 100Ă50. Let's
paranthezise them in the two possible ways and see which way requires the least amount of
multiplications. The two ways are
((M1M2)M3), and
(M1(M2M3)).
To form the product in the ïŹrst way requires 75000 scalar multiplications (5*100*100=50000
to form product (M1M2) and another 5*100*50=25000 for the last multiplications.) This
might seem like a lot, but in comparison to the 525000 scalar multiplications required by the
second parenthesization (50*100*100=500000 plus 5*50*100=25000) it is miniscule! You
can see why determining the parenthesization is important: imagine what would happen if
we needed to multiply 50 matrices!
6.3.1 Forming a Recursive Solution
Note that we concentrate on ïŹnding a how many scalar multiplications are needed instead
of the actual order. This is because once we have found a working algorithm to ïŹnd the
amount it is trivial to create an algorithm for the actual parenthesization. It will, however,
be discussed in the end.
So how would an algorithm for the optimum parenthesization look? By the chapter
title you might expect that a dynamic programming method is in order (not to give the
answer away or anything). So how would a dynamic programming method work? Because
dynamic programming algorithms are based on optimal substructure, what would the optimal
substructure in this problem be?
Suppose that the optimal way to parenthesize
M1M2 ...Mn
splits the product at k:
(M1M2 ...Mk)(Mk+1Mk+2 ...Mn)
54](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-57-320.jpg)
![Dynamic Programming
This rather simple solution is, unfortunately, not a very good one. It spends mountains of
time recomputing data and its running time is exponential.
Using the same adaptation as above we get:
function f(m, n) {
if m == n
return 0
else-if f[m,n] != -1:
return f[m,n]
fi
let minCost := â
for k := m to n - 1 {
v := f(m, k) + f(k + 1, n) + c(k)
if v < minCost
minCost := v
}
f[m,n]=minCost
return minCost
}
6.4 Parsing Any Context-Free Grammar
Note that special types of context-free grammars can be parsed much more eïŹciently than
this technique, but in terms of generality, the DP method is the only way to go.
56](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-59-320.jpg)
![Greedy Algorithms
We ïŹrst begin with a backtracking solution to the problem:
// event-schedule -- schedule as many non-conflicting events as possible
function event-schedule(events array of s[1..n], j[1..n]): set
if n == 0: return â
fi
if n == 1: return {events[1]} fi
let event := events[1]
let S1 := union(event-schedule(events - set of conflicting events), event)
let S2 := event-schedule(events - {event})
if S1.size() >= S2.size():
return S1
else
return S2
fi
end
The above algorithm will faithfully ïŹnd the largest set of non-conïŹicting events. It brushes
aside details of how the set
events - set of conïŹicting events
is computed, but it would require O(n) time. Because the algorithm makes two recursive
calls on itself, each with an argument of size nâ1, and because removing conïŹicts takes
linear time, a recurrence for the time this algorithm takes is:
T(n) = 2·T(nâ1)+O(n)
which is O(2n).
But suppose instead of picking just the ïŹrst element in the array we used some other criterion.
The aim is to just pick the "right" one so that we wouldn't need two recursive calls. First,
let's consider the greedy strategy of picking the shortest events ïŹrst, until we can add no
more events without conïŹicts. The idea here is that the shortest events would likely interfere
less than other events.
There are scenarios were picking the shortest event ïŹrst produces the optimal result. However,
here's a scenario where that strategy is sub-optimal:
Figure 7
58](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-61-320.jpg)
![Dijkstra's Shortest Path Algorithm
Above, the optimal solution is to pick event A and C, instead of just B alone. Perhaps
instead of the shortest event we should pick the events that have the least number of conïŹicts.
This strategy seems more direct, but it fails in this scenario:
Figure 8
Above, we can maximize the number of events by picking A, B, C, D, and E. However, the
events with the least conïŹicts are 6, 2 and 7, 3. But picking one of 6, 2 and one of 7, 3
means that we cannot pick B, C and D, which includes three events instead of just two.
7.2 Dijkstra's Shortest Path Algorithm
With two (high-level, pseudocode) transformations, Dijsktra's algorithm can be derived from
the much less eïŹcient backtracking algorithm. The trick here is to prove the transformations
maintain correctness, but that's the whole insight into Dijkstra's algorithm anyway. [TODO:
important to note the paradox that to solve this problem it's easier to solve a more-general
version. That is, shortest path from s to all nodes, not just to t. Worthy of its own colored
box.]
59](https://image.slidesharecdn.com/algorithms-151129080715-lva1-app6891/85/Algorithms-62-320.jpg)
Algorithms
- 2. March 15, 2013 On the 28th of April 2012 the contents of the English as well as German Wikibooks and Wikipedia projects were licensed under Creative Commons Attribution-ShareAlike 3.0 Unported license. An URI to this license is given in the list of ïŹgures on page 81. If this document is a derived work from the contents of one of these projects and the content was still licensed by the project under this license at the time of derivation this document has to be licensed under the same, a similar or a compatible license, as stated in section 4b of the license. The list of contributors is included in chapter Contributors on page 77. The licenses GPL, LGPL and GFDL are included in chapter Licenses on page 85, since this book and/or parts of it may or may not be licensed under one or more of these licenses, and thus require inclusion of these licenses. The licenses of the ïŹgures are given in the list of ïŹgures on page 81. This PDF was generated by the LATEX typesetting software. The LATEX source code is included as an attachment (source.7z.txt) in this PDF ïŹle. To extract the source from the PDF ïŹle, we recommend the use of http://www.pdflabs.com/tools/pdftk-the-pdf-toolkit/ utility or clicking the paper clip attachment symbol on the lower left of your PDF Viewer, selecting Save Attachment. After extracting it from the PDF ïŹle you have to rename it to source.7z. To uncompress the resulting archive we recommend the use of http://www.7-zip.org/. The LATEX source itself was generated by a program written by Dirk HĂŒnniger, which is freely available under an open source license from http://de.wikibooks.org/wiki/Benutzer:Dirk_Huenniger/wb2pdf. This distribution also contains a conïŹgured version of the pdflatex compiler with all necessary packages and fonts needed to compile the LATEX source included in this PDF ïŹle.
- 3. Contents 1 Introduction 3 1.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 When is EïŹciency Important? . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Inventing an Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Understanding an Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.5 Overview of the Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.6 Algorithm and code example . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Mathematical Background 11 2.1 Asymptotic Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Algorithm Analysis: Solving Recurrence Equations . . . . . . . . . . . . . . 15 2.3 Amortized Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3 Divide and Conquer 19 3.1 Merge Sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Binary Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3 Integer Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.4 Base Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.5 Closest Pair of Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.6 Closest Pair: A Divide-and-Conquer Approach . . . . . . . . . . . . . . . . 30 3.7 Towers Of Hanoi Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4 Randomization 35 4.1 Ordered Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2 Quicksort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.3 ShuïŹing an Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.4 Equal Multivariate Polynomials . . . . . . . . . . . . . . . . . . . . . . . . 40 4.5 Skip Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.6 Treaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.7 Derandomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5 Backtracking 45 5.1 Longest Common Subsequence (exhaustive version) . . . . . . . . . . . . . 45 5.2 Shortest Path Problem (exhaustive version) . . . . . . . . . . . . . . . . . . 47 5.3 Largest Independent Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.4 Bounding Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.5 Constrained 3-Coloring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.6 Traveling Salesperson Problem . . . . . . . . . . . . . . . . . . . . . . . . . 48 III
- 4. Contents 6 Dynamic Programming 49 6.1 Fibonacci Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.2 Longest Common Subsequence (DP version) . . . . . . . . . . . . . . . . . 53 6.3 Matrix Chain Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.4 Parsing Any Context-Free Grammar . . . . . . . . . . . . . . . . . . . . . . 56 7 Greedy Algorithms 57 7.1 Event Scheduling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 7.2 Dijkstra's Shortest Path Algorithm . . . . . . . . . . . . . . . . . . . . . . . 59 7.3 Minimum spanning tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 8 Hill Climbing 61 8.1 Newton's Root Finding Method . . . . . . . . . . . . . . . . . . . . . . . . 62 8.2 Network Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 8.3 The Ford-Fulkerson Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 67 8.4 Applications of Network Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 67 9 Ada Implementation 69 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 9.2 Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 9.3 Chapter 6: Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . 70 10 Contributors 77 List of Figures 81 11 Licenses 85 11.1 GNU GENERAL PUBLIC LICENSE . . . . . . . . . . . . . . . . . . . . . 85 11.2 GNU Free Documentation License . . . . . . . . . . . . . . . . . . . . . . . 86 11.3 GNU Lesser General Public License . . . . . . . . . . . . . . . . . . . . . . 87 1
- 6. 1 Introduction This book covers techniques for the design and analysis of algorithms. The algorithmic techniques covered include: divide and conquer, backtracking, dynamic programming, greedy algorithms, and hill-climbing. Any solvable problem generally has at least one algorithm of each of the following types: 1. the obvious way; 2. the methodical way; 3. the clever way; and 4. the miraculous way. On the ïŹrst and most basic level, the "obvious" solution might try to exhaustively search for the answer. Intuitively, the obvious solution is the one that comes easily if you're familiar with a programming language and the basic problem solving techniques. The second level is the methodical level and is the heart of this book: after understanding the material presented here you should be able to methodically turn most obvious algorithms into better performing algorithms. The third level, the clever level, requires more understanding of the elements involved in the problem and their properties or even a reformulation of the algorithm (e.g., numerical algorithms exploit mathematical properties that are not obvious). A clever algorithm may be hard to understand by being non-obvious that it is correct, or it may be hard to understand that it actually runs faster than what it would seem to require. The fourth and ïŹnal level of an algorithmic solution is the miraculous level: this is reserved for the rare cases where a breakthrough results in a highly non-intuitive solution. Naturally, all of these four levels are relative, and some clever algorithms are covered in this book as well, in addition to the methodical techniques. Let's begin. 1.1 Prerequisites To understand the material presented in this book you need to know a programming language well enough to translate the pseudocode in this book into a working solution. You also need to know the basics about the following data structures: arrays, stacks, queues, linked-lists, trees, heaps (also called priority queues), disjoint sets, and graphs. Additionally, you should know some basic algorithms like binary search, a sorting algorithm (merge sort, heap sort, insertion sort, or others), and breadth-ïŹrst or depth-ïŹrst search. 3
- 7. Introduction If you are unfamiliar with any of these prerequisites you should review the material in the Data Structures1 book ïŹrst. 1.2 When is EïŹciency Important? Not every problem requires the most eïŹcient solution available. For our purposes, the term eïŹcient is concerned with the time and/or space needed to perform the task. When either time or space is abundant and cheap, it may not be worth it to pay a programmer to spend a day or so working to make a program faster. However, here are some cases where eïŹciency matters: âą When resources are limited, a change in algorithms could create great savings and allow limited machines (like cell phones, embedded systems, and sensor networks) to be stretched to the frontier of possibility. âą When the data is large a more eïŹcient solution can mean the diïŹerence between a task ïŹnishing in two days versus two weeks. Examples include physics, genetics, web searches, massive online stores, and network traïŹc analysis. âą Real time applications: the term "real time applications" actually refers to computations that give time guarantees, versus meaning "fast." However, the quality can be increased further by choosing the appropriate algorithm. âą Computationally expensive jobs, like ïŹuid dynamics, partial diïŹerential equations, VLSI design, and cryptanalysis can sometimes only be considered when the solution is found eïŹciently enough. âą When a subroutine is common and frequently used, time spent on a more eïŹcient implementation can result in beneïŹts for every application that uses the subroutine. Examples include sorting, searching, pseudorandom number generation, kernel operations (not to be confused with the operating system kernel), database queries, and graphics. In short, it's important to save time when you do not have any time to spare. When is eïŹciency unimportant? Examples of these cases include prototypes that are used only a few times, cases where the input is small, when simplicity and ease of maintenance is more important, when the area concerned is not the bottle neck, or when there's another process or area in the code that would beneïŹt far more from eïŹcient design and attention to the algorithm(s). 1.3 Inventing an Algorithm Because we assume you have some knowledge of a programming language, let's start with how we translate an idea into an algorithm. Suppose you want to write a function that will take a string as input and output the string in lowercase: 1 http://en.wikibooks.org/wiki/Computer%20Science%3AData%20Structures 4
- 8. Inventing an Algorithm // tolower -- translates all alphabetic, uppercase characters in str to lowercase function tolower(string str): string What ïŹrst comes to your mind when you think about solving this problem? Perhaps these two considerations crossed your mind: 1. Every character in str needs to be looked at 2. A routine for converting a single character to lower case is required The ïŹrst point is "obvious" because a character that needs to be converted might appear anywhere in the string. The second point follows from the ïŹrst because, once we consider each character, we need to do something with it. There are many ways of writing the tolower function for characters: function tolower(character c): character There are several ways to implement this function, including: âą look c up in a table -- a character indexed array of characters that holds the lowercase version of each character. âą check if c is in the range 'A' †c †'Z', and then add a numerical oïŹset to it. These techniques depend upon the character encoding. (As an issue of separation of concerns, perhaps the table solution is stronger because it's clearer you only need to change one part of the code.) However such a subroutine is implemented, once we have it, the implementation of our original problem comes immediately: // tolower -- translates all alphabetic, uppercase characters in str to lowercase function tolower(string str): string let result := "" for-each c in str: result.append(tolower(c)) repeat return result end This code sample is also available in Adaa a http://en.wikibooks.org/wiki/Ada_Programming%2FAlgorithms%23To_Lower The loop is the result of our ability to translate "every character needs to be looked at" into our native programming language. It became obvious that the tolower subroutine call should be in the loop's body. The ïŹnal step required to bring the high-level task into an implementation was deciding how to build the resulting string. Here, we chose to start with the empty string and append characters to the end of it. 5
- 9. Introduction Now suppose you want to write a function for comparing two strings that tests if they are equal, ignoring case: // equal-ignore-case -- returns true if s and t are equal, ignoring case function equal-ignore-case(string s, string t): boolean These ideas might come to mind: 1. Every character in strings s and t will have to be looked at 2. A single loop iterating through both might accomplish this 3. But such a loop should be careful that the strings are of equal length ïŹrst 4. If the strings aren't the same length, then they cannot be equal because the considera- tion of ignoring case doesn't aïŹect how long the string is 5. A tolower subroutine for characters can be used again, and only the lowercase versions will be compared These ideas come from familiarity both with strings and with the looping and conditional constructs in your language. The function you thought of may have looked something like this: // equal-ignore-case -- returns true if s or t are equal, ignoring case function equal-ignore-case(string s[1..n], string t[1..m]): boolean if n != m: return false if they aren't the same length, they aren't equal fi for i := 1 to n: if tolower(s[i]) != tolower(t[i]): return false fi repeat return true end This code sample is also available in Adaa a http://en.wikibooks.org/wiki/Ada_Programming%2FAlgorithms%23Equal_Ignore_Case Or, if you thought of the problem in terms of functional decomposition instead of iterations, you might have thought of a function more like this: // equal-ignore-case -- returns true if s or t are equal, ignoring case function equal-ignore-case(string s, string t): boolean return tolower(s).equals(tolower(t)) end Alternatively, you may feel neither of these solutions is eïŹcient enough, and you would prefer an algorithm that only ever made one pass of s or t. The above two implementations each require two-passes: the ïŹrst version computes the lengths and then compares each character, while the second version computes the lowercase versions of the string and then compares the results to each other. (Note that for a pair of strings, it is also possible to have the length precomputed to avoid the second pass, but that can have its own drawbacks 6
- 10. Understanding an Algorithm at times.) You could imagine how similar routines can be written to test string equality that not only ignore case, but also ignore accents. Already you might be getting the spirit of the pseudocode in this book. The pseudocode language is not meant to be a real programming language: it abstracts away details that you would have to contend with in any language. For example, the language doesn't assume generic types or dynamic versus static types: the idea is that it should be clear what is intended and it should not be too hard to convert it to your native language. (However, in doing so, you might have to make some design decisions that limit the implementation to one particular type or form of data.) There was nothing special about the techniques we used so far to solve these simple string problems: such techniques are perhaps already in your toolbox, and you may have found better or more elegant ways of expressing the solutions in your programming language of choice. In this book, we explore general algorithmic techniques to expand your toolbox even further. Taking a naive algorithm and making it more eïŹcient might not come so immediately, but after understanding the material in this book you should be able to methodically apply diïŹerent solutions, and, most importantly, you will be able to ask yourself more questions about your programs. Asking questions can be just as important as answering questions, because asking the right question can help you reformulate the problem and think outside of the box. 1.4 Understanding an Algorithm Computer programmers need an excellent ability to reason with multiple-layered abstractions. For example, consider the following code: function foo(integer a): if (a / 2) * 2 == a: print "The value " a " is even." fi end To understand this example, you need to know that integer division uses truncation and therefore when the if-condition is true then the least-signiïŹcant bit in a is zero (which means that a must be even). Additionally, the code uses a string printing API and is itself the deïŹnition of a function to be used by diïŹerent modules. Depending on the programming task, you may think on the layer of hardware, on down to the level of processor branch-prediction or the cache. Often an understanding of binary is crucial, but many modern languages have abstractions far enough away "from the hardware" that these lower-levels are not necessary. Somewhere the abstraction stops: most programmers don't need to think about logic gates, nor is the physics of electronics necessary. Nevertheless, an essential part of programming is multiple-layer thinking. But stepping away from computer programs toward algorithms requires another layer: mathematics. A program may exploit properties of binary representations. An algorithm 7
- 11. Introduction can exploit properties of set theory or other mathematical constructs. Just as binary itself is not explicit in a program, the mathematical properties used in an algorithm are not explicit. Typically, when an algorithm is introduced, a discussion (separate from the code) is needed to explain the mathematics used by the algorithm. For example, to really understand a greedy algorithm (such as Dijkstra's algorithm) you should understand the mathematical properties that show how the greedy strategy is valid for all cases. In a way, you can think of the mathematics as its own kind of subroutine that the algorithm invokes. But this "subroutine" is not present in the code because there's nothing to call. As you read this book try to think about mathematics as an implicit subroutine. 1.5 Overview of the Techniques The techniques this book covers are highlighted in the following overview. âą Divide and Conquer: Many problems, particularly when the input is given in an array, can be solved by cutting the problem into smaller pieces (divide), solving the smaller parts recursively (conquer), and then combining the solutions into a single result. Examples include the merge sort and quicksort algorithms. âą Randomization: Increasingly, randomization techniques are important for many ap- plications. This chapter presents some classical algorithms that make use of random numbers. âą Backtracking: Almost any problem can be cast in some form as a backtracking algorithm. In backtracking, you consider all possible choices to solve a problem and recursively solve subproblems under the assumption that the choice is taken. The set of recursive calls generates a tree in which each set of choices in the tree is considered consecutively. Consequently, if a solution exists, it will eventually be found.Backtracking is generally an ineïŹcient, brute-force technique, but there are optimizations that can be performed to reduce both the depth of the tree and the number of branches. The technique is called backtracking because after one leaf of the tree is visited, the algorithm will go back up the call stack (undoing choices that didn't lead to success), and then proceed down some other branch. To be solved with backtracking techniques, a problem needs to have some form of "self-similarity," that is, smaller instances of the problem (after a choice has been made) must resemble the original problem. Usually, problems can be generalized to become self-similar. âą Dynamic Programming: Dynamic programming is an optimization technique for backtracking algorithms. When subproblems need to be solved repeatedly (i.e., when there are many duplicate branches in the backtracking algorithm) time can be saved by solving all of the subproblems ïŹrst (bottom-up, from smallest to largest) and storing the solution to each subproblem in a table. Thus, each subproblem is only visited and solved once instead of repeatedly. The "programming" in this technique's name comes from programming in the sense of writing things down in a table; for example, television programming is making a table of what shows will be broadcast when. âą Greedy Algorithms: A greedy algorithm can be useful when enough information is known about possible choices that "the best" choice can be determined without considering 8
- 12. Algorithm and code example all possible choices. Typically, greedy algorithms are not challenging to write, but they are diïŹcult to prove correct. âą Hill Climbing: The ïŹnal technique we explore is hill climbing. The basic idea is to start with a poor solution to a problem, and then repeatedly apply optimizations to that solution until it becomes optimal or meets some other requirement. An important case of hill climbing is network ïŹow. Despite the name, network ïŹow is useful for many problems that describe relationships, so it's not just for computer networks. Many matching problems can be solved using network ïŹow. 1.6 Algorithm and code example 1.6.1 Level 1 (easiest) 1. Find maximum2 With algorithm and several diïŹerent programming languages 2. Find minimum3 With algorithm and several diïŹerent programming languages 3. Find average4 With algorithm and several diïŹerent programming languages 4. Find mode5 With algorithm and several diïŹerent programming languages 5. Find total6 With algorithm and several diïŹerent programming languages 6. Counting7 With algorithm and several diïŹerent programming languages 1.6.2 Level 2 1. Talking to computer Lv 18 With algorithm and several diïŹerent programming languages 2. Sorting-bubble sort9 With algorithm and several diïŹerent programming languages 3. 1.6.3 Level 3 1. Talking to computer Lv 210 With algorithm and several diïŹerent programming languages 2 http://en.wikibooks.org/wiki/..%2FFind%20maximum 3 http://en.wikibooks.org/wiki/..%2FFind%20minimum 4 http://en.wikibooks.org/wiki/..%2FFind%20average 5 http://en.wikibooks.org/wiki/..%2FFind%20mode 6 http://en.wikibooks.org/wiki/..%2FFind%20total 7 http://en.wikibooks.org/wiki/..%2FCounting 8 http://en.wikibooks.org/wiki/..%2FTalking%20to%20computer%20Lv%201 9 http://en.wikibooks.org/wiki/..%2FSorting-bubble%20sort 10 http://en.wikibooks.org/wiki/..%2FTalking%20to%20computer%20Lv%202 9
- 13. Introduction 1.6.4 Level 4 1. Talking to computer Lv 311 With algorithm and several diïŹerent programming languages 2. Find approximate maximum12 With algorithm and several diïŹerent programming languages 1.6.5 Level 5 1. Quicksort13 11 http://en.wikibooks.org/wiki/..%2FTalking%20to%20computer%20Lv%203 12 http://en.wikibooks.org/wiki/..%2FFind%20approximate%20maximum 13 http://en.wikibooks.org/wiki/Algorithm_Implementation%2FSorting%2FQuicksort 10
- 14. 2 Mathematical Background Before we begin learning algorithmic techniques, we take a detour to give ourselves some necessary mathematical tools. First, we cover mathematical deïŹnitions of terms that are used later on in the book. By expanding your mathematical vocabulary you can be more precise and you can state or formulate problems more simply. Following that, we cover techniques for analysing the running time of an algorithm. After each major algorithm covered in this book we give an analysis of its running time as well as a proof of its correctness 2.1 Asymptotic Notation In addition to correctness another important characteristic of a useful algorithm is its time and memory consumption. Time and memory are both valuable resources and there are important diïŹerences (even when both are abundant) in how we can use them. How can you measure resource consumption? One way is to create a function that describes the usage in terms of some characteristic of the input. One commonly used characteristic of an input dataset is its size. For example, suppose an algorithm takes an input as an array of n integers. We can describe the time this algorithm takes as a function f written in terms of n. For example, we might write: f(n) = n2 +3n+14 where the value of f(n) is some unit of time (in this discussion the main focus will be on time, but we could do the same for memory consumption). Rarely are the units of time actually in seconds, because that would depend on the machine itself, the system it's running, and its load. Instead, the units of time typically used are in terms of the number of some fundamental operation performed. For example, some fundamental operations we might care about are: the number of additions or multiplications needed; the number of element comparisons; the number of memory-location swaps performed; or the raw number of machine instructions executed. In general we might just refer to these fundamental operations performed as steps taken. Is this a good approach to determine an algorithm's resource consumption? Yes and no. When two diïŹerent algorithms are similar in time consumption a precise function might help to determine which algorithm is faster under given conditions. But in many cases it is either diïŹcult or impossible to calculate an analytical description of the exact number of operations needed, especially when the algorithm performs operations conditionally on the values of its input. Instead, what really is important is not the precise time required to complete the function, but rather the degree that resource consumption changes depending 11
- 15. Mathematical Background on its inputs. Concretely, consider these two functions, representing the computation time required for each size of input dataset: f(n) = n3 â12n2 +20n+110 g(n) = n3 +n2 +5n+5 They look quite diïŹerent, but how do they behave? Let's look at a few plots of the function (f(n) is in red, g(n) in blue): Figure 1 Plot of f and g, in range 0 to 5 Figure 2 Plot of f and g, in range 0 to 15 Figure 3 Plot of f and g, in range 0 to 100 Figure 4 Plot of f and g, in range 0 to 1000 In the ïŹrst, very-limited plot the curves appear somewhat diïŹerent. In the second plot they start going in sort of the same way, in the third there is only a very small diïŹerence, and at last they are virtually identical. In fact, they approach n3, the dominant term. As n gets larger, the other terms become much less signiïŹcant in comparison to n3. As you can see, modifying a polynomial-time algorithm's low-order coeïŹcients doesn't help much. What really matters is the highest-order coeïŹcient. This is why we've adopted a notation for this kind of analysis. We say that: f(n) = n3 â12n2 +20n+110 = O(n3 ) We ignore the low-order terms. We can say that: 12
- 16. Asymptotic Notation O(logn) †O( â n) †O(n) †O(nlogn) †O(n2 ) †O(n3 ) †O(2n ) This gives us a way to more easily compare algorithms with each other. Running an insertion sort on n elements takes steps on the order of O(n2). Merge sort sorts in O(nlogn) steps. Therefore, once the input dataset is large enough, merge sort is faster than insertion sort. In general, we write f(n) = O(g(n)) when âc > 0,ân0 > 0,ân â„ n0 : 0 †f(n) †c·g(n). That is, f(n) = O(g(n)) holds if and only if there exists some constants c and n0 such that for all n > n0 f(n) is positive and less than or equal to cg(n). Note that the equal sign used in this notation describes a relationship between f(n) and g(n) instead of reïŹecting a true equality. In light of this, some deïŹne Big-O in terms of a set, stating that: f(n) â O(g(n)) when f(n) â {f(n) : âc > 0,ân0 > 0,ân â„ n0 : 0 †f(n) †c·g(n)}. Big-O notation is only an upper bound; these two are both true: n3 = O(n4 ) n4 = O(n4 ) If we use the equal sign as an equality we can get very strange results, such as: n3 = n4 which is obviously nonsense. This is why the set-deïŹnition is handy. You can avoid these things by thinking of the equal sign as a one-way equality, i.e.: n3 = O(n4 ) does not imply 13
- 17. Mathematical Background O(n4 ) = n3 Always keep the O on the right hand side. 2.1.1 Big Omega Sometimes, we want more than an upper bound on the behavior of a certain function. Big Omega provides a lower bound. In general, we say that f(n) = âŠ(g(n)) when âc > 0,ân0 > 0,ân â„ n0 : 0 †c·g(n) †f(n). i.e. f(n) = âŠ(g(n)) if and only if there exist constants c and n0 such that for all n>n0 f(n) is positive and greater than or equal to cg(n). So, for example, we can say that n2 â2n = âŠ(n2), (c=1/2, n0=4) or n2 â2n = âŠ(n), (c=1, n0=3), but it is false to claim that n2 â2n = âŠ(n3 ). 2.1.2 Big Theta When a given function is both O(g(n)) and âŠ(g(n)), we say it is Î(g(n)), and we have a tight bound on the function. A function f(n) is Î(g(n)) when âc1 > 0,âc2 > 0,ân0 > 0,ân â„ n0 : 0 †c1 ·g(n) †f(n) †c2 ·g(n), but most of the time, when we're trying to prove that a given f(n) = Î(g(n)), instead of using this deïŹnition, we just show that it is both O(g(n)) and âŠ(g(n)). 2.1.3 Little-O and Omega When the asymptotic bound is not tight, we can express this by saying that f(n) = o(g(n)) or f(n) = Ï(g(n)). The deïŹnitions are: f(n) is o(g(n)) iïŹ âc > 0,ân0 > 0,ân â„ n0 : 0 †f(n) < c·g(n) and f(n) is Ï(g(n)) iïŹ âc > 0,ân0 > 0,ân â„ n0 : 0 †c·g(n) < f(n). 14
- 18. Algorithm Analysis: Solving Recurrence Equations Note that a function f is in o(g(n)) when for any coeïŹcient of g, g eventually gets larger than f, while for O(g(n)), there only has to exist a single coeïŹcient for which g eventually gets at least as big as f. [TODO: deïŹne what T(n,m) = O(f(n,m)) means. That is, when the running time of an algorithm has two dependent variables. Ex, a graph with n nodes and m edges. It's important to get the quantiïŹers correct!] 2.2 Algorithm Analysis: Solving Recurrence Equations Merge sort of n elements: T(n) = 2âT(n/2)+c(n) This describes one iteration of the merge sort: the problem space n is reduced to two halves (2 â T(n/2)), and then merged back together at the end of all the recursive calls (c(n)). This notation system is the bread and butter of algorithm analysis, so get used to it. There are some theorems you can use to estimate the big Oh time for a function if its recurrence equation ïŹts a certain pattern. [TODO: write this section] 2.2.1 Substitution method Formulate a guess about the big Oh time of your equation. Then use proof by induction to prove the guess is correct. [TODO: write this section] 2.2.2 Summations [TODO: show the closed forms of commonly needed summations and prove them] 2.2.3 Draw the Tree and Table This is really just a way of getting an intelligent guess. You still have to go back to the substitution method in order to prove the big Oh time. [TODO: write this section] 2.2.4 The Master Theorem Consider a recurrence equation that ïŹts the following formula: T(n) = aT n b +O(nk ) 15
- 19. Mathematical Background for a â„ 1, b > 1 and k â„ 0. Here, a is the number of recursive calls made per call to the function, n is the input size, b is how much smaller the input gets, and k is the polynomial order of an operation that occurs each time the function is called (except for the base cases). For example, in the merge sort algorithm covered later, we have T(n) = 2T n 2 +O(n) because two subproblems are called for each non-base case iteration, and the size of the array is divided in half each time. The O(n) at the end is the "conquer" part of this divide and conquer algorithm: it takes linear time to merge the results from the two recursive calls into the ïŹnal result. Thinking of the recursive calls of T as forming a tree, there are three possible cases to determine where most of the algorithm is spending its time ("most" in this sense is concerned with its asymptotic behaviour): 1. the tree can be top heavy, and most time is spent during the initial calls near the root; 2. the tree can have a steady state, where time is spread evenly; or 3. the tree can be bottom heavy, and most time is spent in the calls near the leaves Depending upon which of these three states the tree is in T will have diïŹerent complexities: The Master Theorem Given T(n) = aT n b +O(nk) for a â„ 1, b > 1 and k â„ 0: âą If a < bk, then T(n) = O(nk) (top heavy) âą If a = bk, then T(n) = O(nk ·logn) (steady state) âą If a > bk, then T(n) = O(nlogb a) (bottom heavy) For the merge sort example above, where T(n) = 2T n 2 +O(n) we have a = 2,b = 2,k = 1=âbk = 2 thus, a = bk and so this is also in the "steady state": By the master theorem, the complexity of merge sort is thus T(n) = O(n1 logn) = O(nlogn) 16
- 20. Amortized Analysis 2.3 Amortized Analysis [Start with an adjacency list representation of a graph and show two nested for loops: one for each node n, and nested inside that one loop for each edge e. If there are n nodes and m edges, this could lead you to say the loop takes O(nm) time. However, only once could the innerloop take that long, and a tighter bound is O(n+m).] 17
- 22. 3 Divide and Conquer The ïŹrst major algorithmic technique we cover is divide and conquer. Part of the trick of making a good divide and conquer algorithm is determining how a given problem could be separated into two or more similar, but smaller, subproblems. More generally, when we are creating a divide and conquer algorithm we will take the following steps: Divide and Conquer Methodology 1. Given a problem, identify a small number of signiïŹcantly smaller subproblems of the same type 2. Solve each subproblem recursively (the smallest possible size of a subproblem is a base-case) 3. Combine these solutions into a solution for the main problem The ïŹrst algorithm we'll present using this methodology is the merge sort. 3.1 Merge Sort The problem that merge sort solves is general sorting: given an unordered array of elements that have a total ordering, create an array that has the same elements sorted. More precisely, for an array a with indexes 1 through n, if the condition for all i, j such that 1 †i < j †n then a[i] †a[j] holds, then a is said to be sorted. Here is the interface: // sort -- returns a sorted copy of array a function sort(array a): array Following the divide and conquer methodology, how can a be broken up into smaller subproblems? Because a is an array of n elements, we might want to start by breaking the array into two arrays of size n/2 elements. These smaller arrays will also be unsorted and it is meaningful to sort these smaller problems; thus we can consider these smaller arrays "similar". Ignoring the base case for a moment, this reduces the problem into a diïŹerent one: Given two sorted arrays, how can they be combined to form a single sorted array that contains all the elements of both given arrays: 19
- 23. Divide and Conquer // merge -- given a and b (assumed to be sorted) returns a merged array that // preserves order function merge(array a, array b): array So far, following the methodology has led us to this point, but what about the base case? The base case is the part of the algorithm concerned with what happens when the problem cannot be broken into smaller subproblems. Here, the base case is when the array only has one element. The following is a sorting algorithm that faithfully sorts arrays of only zero or one elements: // base-sort -- given an array of one element (or empty), return a copy of the // array sorted function base-sort(array a[1..n]): array assert (n <= 1) return a.copy() end Putting this together, here is what the methodology has told us to write so far: // sort -- returns a sorted copy of array a function sort(array a[1..n]): array if n <= 1: return a.copy() else: let sub_size := n / 2 let first_half := sort(a[1,..,sub_size]) let second_half := sort(a[sub_size + 1,..,n]) return merge(first_half, second_half) fi end And, other than the unimplemented merge subroutine, this sorting algorithm is done! Before we cover how this algorithm works, here is how merge can be written: // merge -- given a and b (assumed to be sorted) returns a merged array that // preserves order function merge(array a[1..n], array b[1..m]): array let result := new array[n + m] let i, j := 1 for k := 1 to n + m: if i >= n: result[k] := b[j]; j += 1 else-if j >= m: result[k] := a[i]; i += 1 else: if a[i] < b[j]: result[k] := a[i]; i += 1 else: result[k] := b[j]; j += 1 fi fi repeat end 20
- 24. Merge Sort [TODO: how it works; including correctness proof] This algorithm uses the fact that, given two sorted arrays, the smallest element is always in one of two places. It's either at the head of the ïŹrst array, or the head of the second. 3.1.1 Analysis Let T(n) be the number of steps the algorithm takes to run on input of size n. Merging takes linear time and we recurse each time on two sub-problems of half the original size, so T(n) = 2·T n 2 +O(n). By the master theorem, we see that this recurrence has a "steady state" tree. Thus, the runtime is: T(n) = O(n·logn). 3.1.2 Iterative Version This merge sort algorithm can be turned into an iterative algorithm by iteratively merging each subsequent pair, then each group of four, et cetera. Due to a lack of function overhead, iterative algorithms tend to be faster in practice. However, because the recursive version's call tree is logarithmically deep, it does not require much run-time stack space: Even sorting 4 gigs of items would only require 32 call entries on the stack, a very modest amount considering if even each call required 256 bytes on the stack, it would only require 8 kilobytes. The iterative version of mergesort is a minor modiïŹcation to the recursive version - in fact we can reuse the earlier merging function. The algorithm works by merging small, sorted subsections of the original array to create larger subsections of the array which are sorted. To accomplish this, we iterate through the array with successively larger "strides". // sort -- returns a sorted copy of array a function sort_iterative(array a[1..n]): array let result := a.copy() for power := 0 to log2(n-1) let unit := 2Ëpower for i := 1 to n by unit*2 let a1[1..unit] := result[i..i+unit-1] let a2[1..unit] := result[i+unit..min(i+unit*2-1, n)] result[i..i+unit*2-1] := merge(a1,a2) repeat repeat return result end This works because each sublist of length 1 in the array is, by deïŹnition, sorted. Each iteration through the array (using counting variable i) doubles the size of sorted sublists by 21
- 25. Divide and Conquer merging adjacent sublists into sorted larger versions. The current size of sorted sublists in the algorithm is represented by the unit variable. 3.2 Binary Search Once an array is sorted, we can quickly locate items in the array by doing a binary search. Binary search is diïŹerent from other divide and conquer algorithms in that it is mostly divide based (nothing needs to be conquered). The concept behind binary search will be useful for understanding the partition and quicksort algorithms, presented in the randomization chapter. Finding an item in an already sorted array is similar to ïŹnding a name in a phonebook: you can start by ïŹipping the book open toward the middle. If the name you're looking for is on that page, you stop. If you went too far, you can start the process again with the ïŹrst half of the book. If the name you're searching for appears later than the page, you start from the second half of the book instead. You repeat this process, narrowing down your search space by half each time, until you ïŹnd what you were looking for (or, alternatively, ïŹnd where what you were looking for would have been if it were present). The following algorithm states this procedure precisely: // binary-search -- returns the index of value in the given array, or // -1 if value cannot be found. Assumes array is sorted in ascending order function binary-search(value, array A[1..n]): integer return search-inner(value, A, 1, n + 1) end // search-inner -- search subparts of the array; end is one past the // last element function search-inner(value, array A, start, end): integer if start == end: return -1 // not found fi let length := end - start if length == 1: if value == A[start]: return start else: return -1 fi fi let mid := start + (length / 2) if value == A[mid]: return mid else-if value > A[mid]: return search-inner(value, A, mid + 1, end) else: return search-inner(value, A, start, mid) fi end Note that all recursive calls made are tail-calls, and thus the algorithm is iterative. We can explicitly remove the tail-calls if our programming language does not do that for us already by turning the argument values passed to the recursive call into assignments, and 22
- 26. Binary Search then looping to the top of the function body again: // binary-search -- returns the index of value in the given array, or // -1 if value cannot be found. Assumes array is sorted in ascending order function binary-search(value, array A[1,..n]): integer let start := 1 let end := n + 1 loop: if start == end: return -1 fi // not found let length := end - start if length == 1: if value == A[start]: return start else: return -1 fi fi let mid := start + (length / 2) if value == A[mid]: return mid else-if value > A[mid]: start := mid + 1 else: end := mid fi repeat end Even though we have an iterative algorithm, it's easier to reason about the recursive version. If the number of steps the algorithm takes is T(n), then we have the following recurrence that deïŹnes T(n): T(n) = 1·T n 2 +O(1). The size of each recursive call made is on half of the input size (n), and there is a constant amount of time spent outside of the recursion (i.e., computing length and mid will take the same amount of time, regardless of how many elements are in the array). By the master theorem, this recurrence has values a = 1,b = 2,k = 0, which is a "steady state" tree, and thus we use the steady state case that tells us that T(n) = Î(nk ·logn) = Î(logn). Thus, this algorithm takes logarithmic time. Typically, even when n is large, it is safe to let the stack grow by logn activation records through recursive calls. diïŹculty in initially correct binary search implementations The article on wikipedia on Binary Search also mentions the diïŹculty in writing a correct binary search algorithm: for instance, the java Arrays.binarySearch(..) overloaded function implementation does an interative binary search which didn't work when large integers overïŹowed a simple expression of mid calculation mid = ( end + start) / 2 i.e. end + start > max_positive_integer . Hence the above algorithm is more correct in using a length = 23
- 27. Divide and Conquer end - start, and adding half length to start. The java binary Search algorithm gave a return value useful for ïŹnding the position of the nearest key greater than the search key, i.e. the position where the search key could be inserted. i.e. it returns - (keypos+1) , if the search key wasn't found exactly, but an insertion point was needed for the search key ( insertion_point = -return_value - 1). Looking at boundary values1, an insertion point could be at the front of the list ( ip = 0, return value = -1 ), to the position just after the last element, ( ip = length(A), return value = - length(A) - 1) . As an exercise, trying to implement this functionality on the above iterative binary search can be useful for further comprehension. 3.3 Integer Multiplication If you want to perform arithmetic with small integers, you can simply use the built-in arithmetic hardware of your machine. However, if you wish to multiply integers larger than those that will ïŹt into the standard "word" integer size of your computer, you will have to implement a multiplication algorithm in software or use a software implementation written by someone else. For example, RSA encryption needs to work with integers of very large size (that is, large relative to the 64-bit word size of many machines) and utilizes special multiplication algorithms.2 3.3.1 Grade School Multiplication How do we represent a large, multi-word integer? We can have a binary representation by using an array (or an allocated block of memory) of words to represent the bits of the large integer. Suppose now that we have two integers, X and Y , and we want to multiply them together. For simplicity, let's assume that both X and Y have n bits each (if one is shorter than the other, we can always pad on zeros at the beginning). The most basic way to multiply the integers is to use the grade school multiplication algorithm. This is even 1 http://en.wikibooks.org/wiki/boundary%20values 2 A (mathematical) integer larger than the largest "int" directly supported by your computer's hardware is often called a "BigInt". Working with such large numbers is often called "multiple precision arithmetic". There are entire books on the various algorithms for dealing with such numbers, such as: âą Modern Computer Arithmetic Ë{http://www.loria.fr/~zimmerma/mca/pub226.html} , Richard Brent and Paul Zimmermann, Cambridge University Press, 2010. âą Donald E. Knuth, The Art of Computer Programming , Volume 2: Seminumerical Algorithms (3rd edition), 1997. People who implement such algorithms may âą write a one-oïŹ implementation for one particular application âą write a library that you can use for many applications, such as GMP, the GNU Multiple Pre- cision Arithmetic Library Ë{http://gmplib.org/} or McCutchen's Big Integer Library Ë{https: //mattmccutchen.net/bigint/} or various libraries http://www.leemon.com/crypto/BigInt.html https://github.com/jasondavies/jsbn https://github.com/libtom/libtomcrypt http://www.gnu. org/software/gnu-crypto/ http://www.cryptopp.com/ used to demonstrate RSA encryption âą put those algorithms in the compiler of a programming language that you can use (such as Python and Lisp) that automatically switches from standard integers to BigInts when necessary 24
- 28. Integer Multiplication easier in binary, because we only multiply by 1 or 0: x6 x5 x4 x3 x2 x1 x0 Ă y6 y5 y4 y3 y2 y1 y0 ----------------------- x6 x5 x4 x3 x2 x1 x0 (when y0 is 1; 0 otherwise) x6 x5 x4 x3 x2 x1 x0 0 (when y1 is 1; 0 otherwise) x6 x5 x4 x3 x2 x1 x0 0 0 (when y2 is 1; 0 otherwise) x6 x5 x4 x3 x2 x1 x0 0 0 0 (when y3 is 1; 0 otherwise) ... et cetera As an algorithm, here's what multiplication would look like: // multiply -- return the product of two binary integers, both of length n function multiply(bitarray x[1,..n], bitarray y[1,..n]): bitarray bitarray p = 0 for i:=1 to n: if y[i] == 1: p := add(p, x) fi x := pad(x, 0) // add another zero to the end of x repeat return p end The subroutine add adds two binary integers and returns the result, and the subroutine pad adds an extra digit to the end of the number (padding on a zero is the same thing as shifting the number to the left; which is the same as multiplying it by two). Here, we loop n times, and in the worst-case, we make n calls to add. The numbers given to add will at most be of length 2n. Further, we can expect that the add subroutine can be done in linear time. Thus, if n calls to a O(n) subroutine are made, then the algorithm takes O(n2) time. 3.3.2 Divide and Conquer Multiplication As you may have ïŹgured, this isn't the end of the story. We've presented the "obvious" algorithm for multiplication; so let's see if a divide and conquer strategy can give us something better. One route we might want to try is breaking the integers up into two parts. For example, the integer x could be divided into two parts, xh and xl, for the high-order and low-order halves of x. For example, if x has n bits, we have x = xh ·2n/2 +xl We could do the same for y: y = yh ·2n/2 +yl But from this division into smaller parts, it's not clear how we can multiply these parts such that we can combine the results for the solution to the main problem. First, let's write out xĂy would be in such a system: 25
- 29. Divide and Conquer xĂy = xh Ăyh ·(2n/2 )2 +(xh Ăyl +xl Ăyh)·(2n/2 )+xl Ăyl This comes from simply multiplying the new hi/lo representations of x and y together. The multiplication of the smaller pieces are marked by the "Ă" symbol. Note that the multiplies by 2n/2 and (2n/2)2 = 2n does not require a real multiplication: we can just pad on the right number of zeros instead. This suggests the following divide and conquer algorithm: // multiply -- return the product of two binary integers, both of length n function multiply(bitarray x[1,..n], bitarray y[1,..n]): bitarray if n == 1: return x[1] * y[1] fi // multiply single digits: O(1) let xh := x[n/2 + 1, .., n] // array slicing, O(n) let xl := x[0, .., n / 2] // array slicing, O(n) let yh := y[n/2 + 1, .., n] // array slicing, O(n) let yl := y[0, .., n / 2] // array slicing, O(n) let a := multiply(xh, yh) // recursive call; T(n/2) let b := multiply(xh, yl) // recursive call; T(n/2) let c := multiply(xl, yh) // recursive call; T(n/2) let d := multiply(xl, yl) // recursive call; T(n/2) b := add(b, c) // regular addition; O(n) a := shift(a, n) // pad on zeros; O(n) b := shift(b, n/2) // pad on zeros; O(n) return add(a, b, d) // regular addition; O(n) end We can use the master theorem to analyze the running time of this algorithm. Assuming that the algorithm's running time is T(n), the comments show how much time each step takes. Because there are four recursive calls, each with an input of size n/2, we have: T(n) = 4T(n/2)+O(n) Here, a = 4,b = 2,k = 1, and given that 4 > 21 we are in the "bottom heavy" case and thus plugging in these values into the bottom heavy case of the master theorem gives us: T(n) = O(nlog2 4 ) = O(n2 ). Thus, after all of that hard work, we're still no better oïŹ than the grade school algorithm! Luckily, numbers and polynomials are a data set we know additional information about. In fact, we can reduce the running time by doing some mathematical tricks. First, let's replace the 2n/2 with a variable, z: xĂy = xh âyhz2 +(xh âyl +xl âyh)z +xl âyl This appears to be a quadratic formula, and we know that you only need three co-eïŹcients or points on a graph in order to uniquely describe a quadratic formula. However, in our above algorithm we've been using four multiplications total. Let's try recasting x and y as linear functions: 26
- 30. Base Conversion Px(z) = xh ·z +xl Py(z) = yh ·z +yl Now, for xĂy we just need to compute (Px ·Py)(2n/2). We'll evaluate Px(z) and Py(z) at three points. Three convenient points to evaluate the function will be at (Px ·Py)(1),(Px · Py)(0),(Px ·Py)(â1): [TODO: show how to make the two-parts breaking more eïŹcient; then mention that the best multiplication uses the FFT, but don't actually cover that topic (which is saved for the advanced book)] 3.4 Base Conversion [TODO: Convert numbers from decimal to binary quickly using DnC.] Along with the binary, the science of computers employs bases 8 and 16 for it's very easy to convert between the three while using bases 8 and 16 shortens considerably number representations. To represent 8 ïŹrst digits in the binary system we need 3 bits. Thus we have, 0=000, 1=001, 2=010, 3=011, 4=100, 5=101, 6=110, 7=111. Assume M=(2065)8. In order to obtain its binary representation, replace each of the four digits with the corresponding triple of bits: 010 000 110 101. After removing the leading zeros, binary representation is immediate: M=(10000110101)2. (For the hexadecimal system conversion is quite similar, except that now one should use 4-bit representation of numbers below 16.) This fact follows from the general conversion algorithm and the observation that 8=23 (and, of course, 16=24). Thus it appears that the shortest way to convert numbers into the binary system is to ïŹrst convert them into either octal or hexadecimal representation. Now let see how to implement the general algorithm programmatically. For the sake of reference, representation of a number in a system with base (radix) N may only consist of digits that are less than N. More accurately, if (1)M = akNk +akâ1Nkâ1 +...+a1N1 +a0 with 0 <= ai < N we have a representation of M in base N system and write M = (akakâ1...a0)N If we rewrite (1) as (2)M = a0 +N â(a1 +N â(a2 +N â...)) 27
- 31. Divide and Conquer the algorithm for obtaining coeïŹcients ai becomes more obvious. For example, a0 = M modulo n and a1 = (M/N) modulo n, and so on. 3.4.1 Recursive Implementation Let's represent the algorithm mnemonically: (result is a string or character variable where I shall accumulate the digits of the result one at a time) result = "" if M < N, result = 'M' + result. Stop. S = M mod N, result = 'S' + result M = M/N goto 2 A few words of explanation. "" is an empty string. You may remember it's a zero element for string concatenation. Here we check whether the conversion procedure is over. It's over if M is less than N in which case M is a digit (with some qualiïŹcation for N>10) and no additional action is necessary. Just prepend it in front of all other digits obtained previously. The '+' plus sign stands for the string concatenation. If we got this far, M is not less than N. First we extract its remainder of division by N, prepend this digit to the result as described previously, and reassign M to be M/N. This says that the whole process should be repeated starting with step 2. I would like to have a function say called Conversion that takes two arguments M and N and returns representation of the number M in base N. The function might look like this 1 String Conversion(int M, int N) // return string, accept two integers 2 { 3 if (M < N) // see if it's time to return 4 return new String(""+M); // ""+M makes a string out of a digit 5 else // the time is not yet ripe 6 return Conversion(M/N, N) + new String(""+(M mod N)); // continue 7 } This is virtually a working Java function and it would look very much the same in C++ and require only a slight modiïŹcation for C. As you see, at some point the function calls itself with a diïŹerent ïŹrst argument. One may say that the function is deïŹned in terms of itself. Such functions are called recursive. (The best known recursive function is factorial: n!=n*(n-1)!.) The function calls (applies) itself to its arguments, and then (naturally) applies itself to its new arguments, and then ... and so on. We can be sure that the process will eventually stop because the sequence of arguments (the ïŹrst ones) is decreasing. Thus sooner or later the ïŹrst argument will be less than the second and the process will start emerging from the recursion, still a step at a time. 28
- 32. Closest Pair of Points 3.4.2 Iterative Implementation Not all programming languages allow functions to call themselves recursively. Recursive functions may also be undesirable if process interruption might be expected for whatever reason. For example, in the Tower of Hanoi puzzle, the user may want to interrupt the demonstration being eager to test his or her understanding of the solution. There are complications due to the manner in which computers execute programs when one wishes to jump out of several levels of recursive calls. Note however that the string produced by the conversion algorithm is obtained in the wrong order: all digits are computed ïŹrst and then written into the string the last digit ïŹrst. Recursive implementation easily got around this diïŹculty. With each invocation of the Conversion function, computer creates a new environment in which passed values of M, N, and the newly computed S are stored. Completing the function call, i.e. returning from the function we ïŹnd the environment as it was before the call. Recursive functions store a sequence of computations implicitly. Eliminating recursive calls implies that we must manage to store the computed digits explicitly and then retrieve them in the reversed order. In Computer Science such a mechanism is known as LIFO - Last In First Out. It's best implemented with a stack data structure. Stack admits only two operations: push and pop. Intuitively stack can be visualized as indeed a stack of objects. Objects are stacked on top of each other so that to retrieve an object one has to remove all the objects above the needed one. Obviously the only object available for immediate removal is the top one, i.e. the one that got on the stack last. Then iterative implementation of the Conversion function might look as the following. 1 String Conversion(int M, int N) // return string, accept two integers 2 { 3 Stack stack = new Stack(); // create a stack 4 while (M >= N) // now the repetitive loop is clearly seen 5 { 6 stack.push(M mod N); // store a digit 7 M = M/N; // find new M 8 } 9 // now it's time to collect the digits together 10 String str = new String(""+M); // create a string with a single digit M 11 while (stack.NotEmpty()) 12 str = str+stack.pop() // get from the stack next digit 13 return str; 14 } The function is by far longer than its recursive counterpart; but, as I said, sometimes it's the one you want to use, and sometimes it's the only one you may actually use. 3.5 Closest Pair of Points For a set of points on a two-dimensional plane, if you want to ïŹnd the closest two points, you could compare all of them to each other, at O(n2) time, or use a divide and conquer algorithm. 29
- 33. Divide and Conquer [TODO: explain the algorithm, and show the nË2 algorithm] [TODO: write the algorithm, include intuition, proof of correctness, and runtime analysis] Use this link for the original document. http://www.cs.mcgill.ca/~cs251/ClosestPair/ClosestPairDQ.html 3.6 Closest Pair: A Divide-and-Conquer Approach 3.6.1 Introduction The brute force approach to the closest pair problem (i.e. checking every possible pair of points) takes quadratic time. We would now like to introduce a faster divide-and-conquer algorithm for solving the closest pair problem. Given a set of points in the plane S, our approach will be to split the set into two roughly equal halves (S1 and S2) for which we already have the solutions, and then to merge the halves in linear time to yield an O(nlogn) algorithm. However, the actual solution is far from obvious. It is possible that the desired pair might have one point in S1 and one in S2, does this not force us once again to check all possible pairs of points? The divide-and-conquer approach presented here generalizes directly from the one dimensional algorithm we presented in the previous section. 3.6.2 Closest Pair in the Plane Alright, we'll generalize our 1-D algorithm as directly as possible (see ïŹgure 3.2). Given a set of points S in the plane, we partition it into two subsets S1 and S2 by a vertical line l such that the points in S1 are to the left of l and those in S2 are to the right of l. We now recursively solve the problem on these two sets obtaining minimum distances of d1 (for S1), and d2 (for S2). We let d be the minimum of these. Now, identical to the 1-D case, if the closes pair of the whole set consists of one point from each subset, then these two points must be within d of l. This area is represented as the two strips P1 and P2 on either side of l Up to now, we are completely in step with the 1-D case. At this point, however, the extra dimension causes some problems. We wish to determine if some point in say P1 is less than d away from another point in P2. However, in the plane, we don't have the luxury that we had on the line when we observed that only one point in each set can be within d of the median. In fact, in two dimensions, all of the points could be in the strip! This is disastrous, because we would have to compare n2 pairs of points to merge the set, and hence our divide-and-conquer algorithm wouldn't save us anything in terms of eïŹciency. Thankfully, we can make another life saving observation at this point. For any particular point p in one strip, only points that meet the following constraints in the other strip need to be checked: âą those points within d of p in the direction of the other strip âą those within d of p in the positive and negative y directions 30
- 34. Closest Pair: A Divide-and-Conquer Approach Simply because points outside of this bounding box cannot be less than d units from p (see ïŹgure 3.3). It just so happens that because every point in this box is at least d apart, there can be at most six points within it. Now we don't need to check all n2 points. All we have to do is sort the points in the strip by their y-coordinates and scan the points in order, checking each point against a maximum of 6 of its neighbors. This means at most 6*n comparisons are required to check all candidate pairs. However, since we sorted the points in the strip by their y-coordinates the process of merging our two subsets is not linear, but in fact takes O(nlogn) time. Hence our full algorithm is not yet O(nlogn), but it is still an improvement on the quadratic performance of the brute force approach (as we shall see in the next section). In section 3.4, we will demonstrate how to make this algorithm even more eïŹcient by strengthening our recursive sub-solution. 3.6.3 Summary and Analysis of the 2-D Algorithm We present here a step by step summary of the algorithm presented in the previous section, followed by a performance analysis. The algorithm is simply written in list form because I ïŹnd pseudo-code to be burdensome and unnecessary when trying to understand an algorithm. Note that we pre-sort the points according to their x coordinates, and maintain another structure which holds the points sorted by their y values(for step 4), which in itself takes O(nlogn) time. ClosestPair of a set of points: 1. Divide the set into two equal sized parts by the line l, and recursively compute the minimal distance in each part. 2. Let d be the minimal of the two minimal distances. 3. Eliminate points that lie farther than d apart from l. 4. Consider the remaining points according to their y-coordinates, which we have pre- computed. 5. Scan the remaining points in the y order and compute the distances of each point to all of its neighbors that are distanced no more than d(that's why we need it sorted according to y). Note that there are no more than 5(there is no ïŹgure 3.3 , so this 5 or 6 doesnt make sense without that ïŹgure . Please include it .) such points(see previous section). 6. If any of these distances is less than d then update d. Analysis: âą Let us note T(n) as the eïŹciency of out algorithm âą Step 1 takes 2T(n/2) (we apply our algorithm for both halves) âą Step 3 takes O(n) time âą Step 5 takes O(n) time (as we saw in the previous section) so, T(n) = 2T(n/2)+O(n) which, according the Master Theorem, result 31
- 35. Divide and Conquer T(n)O(nlogn) Hence the merging of the sub-solutions is dominated by the sorting at step 4, and hence takes O(nlogn) time. This must be repeated once for each level of recursion in the divide-and-conquer algorithm, hence the whole of algorithm ClosestPair takes O(logn*nlogn) = O(nlog2n) time. 3.6.4 Improving the Algorithm We can improve on this algorithm slightly by reducing the time it takes to achieve the y-coordinate sorting in Step 4. This is done by asking that the recursive solution computed in Step 1 returns the points in sorted order by their y coordinates. This will yield two sorted lists of points which need only be merged (a linear time operation) in Step 4 in order to yield a complete sorted list. Hence the revised algorithm involves making the following changes: Step 1: Divide the set into..., and recursively compute the distance in each part, returning the points in each set in sorted order by y-coordinate. Step 4: Merge the two sorted lists into one sorted list in O(n) time. Hence the merging process is now dominated by the linear time steps thereby yielding an O(nlogn) algorithm for ïŹnding the closest pair of a set of points in the plane. 3.7 Towers Of Hanoi Problem [TODO: Write about the towers of hanoi algorithm and a program for it] There are n distinct sized discs and three pegs such that discs are placed at the left peg in the order of their sizes. The smallest one is at the top while the largest one is at the bottom. This game is to move all the discs from the left peg 3.7.1 Rules 1) Only one disc can be moved in each step. 2) Only the disc at the top can be moved. 3) Any disc can only be placed on the top of a larger disc. 3.7.2 Solution Intuitive Idea In order to move the largest disc from the left peg to the middle peg, the smallest discs must be moved to the right peg ïŹrst. After the largest one is moved. The smaller discs are then moved from the right peg to the middle peg. 32
- 36. Towers Of Hanoi Problem Recurrence Suppose n is the number of discs. To move n discs from peg a to peg b, 1) If n>1 then move n-1 discs from peg a to peg c 2) Move n-th disc from peg a to peg b 3) If n>1 then move n-1 discs from peg c to peg a Pseudocode void hanoi(n,src,dst){ if (n>1) hanoi(n-1,src,pegs-{src,dst}); print "move n-th disc from src to dst"; if (n>1) hanoi(n-1,pegs-{src,dst},dst); } Analysis The analysis is trivial. T(n) = 2T(nâ1)+O(1) = O(2n) 33
- 38. 4 Randomization As deterministic algorithms are driven to their limits when one tries to solve hard problems with them, a useful technique to speed up the computation is randomization. In randomized algorithms, the algorithm has access to a random source, which can be imagined as tossing coins during the computation. Depending on the outcome of the toss, the algorithm may split up its computation path. There are two main types of randomized algorithms: Las Vegas algorithms and Monte-Carlo algorithms. In Las Vegas algorithms, the algorithm may use the randomness to speed up the computation, but the algorithm must always return the correct answer to the input. Monte-Carlo algorithms do not have the former restriction, that is, they are allowed to give wrong return values. However, returning a wrong return value must have a small probability, otherwise that Monte-Carlo algorithm would not be of any use. Many approximation algorithms use randomization. 4.1 Ordered Statistics Before covering randomized techniques, we'll start with a deterministic problem that leads to a problem that utilizes randomization. Suppose you have an unsorted array of values and you want to ïŹnd âą the maximum value, âą the minimum value, and âą the median value. In the immortal words of one of our former computer science professors, "How can you do?" 4.1.1 ïŹnd-max First, it's relatively straightforward to ïŹnd the largest element: // find-max -- returns the maximum element function find-max(array vals[1..n]): element let result := vals[1] for i from 2 to n: result := max(result, vals[i]) repeat return result end 35
- 39. Randomization An initial assignment of ââ to result would work as well, but this is a useless call to the max function since the ïŹrst element compared gets set to result. By initializing result as such the function only requires n-1 comparisons. (Moreover, in languages capable of metaprogramming, the data type may not be strictly numerical and there might be no good way of assigning ââ; using vals[1] is type-safe.) A similar routine to ïŹnd the minimum element can be done by calling the min function instead of the max function. 4.1.2 ïŹnd-min-max But now suppose you want to ïŹnd the min and the max at the same time; here's one solution: // find-min-max -- returns the minimum and maximum element of the given array function find-min-max(array vals): pair return pair {find-min(vals), find-max(vals)} end Because ïŹnd-max and ïŹnd-min both make n-1 calls to the max or min functions (when vals has n elements), the total number of comparisons made in ïŹnd-min-max is 2nâ2. However, some redundant comparisons are being made. These redundancies can be removed by "weaving" together the min and max functions: // find-min-max -- returns the minimum and maximum element of the given array function find-min-max(array vals[1..n]): pair let min := â let max := ââ if n is odd: min := max := vals[1] vals := vals[2,..,n] // we can now assume n is even n := n - 1 fi for i:=1 to n by 2: // consider pairs of values in vals if vals[i] < vals[i + n by 2]: let a := vals[i] let b := vals[i + n by 2] else: let a := vals[i + n by 2] let b := vals[i] // invariant: a <= b fi if a < min: min := a fi if b > max: max := b fi repeat return pair {min, max} end Here, we only loop n/2 times instead of n times, but for each iteration we make three comparisons. Thus, the number of comparisons made is (3/2)n = 1.5n, resulting in a 3/4 speed up over the original algorithm. 36
- 40. Ordered Statistics Only three comparisons need to be made instead of four because, by construction, it's always the case that a †b. (In the ïŹrst part of the "if", we actually know more speciïŹcally that a < b, but under the else part, we can only conclude that a †b.) This property is utilized by noting that a doesn't need to be compared with the current maximum, because b is already greater than or equal to a, and similarly, b doesn't need to be compared with the current minimum, because a is already less than or equal to b. In software engineering, there is a struggle between using libraries versus writing customized algorithms. In this case, the min and max functions weren't used in order to get a faster ïŹnd- min-max routine. Such an operation would probably not be the bottleneck in a real-life program: however, if testing reveals the routine should be faster, such an approach should be taken. Typically, the solution that reuses libraries is better overall than writing customized solutions. Techniques such as open implementation and aspect-oriented programming may help manage this contention to get the best of both worlds, but regardless it's a useful distinction to recognize. 4.1.3 ïŹnd-median Finally, we need to consider how to ïŹnd the median value. One approach is to sort the array then extract the median from the position vals[n/2]: // find-median -- returns the median element of vals function find-median(array vals[1..n]): element assert (n > 0) sort(vals) return vals[n / 2] end If our values are not numbers close enough in value (or otherwise cannot be sorted by a radix sort) the sort above is going to require O(nlogn) steps. However, it is possible to extract the nth-ordered statistic in O(n) time. The key is eliminating the sort: we don't actually require the entire array to be sorted in order to ïŹnd the median, so there is some waste in sorting the entire array ïŹrst. One technique we'll use to accomplish this is randomness. Before presenting a non-sorting ïŹnd-median function, we introduce a divide and conquer- style operation known as partitioning. What we want is a routine that ïŹnds a random element in the array and then partitions the array into three parts: 1. elements that are less than or equal to the random element; 2. elements that are equal to the random element; and 3. elements that are greater than or equal to the random element. These three sections are denoted by two integers: j and i. The partitioning is performed "in place" in the array: // partition -- break the array three partitions based on a randomly picked element function partition(array vals): pair{j, i} 37
- 41. Randomization Note that when the random element picked is actually represented three or more times in the array it's possible for entries in all three partitions to have the same value as the random element. While this operation may not sound very useful, it has a powerful property that can be exploited: When the partition operation completes, the randomly picked element will be in the same position in the array as it would be if the array were fully sorted! This property might not sound so powerful, but recall the optimization for the ïŹnd-min- max function: we noticed that by picking elements from the array in pairs and comparing them to each other ïŹrst we could reduce the total number of comparisons needed (because the current min and max values need to be compared with only one value each, and not two). A similar concept is used here. While the code for partition is not magical, it has some tricky boundary cases: // partition -- break the array into three ordered partitions from a random element function partition(array vals): pair{j, i} let m := 0 let n := vals.length - 1 let irand := random(m, n) // returns any value from m to n let x := vals[irand] // values in vals[n..] are greater than x // values in vals[0..m] are less than x while (m <= n) if vals[m] <= x m++ else swap(m,n) // exchange vals[n] and vals[m] n-- endif endwhile // partition: [0..m-1] [] [n+1..] note that m=n+1 // if you need non empty sub-arrays: swap(irand,n) // partition: [0..n-1] [n..n] [n+1..] end We can use partition as a subroutine for a general ïŹnd operation: // find -- moves elements in vals such that location k holds the value it would when sorted function find(array vals, integer k) assert (0 <= k < vals.length) // k it must be a valid index if vals.length <= 1: return fi let pair (j, i) := partition(vals) if k <= i: find(a[0,..,i], k) else-if j <= k: find(a[j,..,n], k - j) fi TODO: debug this! end Which leads us to the punch-line: 38
- 42. Quicksort // find-median -- returns the median element of vals function find-median(array vals): element assert (vals.length > 0) let median_index := vals.length / 2; find(vals, median_index) return vals[median_index] end One consideration that might cross your mind is "is the random call really necessary?" For example, instead of picking a random pivot, we could always pick the middle element instead. Given that our algorithm works with all possible arrays, we could conclude that the running time on average for all of the possible inputs is the same as our analysis that used the random function. The reasoning here is that under the set of all possible arrays, the middle element is going to be just as "random" as picking anything else. But there's a pitfall in this reasoning: Typically, the input to an algorithm in a program isn't random at all. For example, the input has a higher probability of being sorted than just by chance alone. Likewise, because it is real data from real programs, the data might have other patterns in it that could lead to suboptimal results. To put this another way: for the randomized median ïŹnding algorithm, there is a very small probability it will run suboptimally, independent of what the input is; while for a deterministic algorithm that just picks the middle element, there is a greater chance it will run poorly on some of the most frequent input types it will receive. This leads us to the following guideline: Randomization Guideline: If your algorithm depends upon randomness, be sure you introduce the randomness yourself instead of depending upon the data to be random. Note that there are "derandomization" techniques that can take an average-case fast algorithm and turn it into a fully deterministic algorithm. Sometimes the overhead of derandomization is so much that it requires very large datasets to get any gains. Nevertheless, derandomization in itself has theoretical value. The randomized ïŹnd algorithm was invented by C. A. R. "Tony" Hoare. While Hoare is an important ïŹgure in computer science, he may be best known in general circles for his quicksort algorithm, which we discuss in the next section. 4.2 Quicksort The median-ïŹnding partitioning algorithm in the previous section is actually very close to the implementation of a full blown sorting algorithm. Until this section is written, building a Quicksort Algorithm is left as an exercise for the reader. [TODO: Quicksort Algorithm] 39
- 43. Randomization 4.3 ShuïŹing an Array This keeps data in during shuffle temporaryArray = { } This records if an item has been shuffled usedItemArray = { } Number of item in array itemNum = 0 while ( itemNum != lengthOf( inputArray) ){ usedItemArray[ itemNum ] = false None of the items have been shuffled itemNum = itemNum + 1 } itemNum = 0 we'll use this again itemPosition = randdomNumber( 0 --- (lengthOf(inputArray) - 1 )) while( itemNum != lengthOf( inputArray ) ){ while( usedItemArray[ itemPosition ] != false ){ itemPosition = randdomNumber( 0 --- (lengthOf(inputArray) - 1 )) } temporaryArray[ itemPosition ] = inputArray[ itemNum ] } inputArray = temporaryArray 4.4 Equal Multivariate Polynomials [TODO: as of now, there is no known deterministic polynomial time solution, but there is a randomized polytime solution. The canonical example used to be IsPrime, but a deterministic, polytime solution has been found.] 4.5 Skip Lists [TODO: Talk about skips lists. The point is to show how randomization can sometimes make a structure easier to understand, compared to the complexity of balanced trees.] In order to compare the complexity of implementation, it is a good idea to read about skip lists and red-black trees ; the internet has the skip list's author paper spruiking skip lists in a very informed and lucid way, whilst Sedgewick's graduated explanation of red-black trees by describing them as models of 2-3 trees is also on the internet on the website promoting his latest algorithms book. 4.5.1 Recap on red-black trees What Sedgewick does is to diagrammatically show that the mechanisms maintaining 2-3 tree nodes as being either 2 nodes (a node with 2 children and 1 value) or 3 nodes (2 values separating 3 children) by making sure 4 nodes are always split into three 2-nodes , and the middle 2-node is passed up into the parent node, which may also split. 2-3 tree nodes are really very small B-trees in behavior. The inductive proof might be that if one tries to load everything into the left side of the tree by say a descending sequence of keys as input, the tree still looks balanced after a height of 4 is reached, then it will looked balanced at any height. 40
- 44. Skip Lists He then goes to assert that a 2 node can be modeled as binary node which has a black link from its parent, and a black link to each of its two children, and a link's color is carried by a color attribute on the child node of the link; and a 3-node is modeled as a binary node which has a red link between a child to a parent node whose other child is marked black, and the parent itself is marked black ; hence the 2 nodes of a 3-node is the parent red-linked to the red child. A 4-node which must be split, then becomes any 3 node combination which has 2 red links, or 2 nodes marked red, and one node , a top most parent, marked black. The 4 node can occur as a straight line of red links, between grandparent, red parent, and red grand child, a zig-zag, or a bow , black parent, with 2 red children, but everything gets converted to the last in order to simplify splitting of the 4-node. To make calculations easier, if the red child is on the right of the parent, then a rotation should be done to make the relationship a left child relationship. This left rotation1 is done by saving the red child's left child, making the parent the child's left child, the child's old left child the parents right child ( formerly the red child), and the old parent's own parent the parent of the red child (by returning a reference to the old red child instead of the old parent), and then making the red child black, and the old parent red. Now the old parent has become a left red child of the former red child, but the relationship is still the two same keys making up the 3-node. When an insertion is done, the node to be inserted is always red, to model the 2-3 tree behavior of always adding to an existing node initially ( a 2-node with all black linked children marked black, where terminal null links are black, will become a 3-node with a new red child). Again, if the red child is on the right, a left rotation is done. It then suïŹces to see if the parent of the newly inserted red node is red (actually, Sedgewick does this as a recursive post insertion check, as each recursive function returns and the tree is walked up again, he checks for a node whose left child is red, and whose left child has a red left child ) ; and if so, a 4-node exists, because there are two red links together. Since all red children have been rotated to be left children, it suïŹces to right rotate at the parent's parent, in order to make the middle node of the 4- node the ancestor linked node, and splitting of the 4 node and passing up the middle node is done by then making left and right nodes black, and the center node red, (which is equivalent to making the end keys of a 4-node into two 2-nodes, and the middle node passed up and merged with node above it). Then , the above process beginning with "if the red child is on the right of the parent ..." should then be carried out recursively with the red center node, until all red nodes are left child's , and the parent is not marked red. The main operation of rotation is probably only marginally more diïŹcult than understanding insertion into a singly linked list, which is what skip lists are built on , except that the nodes to be inserted have a variable height. 4.5.2 Skip list structure A skip list node is an array of singly-linked list nodes, of randomized array length (height) , where a node array element of a given index has a pointer to another node array element only at the same index (=level) , of another skip list node. The start skip list node is 1 http://en.wikibooks.org/wiki/..%2FLeft%20rotation 41
- 45. Randomization referenced as the header of the skip list, and must be as high as the highest node in the skip list , because an element at a given index(height) only point to another element at the same index in another node, and hence is only reachable if the header node has the given level (height). In order to get a height of a certain level n, a loop is iterated n times where a random function has successful generated a value below a certain threshold on n iterations. So for an even chance threshold of 0.5, and a random function generating 0 to 1.0, to achieve a height of 4, it would be 0.5 Ë 4 total probability or (1/2)Ë4 = 1/16. Therefore, high nodes are much less common than short nodes , which have a probability of 0.5 of the threshold succeeding the ïŹrst time. Insertion of a newly generated node of a randomized height, begins with a search at the highest level of the skip list's node, or the highest level of the inserting node, whichever is smaller. Once the position is found, if the node has an overall height greater than the skip list header, the skip list header is increased to the height of the excess levels, and all the new level elements of the header node are made to point to the inserting node ( with the usual rule of pointing only to elements of the same level). Beginning with header node of the skip list ,a search is made as in an ordered linked list for where to insert the node. The idea is to ïŹnd the last node which has a key smaller than insertion key by ïŹnding a next node's key greater than the inserting key; and this last smaller node will be the previous node in a linked list insertion. However, the previous node is diïŹerent at diïŹerent levels, and so an array must be used to hold the previous node at each level. When the smallest previous node is found, search is recommenced at the next level lower, and this continues until the lowest level . Once the previous node is known for all levels of the inserting node, the inserting node's next pointer at each level must be made to point to the next pointer of the node in the saved array at the same level . Then all the nodes in the array must have their next pointer point to the newly inserted node. Although it is claimed to be easier to implement, there is two simultaneous ideas going on, and the locality of change is greater than say just recursively rotating tree nodes, so it is probably easier to implement, if the original paper by Pugh is printed out and in front of you, and you copy the skeleton of the spelled out algorithm as pseudocode from the paper down into comments, and then implement the comments. It is still basically singly linked list insertion, with a handle to the node just before , whose next pointer must be copied as the inserted node's next pointer, before the next pointer is updated as the inserted node; but there are other tricky things to remember, such as having two nested loops, a temporary array of previous node references to remember which node is the previous node at which level ; not inserting a node if the key already exists and is found, making sure the list header doesn't need to be updated because the height of the inserting node is the greatest encountered so far, and making multiple linked list insertion by iterating through the temporary array of previous pointers. 4.5.3 Role of Randomness The idea of making higher nodes geometrically randomly less common, means there are less keys to compare with the higher the level of comparison, and since these are randomly selected, this should get rid of problems of degenerate input that makes it necessary to do 42
- 46. Treaps tree balancing in tree algorithms. Since the higher level list have more widely separated elements, but the search algorithm moves down a level after each search terminates at a level, the higher levels help "skip" over the need to search earlier elements on lower lists. Because there are multiple levels of skipping, it becomes less likely that a meagre skip at a higher level won't be compensated by better skips at lower levels, and Pugh claims O(logN) performance overall. Conceptually , is it easier to understand than balancing trees and hence easier to implement ? The development of ideas from binary trees, balanced binary trees, 2-3 trees, red-black trees, and B-trees make a stronger conceptual network but is progressive in development, so arguably, once red-black trees are understood, they have more conceptual context to aid memory , or refresh of memory. 4.5.4 Idea for an exercise Replace the Linux completely fair scheduler red-black tree implementation with a skip list , and see how your brand of Linux runs after recompiling. 4.6 Treaps A treap is a two keyed binary tree, that uses a second randomly generated key and the previously discussed tree operation of parent-child rotation to randomly rotate the tree so that overall, a balanced tree is produced. Recall that binary trees work by having all nodes in the left subtree small than a given node, and all nodes in a right subtree greater. Also recall that node rotation does not break this order ( some people call it an invariant), but changes the relationship of parent and child, so that if the parent was smaller than a right child, then the parent becomes the left child of the formerly right child. The idea of a tree-heap or treap, is that a binary heap relationship is maintained between parents and child, and that is a parent node has higher priority than its children, which is not the same as the left , right order of keys in a binary tree, and hence a recently inserted leaf node in a binary tree which happens to have a high random priority, can be rotated so it is relatively higher in the tree, having no parent with a lower priority. See the preamble to skip lists about red-black trees on the details of left rotation2. A treap is an alternative to both red-black trees, and skip lists, as a self-balancing sorted storage structure. 4.7 Derandomization [TODO: Deterministic algorithms for Quicksort exist that perform as well as quicksort in the average case and are guaranteed to perform at least that well in all cases. Best of all, no randomization is needed. Also in the discussion should be some perspective on using randomization: some randomized algorithms give you better conïŹdence probabilities than 2 http://en.wikibooks.org/wiki/..%2FLeft%20rotation 43
- 47. Randomization the actual hardware itself! (e.g. sunspots can randomly ïŹip bits in hardware, causing failure, which is a risk we take quite often)] [Main idea: Look at all blocks of 5 elements, and pick the median (O(1) to pick), put all medians into an array (O(n)), recursively pick the medians of that array, repeat until you have < 5 elements in the array. This recursive median constructing of every ïŹve elements takes time T(n)=T(n/5) + O(n), which by the master theorem is O(n). Thus, in O(n) we can ïŹnd the right pivot. Need to show that this pivot is suïŹciently good so that we're still O(n log n) no matter what the input is. This version of quicksort doesn't need rand, and it never performs poorly. Still need to show that element picked out is suïŹciently good for a pivot.] 4.8 Exercises 1. Write a ïŹnd-min function and run it on several diïŹerent inputs to demonstrate its correctness. 44
- 48. 5 Backtracking Backtracking is a general algorithmic technique that considers searching every possible combination in order to solve an optimization problem. Backtracking is also known as depth-ïŹrst search or branch and bound. By inserting more knowledge of the problem, the search tree can be pruned to avoid considering cases that don't look promising. While backtracking is useful for hard problems to which we do not know more eïŹcient solutions, it is a poor solution for the everyday problems that other techniques are much better at solving. However, dynamic programming and greedy algorithms can be thought of as optimizations to backtracking, so the general technique behind backtracking is useful for understanding these more advanced concepts. Learning and understanding backtracking techniques ïŹrst provides a good stepping stone to these more advanced techniques because you won't have to learn several new concepts all at once. Backtracking Methodology # View picking a solution as a sequence of choices# For each choice, consider every option recursively# Return the best solution found This methodology is generic enough that it can be applied to most problems. However, even when taking care to improve a backtracking algorithm, it will probably still take exponen- tial time rather than polynomial time. Additionally, exact time analysis of backtracking algorithms can be extremely diïŹcult: instead, simpler upperbounds that may not be tight are given. 5.1 Longest Common Subsequence (exhaustive version) Note that the solution to the longest common subsequence (LCS) problem discussed in this section is not eïŹcient. However, it is useful for understanding the dynamic programming version of the algorithm that is covered later. The LCS problem is similar to what the Unix "diïŹ" program does. The diïŹ command in Unix takes two text ïŹles, A and B, as input and outputs the diïŹerences line-by-line from A and B. For example, diïŹ can show you that lines missing from A have been added to B, and lines present in A have been removed from B. The goal is to get a list of additions and removals that could be used to transform A to B. An overly conservative solution to the problem would say that all lines from A were removed, and that all lines from B were added. While this would solve the problem in a crude sense, we are concerned with the minimal number of additions and removals to achieve a correct transformation. Consider how you may implement a solution to this problem yourself. 45
- 49. Backtracking The LCS problem, instead of dealing with lines in text ïŹles, is concerned with ïŹnding common items between two diïŹerent arrays. For example, let a := array {"The", "great", "square", "has", "no", "corners"} let b := array {"The", "great", "image", "has", "no", "form"} We want to ïŹnd the longest subsequence possible of items that are found in both a and b in the same order. The LCS of a and b is "The", "great", "has", "no" Now consider two more sequences: let c := array {1, 2, 4, 8, 16, 32} let d := array {1, 2, 3, 32, 8} Here, there are two longest common subsequences of c and d: 1, 2, 32; and 1, 2, 8 Note that 1, 2, 32, 8 is not a common subsequence, because it is only a valid subsequence of d and not c (because c has 8 before the 32). Thus, we can conclude that for some cases, solutions to the LCS problem are not unique. If we had more information about the sequences available we might prefer one subsequence to another: for example, if the sequences were lines of text in computer programs, we might choose the subsequences that would keep function deïŹnitions or paired comment delimiters intact (instead of choosing delimiters that were not paired in the syntax). On the top level, our problem is to implement the following function // lcs -- returns the longest common subsequence of a and b function lcs(array a, array b): array which takes in two arrays as input and outputs the subsequence array. How do you solve this problem? You could start by noticing that if the two sequences start with the same word, then the longest common subsequence always contains that word. You can automatically put that word on your list, and you would have just reduced the problem to ïŹnding the longest common subset of the rest of the two lists. Thus, the problem was made smaller, which is good because it shows progress was made. But if the two lists do not begin with the same word, then one, or both, of the ïŹrst element in a or the ïŹrst element in b do not belong in the longest common subsequence. But yet, one of them might be. How do you determine which one, if any, to add? 46
- 50. Shortest Path Problem (exhaustive version) The solution can be thought in terms of the back tracking methodology: Try it both ways and see! Either way, the two sub-problems are manipulating smaller lists, so you know that the recursion will eventually terminate. Whichever trial results in the longer common subsequence is the winner. Instead of "throwing it away" by deleting the item from the array we use array slices. For example, the slice a[1,..,5] represents the elements {a[1], a[2], a[3], a[4], a[5]} of the array as an array itself. If your language doesn't support slices you'll have to pass beginning and/or ending indices along with the full array. Here, the slices are only of the form a[1,..] which, when using 0 as the index to the ïŹrst element in the array, results in an array slice that doesn't have the 0th element. (Thus, a non-sliced version of this algorithm would only need to pass the beginning valid index around instead, and that value would have to be subtracted from the complete array's length to get the pseudo-slice's length.) // lcs -- returns the longest common subsequence of a and b function lcs(array a, array b): array if a.length == 0 OR b.length == 0: // if we're at the end of either list, then the lcs is empty return new array {} else-if a[0] == b[0]: // if the start element is the same in both, then it is on the lcs, // so we just recurse on the remainder of both lists. return append(new array {a[0]}, lcs(a[1,..], b[1,..])) else // we don't know which list we should discard from. Try both ways, // pick whichever is better. let discard_a := lcs(a[1,..], b) let discard_b := lcs(a, b[1,..]) if discard_a.length > discard_b.length: let result := discard_a else let result := discard_b fi return result fi end 5.2 Shortest Path Problem (exhaustive version) To be improved as Dijkstra's algorithm in a later section. 47
- 51. Backtracking 5.3 Largest Independent Set 5.4 Bounding Searches If you've already found something "better" and you're on a branch that will never be as good as the one you already saw, you can terminate that branch early. (Example to use: sum of numbers beginning with 1 2, and then each number following is a sum of any of the numbers plus the last number. Show performance improvements.) 5.5 Constrained 3-Coloring This problem doesn't have immediate self-similarity, so the problem ïŹrst needs to be generalized. Methodology: If there's no self-similarity, try to generalize the problem until it has it. 5.6 Traveling Salesperson Problem Here, backtracking is one of the best solutions known. 48
- 52. 6 Dynamic Programming Dynamic programming can be thought of as an optimization technique for particular classes of backtracking algorithms where subproblems are repeatedly solved. Note that the term dynamic in dynamic programming should not be confused with dynamic programming languages, like Scheme or Lisp. Nor should the term programming be confused with the act of writing computer programs. In the context of algorithms, dynamic programming always refers to the technique of ïŹlling in a table with values computed from other table values. (It's dynamic because the values in the table are ïŹlled in by the algorithm based on other values of the table, and it's programming in the sense of setting things in a table, like how television programming is concerned with when to broadcast what shows.) 6.1 Fibonacci Numbers Before presenting the dynamic programming technique, it will be useful to ïŹrst show a related technique, called memoization, on a toy example: The Fibonacci numbers. What we want is a routine to compute the nth Fibonacci number: // fib -- compute Fibonacci(n) function fib(integer n): integer By deïŹnition, the nth Fibonacci number, denoted Fn is F0 = 0 F1 = 1 Fn = Fnâ1 +Fnâ2 How would one create a good algorithm for ïŹnding the nth Fibonacci-number? Let's begin with the naive algorithm, which codes the mathematical deïŹnition: // fib -- compute Fibonacci(n) function fib(integer n): integer assert (n >= 0) if n == 0: return 0 fi if n == 1: return 1 fi 49
- 53. Dynamic Programming return fib(n - 1) + fib(n - 2) end This code sample is also available in Adaa a http://en.wikibooks.org/wiki/Ada_Programming%2FAlgorithms%23Simple_Implementation Note that this is a toy example because there is already a mathematically closed form for Fn: F(n) = Ïn â(1âÏ)n â 5 where: Ï = 1+ â 5 2 This latter equation is known as the Golden Ratio1. Thus, a program could eïŹciently calculate Fn for even very large n. However, it's instructive to understand what's so ineïŹcient about the current algorithm. To analyze the running time of fib we should look at a call tree for something even as small as the sixth Fibonacci number: Figure 5 Every leaf of the call tree has the value 0 or 1, and the sum of these values is the ïŹnal result. So, for any n, the number of leaves in the call tree is actually Fn itself! The closed form thus tells us that the number of leaves in fib(n) is approximately equal to 1+ â 5 2 n â 1.618n = 2lg(1.618n) = 2nlg(1.618) â 20.69n . 1 http://en.wikipedia.org/wiki/golden_ratio 50
- 54. Fibonacci Numbers (Note the algebraic manipulation used above to make the base of the exponent the number 2.) This means that there are far too many leaves, particularly considering the repeated patterns found in the call tree above. One optimization we can make is to save a result in a table once it's already been computed, so that the same result needs to be computed only once. The optimization process is called memoization and conforms to the following methodology: Memoization Methodology # Start with a backtracking algorithm# Look up the problem in a table; if there's a valid entry for it, return that value# Otherwise, compute the problem recursively, and then store the result in the table before returning the value Consider the solution presented in the backtracking chapter for the Longest Common Subsequence problem. In the execution of that algorithm, many common subproblems were computed repeatedly. As an optimization, we can compute these subproblems once and then store the result to read back later. A recursive memoization algorithm can be turned "bottom-up" into an iterative algorithm that ïŹlls in a table of solutions to subproblems. Some of the subproblems solved might not be needed by the end result (and that is where dynamic programming diïŹers from memoization), but dynamic programming can be very eïŹcient because the iterative version can better use the cache and have less call overhead. Asymptotically, dynamic programming and memoization have the same complexity. So how would a ïŹbonacci program using memoization work? Consider the following program (f[n] contains the nth Fibonacci-number if has been calculated, -1 otherwise): function fib(integer n): integer if n == 0 or n == 1: return n else-if f[n] != -1: return f[n] else f[n] = fib(n - 1) + fib(n - 2) return f[n] fi end This code sample is also available in Adaa a http://en.wikibooks.org/wiki/Ada_Programming%2FAlgorithms%23Cached_Implementation The code should be pretty obvious. If the value of ïŹb(n) already has been calculated it's stored in f[n] and then returned instead of calculating it again. That means all the copies of the sub-call trees are removed from the calculation. 51
- 55. Dynamic Programming Figure 6 The values in the blue boxes are values that already have been calculated and the calls can thus be skipped. It is thus a lot faster than the straight-forward recursive algorithm. Since every value less than n is calculated once, and only once, the ïŹrst time you execute it, the asymptotic running time is O(n). Any other calls to it will take O(1) since the values have been precalculated (assuming each subsequent call's argument is less than n). The algorithm does consume a lot of memory. When we calculate ïŹb(n), the values ïŹb(0) to ïŹb(n) are stored in main memory. Can this be improved? Yes it can, al- though the O(1) running time of subsequent calls are obviously lost since the values aren't stored. Since the value of ïŹb(n) only depends on ïŹb(n-1) and ïŹb(n-2) we can discard the other values by going bottom-up. If we want to calculate ïŹb(n), we ïŹrst calculate ïŹb(2) = ïŹb(0) + ïŹb(1). Then we can calculate ïŹb(3) by adding ïŹb(1) and ïŹb(2). After that, ïŹb(0) and ïŹb(1) can be discarded, since we don't need them to calculate any more values. From ïŹb(2) and ïŹb(3) we calculate ïŹb(4) and discard ïŹb(2), then we calculate ïŹb(5) and discard ïŹb(3), etc. etc. The code goes something like this: function fib(integer n): integer if n == 0 or n == 1: return n fi let u := 0 let v := 1 for i := 2 to n: let t := u + v u := v v := t repeat return v end This code sample is also available in Adaa a http://en.wikibooks.org/wiki/Ada_Programming%2FAlgorithms%23Memory_Optimized_ Implementation 52
- 56. Longest Common Subsequence (DP version) We can modify the code to store the values in an array for subsequent calls, but the point is that we don't have to. This method is typical for dynamic programming. First we identify what subproblems need to be solved in order to solve the entire problem, and then we calculate the values bottom-up using an iterative process. 6.2 Longest Common Subsequence (DP version) This will remind us of the backtracking version and then improve it via memoization. Finally, the recursive algorithm will be made iterative and be full-ïŹedged DP. [TODO: write this section] 6.3 Matrix Chain Multiplication Suppose that you need to multiply a series of n matrices M1,...,Mn together to form a product matrix P: P = M1 ·M2 ···Mnâ1 ·Mn This will require nâ1 multiplications, but what is the fastest way we can form this product? Matrix multiplication is associative, that is, (A·B)·C = A·(B ·C) for any A,B,C, and so we have some choice in what multiplication we perform ïŹrst. (Note that matrix multiplication is not commutative, that is, it does not hold in general that A·B = B ·A.) Because you can only multiply two matrices at a time the product M1 ·M2 ·M3 ·M4 can be paranthesized in these ways: ((M1M2)M3)M4 (M1(M2M3))M4 M1((M2M3)M4) (M1M2)(M3M4) M1(M2(M3M4)) 53
- 57. Dynamic Programming Two matrices M1 and M2 can be multiplied if the number of columns in M1 equals the number of rows in M2. The number of rows in their product will equal the number rows in M1 and the number of columns will equal the number of columns in M2. That is, if the dimensions of M1 is aĂb and M2 has dimensions bĂc their product will have dimensions aĂc. To multiply two matrices with each other we use a function called matrix-multiply that takes two matrices and returns their product. We will leave implementation of this function alone for the moment as it is not the focus of this chapter (how to multiply two matrices in the fastest way has been under intensive study for several years [TODO: propose this topic for the Advanced book]). The time this function takes to multiply two matrices of size aĂb and bĂc is proportional to the number of scalar multiplications, which is proportional to abc. Thus, paranthezation matters: Say that we have three matrices M1, M2 and M3. M1 has dimensions 5Ă100, M2 has dimensions 100Ă100 and M3 has dimensions 100Ă50. Let's paranthezise them in the two possible ways and see which way requires the least amount of multiplications. The two ways are ((M1M2)M3), and (M1(M2M3)). To form the product in the ïŹrst way requires 75000 scalar multiplications (5*100*100=50000 to form product (M1M2) and another 5*100*50=25000 for the last multiplications.) This might seem like a lot, but in comparison to the 525000 scalar multiplications required by the second parenthesization (50*100*100=500000 plus 5*50*100=25000) it is miniscule! You can see why determining the parenthesization is important: imagine what would happen if we needed to multiply 50 matrices! 6.3.1 Forming a Recursive Solution Note that we concentrate on ïŹnding a how many scalar multiplications are needed instead of the actual order. This is because once we have found a working algorithm to ïŹnd the amount it is trivial to create an algorithm for the actual parenthesization. It will, however, be discussed in the end. So how would an algorithm for the optimum parenthesization look? By the chapter title you might expect that a dynamic programming method is in order (not to give the answer away or anything). So how would a dynamic programming method work? Because dynamic programming algorithms are based on optimal substructure, what would the optimal substructure in this problem be? Suppose that the optimal way to parenthesize M1M2 ...Mn splits the product at k: (M1M2 ...Mk)(Mk+1Mk+2 ...Mn) 54
- 58. Matrix Chain Multiplication Then the optimal solution contains the optimal solutions to the two subproblems (M1 ...Mk) (Mk+1 ...Mn) That is, just in accordance with the fundamental principle of dynamic programming, the solution to the problem depends on the solution of smaller sub-problems. Let's say that it takes c(n) scalar multiplications to multiply matrices Mn and Mn+1, and f(m,n) is the number of scalar multiplications to be performed in an optimal parenthesization of the matrices Mm ...Mn. The deïŹnition of f(m,n) is the ïŹrst step toward a solution. When n â m = 1, the formulation is trivial; it is just c(m). But what is it when the distance is larger? Using the observation above, we can derive a formulation. Sup- pose an optimal solution to the problem divides the matrices at matrices k and k+1 (i.e. (Mm ...Mk)(Mk+1 ...Mn)) then the number of scalar multiplications are. f(m,k)+f(k +1,n)+c(k) That is, the amount of time to form the ïŹrst product, the amount of time it takes to form the second product, and the amount of time it takes to multiply them together. But what is this optimal value k? The answer is, of course, the value that makes the above formula assume its minimum value. We can thus form the complete deïŹnition for the function: f(m,n) = minmâ€k<n f(m,k)+f(k +1,n)+c(k) if nâm > 1 0 if n = m A straight-forward recursive solution to this would look something like this (the language is Wikicode2): function f(m, n) { if m == n return 0 let minCost := â for k := m to n - 1 { v := f(m, k) + f(k + 1, n) + c(k) if v < minCost minCost := v } return minCost } 2 http://en.wikipedia.org/wiki/Wikipedia:Wikicode 55
- 59. Dynamic Programming This rather simple solution is, unfortunately, not a very good one. It spends mountains of time recomputing data and its running time is exponential. Using the same adaptation as above we get: function f(m, n) { if m == n return 0 else-if f[m,n] != -1: return f[m,n] fi let minCost := â for k := m to n - 1 { v := f(m, k) + f(k + 1, n) + c(k) if v < minCost minCost := v } f[m,n]=minCost return minCost } 6.4 Parsing Any Context-Free Grammar Note that special types of context-free grammars can be parsed much more eïŹciently than this technique, but in terms of generality, the DP method is the only way to go. 56
- 60. 7 Greedy Algorithms In the backtracking algorithms we looked at, we saw algorithms that found decision points and recursed over all options from that decision point. A greedy algorithm can be thought of as a backtracking algorithm where at each decision point "the best" option is already known and thus can be picked without having to recurse over any of the alternative options. The name "greedy" comes from the fact that the algorithms make decisions based on a single criterion, instead of a global analysis that would take into account the decision's eïŹect on further steps. As we will see, such a backtracking analysis will be unnecessary in the case of greedy algorithms, so it is not greedy in the sense of causing harm for only short-term gain. Unlike backtracking algorithms, greedy algorithms can't be made for every problem. Not every problem is "solvable" using greedy algorithms. Viewing the ïŹnding solution to an optimization problem as a hill climbing problem greedy algorithms can be used for only those hills where at every point taking the steepest step would lead to the peak always. Greedy algorithms tend to be very eïŹcient and can be implemented in a relatively straight- forward fashion. Many a times in O(n) complexity as there would be a single choice at every point. However, most attempts at creating a correct greedy algorithm fail unless a precise proof of the algorithm's correctness is ïŹrst demonstrated. When a greedy strategy fails to produce optimal results on all inputs, we instead refer to it as a heuristic instead of an algorithm. Heuristics can be useful when speed is more important than exact results (for example, when "good enough" results are suïŹcient). 7.1 Event Scheduling Problem The ïŹrst problem we'll look at that can be solved with a greedy algorithm is the event scheduling problem. We are given a set of events that have a start time and ïŹnish time, and we need to produce a subset of these events such that no events intersect each other (that is, having overlapping times), and that we have the maximum number of events scheduled as possible. Here is a formal statement of the problem: Input: events: a set of intervals (si,fi) where si is the start time, and fi is the ïŹnish time. Solution: A subset S of Events. Constraint: No events can intersect (start time exclusive). That is, for all intervals i = (si,fi),j = (sj,fj) where si < sj it holds that fi †sj. Objective: Maximize the number of scheduled events, i.e. maximize the size of the set S. 57
- 61. Greedy Algorithms We ïŹrst begin with a backtracking solution to the problem: // event-schedule -- schedule as many non-conflicting events as possible function event-schedule(events array of s[1..n], j[1..n]): set if n == 0: return â fi if n == 1: return {events[1]} fi let event := events[1] let S1 := union(event-schedule(events - set of conflicting events), event) let S2 := event-schedule(events - {event}) if S1.size() >= S2.size(): return S1 else return S2 fi end The above algorithm will faithfully ïŹnd the largest set of non-conïŹicting events. It brushes aside details of how the set events - set of conïŹicting events is computed, but it would require O(n) time. Because the algorithm makes two recursive calls on itself, each with an argument of size nâ1, and because removing conïŹicts takes linear time, a recurrence for the time this algorithm takes is: T(n) = 2·T(nâ1)+O(n) which is O(2n). But suppose instead of picking just the ïŹrst element in the array we used some other criterion. The aim is to just pick the "right" one so that we wouldn't need two recursive calls. First, let's consider the greedy strategy of picking the shortest events ïŹrst, until we can add no more events without conïŹicts. The idea here is that the shortest events would likely interfere less than other events. There are scenarios were picking the shortest event ïŹrst produces the optimal result. However, here's a scenario where that strategy is sub-optimal: Figure 7 58
- 62. Dijkstra's Shortest Path Algorithm Above, the optimal solution is to pick event A and C, instead of just B alone. Perhaps instead of the shortest event we should pick the events that have the least number of conïŹicts. This strategy seems more direct, but it fails in this scenario: Figure 8 Above, we can maximize the number of events by picking A, B, C, D, and E. However, the events with the least conïŹicts are 6, 2 and 7, 3. But picking one of 6, 2 and one of 7, 3 means that we cannot pick B, C and D, which includes three events instead of just two. 7.2 Dijkstra's Shortest Path Algorithm With two (high-level, pseudocode) transformations, Dijsktra's algorithm can be derived from the much less eïŹcient backtracking algorithm. The trick here is to prove the transformations maintain correctness, but that's the whole insight into Dijkstra's algorithm anyway. [TODO: important to note the paradox that to solve this problem it's easier to solve a more-general version. That is, shortest path from s to all nodes, not just to t. Worthy of its own colored box.] 59
- 63. Greedy Algorithms 7.3 Minimum spanning tree w:Minimum spanning tree1 1 http://en.wikipedia.org/wiki/Minimum%20spanning%20tree 60
- 64. 8 Hill Climbing Hill climbing is a technique for certain classes of optimization problems. The idea is to start with a sub-optimal solution to a problem (i.e., start at the base of a hill) and then repeatedly improve the solution (walk up the hill) until some condition is maximized (the top of the hill is reached). Hill-Climbing Methodology # Construct a sub-optimal solution that meets the constraints of the problem# Take the solution and make an improvement upon it# Repeatedly improve the solution until no more improvements are necessary/possible One of the most popular hill-climbing problems is the network ïŹow problem. Although network ïŹow may sound somewhat speciïŹc it is important because it has high expressive power: for example, many algorithmic problems encountered in practice can actually be considered special cases of network ïŹow. After covering a simple example of the hill-climbing approach for a numerical problem we cover network ïŹow and then present examples of applications of network ïŹow. 61
- 65. Hill Climbing 8.1 Newton's Root Finding Method Figure 9 An illustration of Newton's method: The zero of the f(x) function is at x. We see that the guess xn+1 is a better guess than xn because it is closer to x. (from Wikipediaa) a http://en.wikipedia.org/wiki/Newton%27s%20method Newton's Root Finding Method is a three-centuries-old algorithm for ïŹnding numerical approximations to roots of a function (that is a point x where the function f(x) becomes zero), starting from an initial guess. You need to know the function f(x) and its ïŹrst derivative f (x) for this algorithm. The idea is the following: In the vicinity of the initial guess x0 we can form the Taylor expansion of the function f(x) = f(x0 + )â f(x0)+ f (x0)+ 2 2 f (x0)+... which gives a good approximation to the function near x0. Taking only the ïŹrst two terms on the right hand side, setting them equal to zero, and solving for , we obtain = â f(x0) f (x0) which we can use to construct a better solution x1 = x0 + = x0 â f(x0) f (x0) . 62
- 66. Network Flow This new solution can be the starting point for applying the same procedure again. Thus, in general a better approximation can be constructed by repeatedly applying xn+1 = xn â f(xn) f (xn) . As shown in the illustration, this is nothing else but the construction of the zero from the tangent at the initial guessing point. In general, Newton's root ïŹnding method converges quadratically, except when the ïŹrst derivative of the solution f (x) = 0 vanishes at the root. Coming back to the "Hill climbing" analogy, we could apply Newton's root ïŹnding method not to the function f(x), but to its ïŹrst derivative f (x), that is look for x such that f (x) = 0. This would give the extremal positions of the function, its maxima and minima. Starting Newton's method close enough to a maximum this way, we climb the hill. Instead of regarding continuous functions, the hill-climbing method can also be applied to discrete networks. 8.2 Network Flow Suppose you have a directed graph (possibly with cycles) with one vertex labeled as the source and another vertex labeled as the destination or the "sink". The source vertex only has edges coming out of it, with no edges going into it. Similarly, the destination vertex only has edges going into it, with no edges coming out of it. We can assume that the graph fully connected with no dead-ends; i.e., for every vertex (except the source and the sink), there is at least one edge going into the vertex and one edge going out of it. We assign a "capacity" to each edge, and initially we'll consider only integral-valued capacities. The following graph meets our requirements, where "s" is the source and "t" is the destination: 63
- 67. Hill Climbing Figure 10 We'd like now to imagine that we have some series of inputs arriving at the source that we want to carry on the edges over to the sink. The number of units we can send on an edge at a time must be less than or equal to the edge's capacity. You can think of the vertices as cities and the edges as roads between the cities and we want to send as many cars from the source city to the destination city as possible. The constraint is that we cannot send more cars down a road than its capacity can handle. The goal of network ïŹow is to send as much traïŹc from s to t as each street can bear. To organize the traïŹc routes, we can build a list of diïŹerent paths from city s to city t. Each path has a carrying capacity equal to the smallest capacity value for any edge on the path; for example, consider the following path p: 64
- 68. Network Flow Figure 11 Even though the ïŹnal edge of p has a capacity of 8, that edge only has one car traveling on it because the edge before it only has a capacity of 1 (thus, that edge is at full capacity). After using this path, we can compute the residual graph by subtracting 1 from the capacity of each edge: 65
- 69. Hill Climbing Figure 12 (We subtracted 1 from the capacity of each edge in p because 1 was the carrying capacity of p.) We can say that path p has a ïŹow of 1. Formally, a ïŹow is an assignment f(e) of values to the set of edges in the graph G = (V,E) such that: 1. âe â E : f(e) â R 2. â(u,v) â E : f((u,v)) = âf((v,u)) 3. âu â V,u = s,t : vâV f(u,v) = 0 4. âe â E : f(e) †c(e) Where s is the source node and t is the sink node, and c(e) â„ 0 is the capacity of edge e. We deïŹne the value of a ïŹow f to be: Value(f) = vâV f((s,v)) The goal of network ïŹow is to ïŹnd an f such that Value(f) is maximal. To be maximal means that there is no other ïŹow assignment that obeys the constraints 1-4 that would have a higher value. The traïŹc example can describe what the four ïŹow constraints mean: 1. âe â E : f(e) â R. This rule simply deïŹnes a ïŹow to be a function from edges in the graph to real numbers. The function is deïŹned for every edge in the graph. You could also consider the "function" to simply be a mapping: Every edge can be an index into an array and the value of the array at an edge is the value of the ïŹow function at that edge. 66
- 70. The Ford-Fulkerson Algorithm 2. â(u,v) â E : f((u,v)) = âf((v,u)). This rule says that if there is some traïŹc ïŹowing from node u to node v then there should be considered negative that amount ïŹowing from v to u. For example, if two cars are ïŹowing from city u to city v, then negative two cars are going in the other direction. Similarly, if three cars are going from city u to city v and two cars are going city v to city u then the net eïŹect is the same as if one car was going from city u to city v and no cars are going from city v to city u. 3. âu â V,u = s,t : vâV f(u,v) = 0. This rule says that the net ïŹow (except for the source and the destination) should be neutral. That is, you won't ever have more cars going into a city than you would have coming out of the city. New cars can only come from the source, and cars can only be stored in the destination. Similarly, whatever ïŹows out of s must eventually ïŹow into t. Note that if a city has three cars coming into it, it could send two cars to one city and the remaining car to a diïŹerent city. Also, a city might have cars coming into it from multiple sources (although all are ultimately from city s). 4. âe â E : f(e) †c(e). 8.3 The Ford-Fulkerson Algorithm The following algorithm computes the maximal ïŹow for a given graph with non-negative capacities. What the algorithm does can be easy to understand, but it's non-trivial to show that it terminates and provides an optimal solution. function net-flow(graph (V, E), node s, node t, cost c): flow initialize f(e) := 0 for all e in E loop while not done for all e in E: // compute residual capacities let cf(e) := c(e) - f(e) repeat let Gf := (V, {e : e in E and cf(e) > 0}) find a path p from s to t in Gf // e.g., use depth first search if no path p exists: signal done let path-capacities := map(p, cf) // a path is a set of edges let m := min-val-of(path-capacities) // smallest residual capacity of p for all (u, v) in p: // maintain flow constraints f((u, v)) := f((u, v)) + m f((v, u)) := f((v, u)) - m repeat repeat end 8.4 Applications of Network Flow 1. ïŹnding out maximum bi - partite matching . 2. ïŹnding out min cut of a graph . 67
- 72. 9 Ada Implementation 9.1 Introduction Welcome to the Ada implementations of the Algorithms1 Wikibook. For those who are new to Ada Programming2 a few notes: âą All examples are fully functional with all the needed input and output operations. However, only the code needed to outline the algorithms at hand is copied into the text - the full samples are available via the download links. (Note: It can take up to 48 hours until the cvs is updated). âą We seldom use predeïŹned types in the sample code but deïŹne special types suitable for the algorithms at hand. âą Ada allows for default function parameters; however, we always ïŹll in and name all parameters, so the reader can see which options are available. âą We seldom use shortcuts - like using the attributes Image or Value for String <=> Integer conversions. All these rules make the code more elaborate than perhaps needed. However, we also hope it makes the code easier to understand Category:Ada Programming3 9.2 Chapter 1: Introduction The following subprograms are implementations of the Inventing an Algorithm examples4. 9.2.1 To Lower The Ada example code does not append to the array as the algorithms. Instead we create an empty array of the desired length and then replace the characters inside. File: to_lower_1.adb function To_Lower (C : Character) return Character renames 1 http://en.wikibooks.org/wiki/Algorithms 2 http://en.wikibooks.org/wiki/Ada%20Programming 3 http://en.wikibooks.org/wiki/Category%3AAda%20Programming 4 Chapter 1.3 on page 4 69
- 73. Ada Implementation Ada.Characters.Handling.To_Lower; -- tolower - translates all alphabetic, uppercase characters -- in str to lowercase function To_Lower (Str : String) return String is Result : String (Str'Range); begin for C in Str'Range loop Result (C) := To_Lower (Str (C)); end loop; return Result; end To_Lower; Would the append approach be impossible with Ada? No, but it would be signiïŹcantly more complex and slower. 9.2.2 Equal Ignore Case File: to_lower_2.adb -- equal-ignore-case -- returns true if s or t are equal, -- ignoring case function Equal_Ignore_Case (S : String; T : String) return Boolean is O : constant Integer := S'First - T'First; begin if T'Length /= S'Length then return False; -- if they aren't the same length, they -- aren't equal else for I in S'Range loop if To_Lower (S (I)) /= To_Lower (T (I + O)) then return False; end if; end loop; end if; return True; end Equal_Ignore_Case; 9.3 Chapter 6: Dynamic Programming 9.3.1 Fibonacci numbers The following codes are implementations of the Fibonacci-Numbers examples5. 5 Chapter 6.1 on page 49 70
- 74. Chapter 6: Dynamic Programming Simple Implementation File: ïŹbonacci_1.adb ... To calculate Fibonacci numbers negative values are not needed so we deïŹne an integer type which starts at 0. With the integer type deïŹned you can calculate up until Fib (87). Fib (88) will result in an Constraint_Error. type Integer_Type is range 0 .. 999_999_999_999_999_999; You might notice that there is not equivalence for the assert (n >= 0) from the original example. Ada will test the correctness of the parameter before the function is called. function Fib (n : Integer_Type) return Integer_Type is begin if n = 0 then return 0; elsif n = 1 then return 1; else return Fib (n - 1) + Fib (n - 2); end if; end Fib; ... Cached Implementation File: ïŹbonacci_2.adb ... For this implementation we need a special cache type can also store a -1 as "not calculated" marker type Cache_Type is range -1 .. 999_999_999_999_999_999; The actual type for calculating the ïŹbonacci numbers continues to start at 0. As it is a subtype of the cache type Ada will automatically convert between the two. (the conversion is - of course - checked for validity) subtype Integer_Type is Cache_Type range 0 .. Cache_Type'Last; 71
- 75. Ada Implementation In order to know how large the cache need to be we ïŹrst read the actual value from the command line. Value : constant Integer_Type := Integer_Type'Value (Ada.Command_Line.Argument (1)); The Cache array starts with element 2 since Fib (0) and Fib (1) are constants and ends with the value we want to calculate. type Cache_Array is array (Integer_Type range 2 .. Value) of Cache_Type; The Cache is initialized to the ïŹrst valid value of the cache type â this is -1. F : Cache_Array := (others => Cache_Type'First); What follows is the actual algorithm. function Fib (N : Integer_Type) return Integer_Type is begin if N = 0 or else N = 1 then return N; elsif F (N) /= Cache_Type'First then return F (N); else F (N) := Fib (N - 1) + Fib (N - 2); return F (N); end if; end Fib; ... This implementation is faithful to the original from the Algorithms6 book. However, in Ada you would normally do it a little diïŹerent: File: ïŹbonacci_3.adb when you use a slightly larger array which also stores the elements 0 and 1 and initializes them to the correct values type Cache_Array is array (Integer_Type range 0 .. Value) of Cache_Type; F : Cache_Array := (0 => 0, 1 => 1, others => Cache_Type'First); 6 http://en.wikibooks.org/wiki/Algorithms 72
- 76. Chapter 6: Dynamic Programming and then you can remove the ïŹrst if path. return N; els 73
- 77. Ada Implementation if F (N) /= Cache_Type'First then . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
- 78. Chapter 6: Dynamic Programming This will save about 45% of the execution-time (measured on Linux i686) while needing only two more elements in the cache array. Memory Optimized Implementation This version looks just like the original in WikiCode. File: ïŹbonacci_4.adb type Integer_Type is range 0 .. 999_999_999_999_999_999; function Fib (N : Integer_Type) return Integer_Type is U : Integer_Type := 0; V : Integer_Type := 1; begin for I in 2 .. N loop Calculate_Next : declare T : constant Integer_Type := U + V; begin U := V; V := T; end Calculate_Next; end loop; return V; end Fib; No 64 bit integers Your Ada compiler does not support 64 bit integer numbers? Then you could try to use decimal numbers7 instead. Using decimal numbers results in a slower program (takes about three times as long) but the result will be the same. The following example shows you how to deïŹne a suitable decimal type. Do experiment with the digits and range parameters until you get the optimum out of your Ada compiler. File: fibonacci_5.adb type Integer_Type is delta 1.0 digits 18 range 0.0 .. 999_999_999_999_999_999.0; You should know that ïŹoating point numbers are unsuitable for the calculation of ïŹbonacci numbers. They will not report an error condition when the number calculated becomes too large â instead they will lose in precision which makes the result meaningless. 7 http://en.wikibooks.org/wiki/Ada%20Programming%2FTypes%2Fdelta 75
- 80. 10 Contributors Edits User 16 Adrignola1 3 Andreas Ipp2 5 Avicennasis3 1 ChrisMorrisOrg4 1 ChuckhoïŹmann5 9 Codebrain6 2 DavidCary7 2 Derek Ross8 2 Dirk HĂŒnniger9 3 Dnas10 1 Elaurier11 4 Filburli12 4 Fishpi13 1 Frikk14 1 Fry-kun15 1 Geocachernemesis16 11 Gkhan17 1 Guanabot18 20 Hagindaz19 1 HorsemansWiki20 4 Hwwong21 1 http://en.wikibooks.org/w/index.php?title=User:Adrignola 2 http://en.wikibooks.org/w/index.php?title=User:Andreas_Ipp 3 http://en.wikibooks.org/w/index.php?title=User:Avicennasis 4 http://en.wikibooks.org/w/index.php?title=User:ChrisMorrisOrg 5 http://en.wikibooks.org/w/index.php?title=User:Chuckhoffmann 6 http://en.wikibooks.org/w/index.php?title=User:Codebrain 7 http://en.wikibooks.org/w/index.php?title=User:DavidCary 8 http://en.wikibooks.org/w/index.php?title=User:Derek_Ross 9 http://en.wikibooks.org/w/index.php?title=User:Dirk_H%C3%BCnniger 10 http://en.wikibooks.org/w/index.php?title=User:Dnas 11 http://en.wikibooks.org/w/index.php?title=User:Elaurier 12 http://en.wikibooks.org/w/index.php?title=User:Filburli 13 http://en.wikibooks.org/w/index.php?title=User:Fishpi 14 http://en.wikibooks.org/w/index.php?title=User:Frikk 15 http://en.wikibooks.org/w/index.php?title=User:Fry-kun 16 http://en.wikibooks.org/w/index.php?title=User:Geocachernemesis 17 http://en.wikibooks.org/w/index.php?title=User:Gkhan 18 http://en.wikibooks.org/w/index.php?title=User:Guanabot 19 http://en.wikibooks.org/w/index.php?title=User:Hagindaz 20 http://en.wikibooks.org/w/index.php?title=User:HorsemansWiki 21 http://en.wikibooks.org/w/index.php?title=User:Hwwong 77
- 81. Contributors 2 Intgr22 2 Iwasapenguin23 2 James Dennett24 1 JasonWoof25 2 Jfmantis26 22 Jguk27 1 Jleedev28 1 Jomegat29 2 JustinWick30 8 Jyasskin31 1 K.Rakesh vidya chandra32 1 Kd8cpk33 54 Krischik34 1 Kusti35 1 Liblamb36 2 Lynx772537 1 Mabdul38 3 Mahanga39 4 ManuelGR40 2 Mmartin41 147 Mshonle42 3 Nikai43 1 Panic2k444 2 QuiteUnusual45 2 R3m0t46 22 http://en.wikibooks.org/w/index.php?title=User:Intgr 23 http://en.wikibooks.org/w/index.php?title=User:Iwasapenguin 24 http://en.wikibooks.org/w/index.php?title=User:James_Dennett 25 http://en.wikibooks.org/w/index.php?title=User:JasonWoof 26 http://en.wikibooks.org/w/index.php?title=User:Jfmantis 27 http://en.wikibooks.org/w/index.php?title=User:Jguk 28 http://en.wikibooks.org/w/index.php?title=User:Jleedev 29 http://en.wikibooks.org/w/index.php?title=User:Jomegat 30 http://en.wikibooks.org/w/index.php?title=User:JustinWick 31 http://en.wikibooks.org/w/index.php?title=User:Jyasskin 32 http://en.wikibooks.org/w/index.php?title=User:K.Rakesh_vidya_chandra 33 http://en.wikibooks.org/w/index.php?title=User:Kd8cpk 34 http://en.wikibooks.org/w/index.php?title=User:Krischik 35 http://en.wikibooks.org/w/index.php?title=User:Kusti 36 http://en.wikibooks.org/w/index.php?title=User:Liblamb 37 http://en.wikibooks.org/w/index.php?title=User:Lynx7725 38 http://en.wikibooks.org/w/index.php?title=User:Mabdul 39 http://en.wikibooks.org/w/index.php?title=User:Mahanga 40 http://en.wikibooks.org/w/index.php?title=User:ManuelGR 41 http://en.wikibooks.org/w/index.php?title=User:Mmartin 42 http://en.wikibooks.org/w/index.php?title=User:Mshonle 43 http://en.wikibooks.org/w/index.php?title=User:Nikai 44 http://en.wikibooks.org/w/index.php?title=User:Panic2k4 45 http://en.wikibooks.org/w/index.php?title=User:QuiteUnusual 46 http://en.wikibooks.org/w/index.php?title=User:R3m0t 78
- 82. Chapter 6: Dynamic Programming 2 Recent Runes47 17 Robert Horning48 3 Sartak49 1 Sigma 750 17 Spamduck51 1 SudarshanP52 2 Suruena53 1 Swhalen54 2 Swift55 1 Tcsetattr56 1 Vito Genovese57 12 WhirlWind58 1 Yhh59 2 Ylai60 3 Ylloh61 12 Yuyuchan330162 1 Znetweb63 47 http://en.wikibooks.org/w/index.php?title=User:Recent_Runes 48 http://en.wikibooks.org/w/index.php?title=User:Robert_Horning 49 http://en.wikibooks.org/w/index.php?title=User:Sartak 50 http://en.wikibooks.org/w/index.php?title=User:Sigma_7 51 http://en.wikibooks.org/w/index.php?title=User:Spamduck 52 http://en.wikibooks.org/w/index.php?title=User:SudarshanP 53 http://en.wikibooks.org/w/index.php?title=User:Suruena 54 http://en.wikibooks.org/w/index.php?title=User:Swhalen 55 http://en.wikibooks.org/w/index.php?title=User:Swift 56 http://en.wikibooks.org/w/index.php?title=User:Tcsetattr 57 http://en.wikibooks.org/w/index.php?title=User:Vito_Genovese 58 http://en.wikibooks.org/w/index.php?title=User:WhirlWind 59 http://en.wikibooks.org/w/index.php?title=User:Yhh 60 http://en.wikibooks.org/w/index.php?title=User:Ylai 61 http://en.wikibooks.org/w/index.php?title=User:Ylloh 62 http://en.wikibooks.org/w/index.php?title=User:Yuyuchan3301 63 http://en.wikibooks.org/w/index.php?title=User:Znetweb 79
- 84. List of Figures âą GFDL: Gnu Free Documentation License. http://www.gnu.org/licenses/fdl.html âą cc-by-sa-3.0: Creative Commons Attribution ShareAlike 3.0 License. http:// creativecommons.org/licenses/by-sa/3.0/ âą cc-by-sa-2.5: Creative Commons Attribution ShareAlike 2.5 License. http:// creativecommons.org/licenses/by-sa/2.5/ âą cc-by-sa-2.0: Creative Commons Attribution ShareAlike 2.0 License. http:// creativecommons.org/licenses/by-sa/2.0/ âą cc-by-sa-1.0: Creative Commons Attribution ShareAlike 1.0 License. http:// creativecommons.org/licenses/by-sa/1.0/ âą cc-by-2.0: Creative Commons Attribution 2.0 License. http://creativecommons. org/licenses/by/2.0/ âą cc-by-2.0: Creative Commons Attribution 2.0 License. http://creativecommons. org/licenses/by/2.0/deed.en âą cc-by-2.5: Creative Commons Attribution 2.5 License. http://creativecommons. org/licenses/by/2.5/deed.en âą cc-by-3.0: Creative Commons Attribution 3.0 License. http://creativecommons. org/licenses/by/3.0/deed.en âą GPL: GNU General Public License. http://www.gnu.org/licenses/gpl-2.0.txt âą LGPL: GNU Lesser General Public License. http://www.gnu.org/licenses/lgpl. html âą PD: This image is in the public domain. âą ATTR: The copyright holder of this ïŹle allows anyone to use it for any purpose, provided that the copyright holder is properly attributed. Redistribution, derivative work, commercial use, and all other use is permitted. âą EURO: This is the common (reverse) face of a euro coin. The copyright on the design of the common face of the euro coins belongs to the European Commission. Authorised is reproduction in a format without relief (drawings, paintings, ïŹlms) provided they are not detrimental to the image of the euro. âą LFK: Lizenz Freie Kunst. http://artlibre.org/licence/lal/de âą CFR: Copyright free use. 81
- 85. List of Figures âą EPL: Eclipse Public License. http://www.eclipse.org/org/documents/epl-v10. php Copies of the GPL, the LGPL as well as a GFDL are included in chapter Licenses64. Please note that images in the public domain do not require attribution. You may click on the image numbers in the following table to open the webpage of the images in your webbrower. 64 Chapter 11 on page 85 82
- 86. List of Figures 1 GFDL 2 GFDL 3 GFDL 4 GFDL 5 GFDL 6 GFDL 7 GFDL 8 GFDL 9 Original uploader was Olegalexandrov65 at en.wikipedia66 PD 10 GFDL 11 GFDL 12 GFDL 65 http://en.wikibooks.org/wiki/%3Aen%3AUser%3AOlegalexandrov 66 http://en.wikipedia.org 83
- 88. 11 Licenses 11.1 GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright © 2007 Free Software Foundation, Inc. <http://fsf.org/> Everyone is permitted to copy and distribute verba- tim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practi- cal works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guaran- tee your freedom to share and change all versions of a programâto make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Li- censes are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to sur- render the rights. Therefore, you have certain re- sponsibilities if you distribute copies of the soft- ware, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a pro- gram, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you re- ceived. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) oïŹer you this License giving you legal permission to copy, distribute and/or modify it. For the developersâ and authorsâ protection, the GPL clearly explains that there is no warranty for this free software. For both usersâ and authorsâ sake, the GPL requires that modiïŹed versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modiïŹed versions of the software in- side them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting usersâ freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to pro- hibit the practice for those products. If such prob- lems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it eïŹectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non- free. The precise terms and conditions for copying, dis- tribution and modiïŹcation follow. TERMS AND CONDITIONS 0. DeïŹnitions. âThis Licenseâ refers to version 3 of the GNU Gen- eral Public License. âCopyrightâ also means copyright-like laws that ap- ply to other kinds of works, such as semiconductor masks. âThe Programâ refers to any copyrightable work licensed under this License. Each licensee is ad- dressed as âyouâ. âLicenseesâ and ârecipientsâ may be individuals or organizations. To âmodifyâ a work means to copy from or adapt all or part of the work in a fashion requiring copy- right permission, other than the making of an exact copy. The resulting work is called a âmodiïŹed ver- sionâ of the earlier work or a work âbased onâ the earlier work. A âcovered workâ means either the unmodiïŹed Pro- gram or a work based on the Program. To âpropagateâ a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under appli- cable copyright law, except executing it on a com- puter or modifying a private copy. Propagation in- cludes copying, distribution (with or without mod- iïŹcation), making available to the public, and in some countries other activities as well. To âconveyâ a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not convey- ing. An interactive user interface displays âAppropriate Legal Noticesâ to the extent that it includes a con- venient and prominently visible feature that (1) dis- plays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (ex- cept to the extent that warranties are provided), that licensees may convey the work under this Li- cense, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The âsource codeâ for a work means the preferred form of the work for making modiïŹcations to it. âObject codeâ means any non-source form of a work. A âStandard Interfaceâ means an interface that ei- ther is an oïŹcial standard deïŹned by a recognized standards body, or, in the case of interfaces spec- iïŹed for a particular programming language, one that is widely used among developers working in that language. The âSystem Librariesâ of an executable work in- clude anything, other than the work as a whole, that (a) is included in the normal form of packag- ing a Major Component, but which is not part of that Major Component, and (b) serves only to en- able use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A âMajor Componentâ, in this context, means a major essential component (kernel, window system, and so on) of the speciïŹc operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The âCorresponding Sourceâ for a work in object code form means all the source code needed to gen- erate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the workâs System Libraries, or general- purpose tools or generally available free programs which are used unmodiïŹed in performing those ac- tivities but which are not part of the work. For example, Corresponding Source includes interface deïŹnition ïŹles associated with source ïŹles for the work, and the source code for shared libraries and dynamically linked subprograms that the work is speciïŹcally designed to require, such as by intimate data communication or control ïŹow between those subprograms and other parts of the work. The Corresponding Source need not include any- thing that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly aïŹrms your unlimited per- mission to run the unmodiïŹed Program. The out- put from running a covered work is covered by this License only if the output, given its content, con- stitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as pro- vided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modiïŹcations exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclu- sively on your behalf, under your direction and con- trol, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permit- ted solely under the conditions stated below. Subli- censing is not allowed; section 10 makes it unneces- sary. 3. Protecting Usersâ Legal Rights From Anti- Circumvention Law. No covered work shall be deemed part of an eïŹec- tive technological measure under any applicable law fulïŹlling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumven- tion of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technologi- cal measures to the extent such circumvention is ef- fected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modiïŹcation of the work as a means of enforcing, against the workâs users, your or third partiesâ legal rights to forbid circumvention of technological measures. 4. Con- veying Verbatim Copies. You may convey verbatim copies of the Programâs source code as you receive it, in any medium, pro- vided that you conspicuously and appropriately publish on each copy an appropriate copyright no- tice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipi- ents a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may oïŹer support or war- ranty protection for a fee. 5. Conveying ModiïŹed Source Versions. You may convey a work based on the Program, or the modiïŹcations to produce it from the Program, in the form of source code under the terms of sec- tion 4, provided that you also meet all of these con- ditions: * a) The work must carry prominent notices stating that you modiïŹed it, and giving a relevant date. * b) The work must carry prominent notices stating that it is released under this License and any con- ditions added under section 7. This requirement modiïŹes the requirement in section 4 to âkeep in- tact all noticesâ. * c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such per- mission if you have separately received it. * d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not dis- play Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other sepa- rate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distri- bution medium, is called an âaggregateâ if the com- pilation and its resulting copyright are not used to limit the access or legal rights of the compilationâs users beyond what the individual works permit. In- clusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Correspond- ing Source under the terms of this License, in one of these ways: * a) Convey the object code in, or embodied in, a physical product (including a physical distribu- tion medium), accompanied by the Corresponding Source ïŹxed on a durable physical medium custom- arily used for software interchange. * b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accom- panied by a written oïŹer, valid for at least three years and valid for as long as you oïŹer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this Li- cense, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. * c) Convey individual copies of the object code with a copy of the written oïŹer to provide the Corresponding Source. This alternative is al- lowed only occasionally and noncommercially, and only if you received the object code with such an of- fer, in accord with subsection 6b. * d) Convey the object code by oïŹering access from a designated place (gratis or for a charge), and oïŹer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Correspond- ing Source along with the object code. If the place to copy the object code is a network server, the Cor- responding Source may be on a diïŹerent server (op- erated by you or a third party) that supports equiv- alent copying facilities, provided you maintain clear directions next to the object code saying where to ïŹnd the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. * e) Con- vey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are be- ing oïŹered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A âUser Productâ is either (1) a âconsumer prod- uctâ, which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In deter- mining whether a product is a consumer product, doubtful cases shall be resolved in favor of cover- age. For a particular product received by a par- ticular user, ânormally usedâ refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the prod- uct has substantial commercial, industrial or non- consumer uses, unless such uses represent the only signiïŹcant mode of use of the product. âInstallation Informationâ for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modiïŹed versions of a covered work in that User Product from a modiïŹed version of its Corre- sponding Source. The information must suïŹce to ensure that the continued functioning of the modi- ïŹed object code is in no case prevented or interfered with solely because modiïŹcation has been made. If you convey an object code work under this sec- tion in, or with, or speciïŹcally for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the re- cipient in perpetuity or for a ïŹxed term (regard- less of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Informa- tion. But this requirement does not apply if neither you nor any third party retains the ability to install modiïŹed object code on the User Product (for ex- ample, the work has been installed in ROM). The requirement to provide Installation Informa- tion does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modiïŹed or installed by the re- cipient, or for the User Product in which it has been modiïŹed or installed. Access to a network may be denied when the modiïŹcation itself materially and adversely aïŹects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. âAdditional permissionsâ are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permis- sions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid un- der applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own re- moval in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this Li- cense, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: * a) Disclaiming warranty or limiting liability dif- ferently from the terms of sections 15 and 16 of this License; or * b) Requiring preservation of speciïŹed reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or * c) Prohibit- ing misrepresentation of the origin of that material, or requiring that modiïŹed versions of such material be marked in reasonable ways as diïŹerent from the original version; or * d) Limiting the use for pub- licity purposes of names of licensors or authors of the material; or * e) Declining to grant rights under trademark law for use of some trade names, trade- marks, or service marks; or * f) Requiring indem- niïŹcation of licensors and authors of that material by anyone who conveys the material (or modiïŹed versions of it) with contractual assumptions of lia- bility to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are con- sidered âfurther restrictionsâ within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is gov- erned by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source ïŹles, a statement of the additional terms that ap- ply to those ïŹles, or a notice indicating where to ïŹnd the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above require- ments apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and ïŹnally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessa- tion. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notiïŹes you of the violation by some reason- able means, this is the ïŹrst time you have received notice of violation of this License (for any work)
- 89. from that copyright holder, and you cure the vi- olation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have re- ceived copies or rights from you under this License. If your rights have been terminated and not perma- nently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in or- der to receive or run a copy of the Program. Ancil- lary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require accep- tance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by mod- ifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Auto- matic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An âentity transactionâ is a transaction transfer- ring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a cov- ered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the partyâs predecessor in interest had or could give un- der the previous paragraph, plus a right to posses- sion of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable eïŹorts. You may not impose any further restrictions on the exercise of the rights granted or aïŹrmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not ini- tiate litigation (including a cross-claim or counter- claim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, oïŹering for sale, or importing the Program or any portion of it. 11. Patents. A âcontributorâ is a copyright holder who autho- rizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributorâs âcontribu- tor versionâ. A contributorâs âessential patent claimsâ are all patent claims owned or controlled by the contribu- tor, whether already acquired or hereafter acquired, that would be infringed by some manner, permit- ted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modiïŹcation of the contributor version. For pur- poses of this deïŹnition, âcontrolâ includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, world- wide, royalty-free patent license under the contrib- utorâs essential patent claims, to make, use, sell, of- fer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a âpatent li- censeâ is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To âgrantâ such a patent license to a party means to make such an agreement or commitment not to en- force a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the beneïŹt of the patent license for this particular work, or (3) arrange, in a manner consistent with the re- quirements of this License, to extend the patent license to downstream recipients. âKnowingly re- lyingâ means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipientâs use of the cov- ered work in a country, would infringe one or more identiïŹable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single trans- action or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties re- ceiving the covered work authorizing them to use, propagate, modify or convey a speciïŹc copy of the covered work, then the patent license you grant is automatically extended to all recipients of the cov- ered work and works based on it. A patent license is âdiscriminatoryâ if it does not in- clude within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are speciïŹcally granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of dis- tributing software, under which you make payment to the third party based on the extent of your ac- tivity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discrimina- tory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with speciïŹc products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as ex- cluding or limiting any implied license or other de- fenses to infringement that may otherwise be avail- able to you under applicable patent law. 12. No Surrender of Othersâ Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultane- ously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a roy- alty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to re- frain entirely from conveying the Program. 13. Use with the GNU AïŹero General Public License. Notwithstanding any other provision of this Li- cense, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU AïŹero General Public License into a single combined work, and to convey the result- ing work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU AïŹero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public Li- cense from time to time. Such new versions will be similar in spirit to the present version, but may dif- fer in detail to address new problems or concerns. Each version is given a distinguishing version num- ber. If the Program speciïŹes that a certain num- bered version of the GNU General Public License âor any later versionâ applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foun- dation. If the Program speciïŹes that a proxy can decide which future versions of the GNU General Public License can be used, that proxyâs public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or diïŹerent permissions. However, no additional obli- gations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PRO- GRAM, TO THE EXTENT PERMITTED BY AP- PLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLD- ERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM âAS ISâ WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IM- PLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFEC- TIVE, YOU ASSUME THE COST OF ALL NECES- SARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLI- CABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CON- VEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, IN- CLUDING ANY GENERAL, SPECIAL, INCIDEN- TAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIM- ITED TO LOSS OF DATA OR DATA BEING REN- DERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. In- terpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of lia- bility provided above cannot be given local legal ef- fect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Ap- ply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the pro- gram. It is safest to attach them to the start of each source ïŹle to most eïŹectively state the exclu- sion of warranty; and each ïŹle should have at least the âcopyrightâ line and a pointer to where the full notice is found. <one line to give the programâs name and a brief idea of what it does.> Copyright (C) <year> <name of author> This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the Li- cense, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WAR- RANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PAR- TICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU Gen- eral Public License along with this program. If not, see <http://www.gnu.org/licenses/>. Also add information on how to contact you by elec- tronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: <program> Copyright (C) <year> <name of au- thor> This program comes with ABSOLUTELY NO WARRANTY; for details type âshow wâ. This is free software, and you are welcome to redistribute it under certain conditions; type âshow câ for details. The hypothetical commands âshow wâ and âshow câ should show the appropriate parts of the General Public License. Of course, your programâs com- mands might be diïŹerent; for a GUI interface, you would use an âabout boxâ. You should also get your employer (if you work as a programmer) or school, if any, to sign a âcopyright disclaimerâ for the program, if nec- essary. For more information on this, and how to apply and follow the GNU GPL, see <http://www.gnu.org/licenses/>. The GNU General Public License does not permit incorporating your program into proprietary pro- grams. If your program is a subroutine library, you may consider it more useful to permit linking pro- prietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But ïŹrst, please read <http://www.gnu.org/philosophy/why- not-lgpl.html>. 11.2 GNU Free Documentation License Version 1.3, 3 November 2008 Copyright © 2000, 2001, 2002, 2007, 2008 Free Soft- ware Foundation, Inc. <http://fsf.org/> Everyone is permitted to copy and distribute verba- tim copies of this license document, but changing it is not allowed. 0. PREAMBLE The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the eïŹective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License pre- serves for the author and publisher a way to get credit for their work, while not being considered responsible for modiïŹcations made by others. This License is a kind of "copyleft", which means that derivative works of the document must them- selves be free in the same sense. It complements the GNU General Public License, which is a copy- left license designed for free software. We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not lim- ited to software manuals; it can be used for any tex- tual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference. 1. APPLICABILITY AND DEFINITIONS This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in dura- tion, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a li- censee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law. A "ModiïŹed Version" of the Document means any work containing the Document or a portion of it, ei- ther copied verbatim, or with modiïŹcations and/or translated into another language. A "Secondary Section" is a named appendix or a front-matter section of the Document that deals ex- clusively with the relationship of the publishers or authors of the Document to the Documentâs overall subject (or to related matters) and contains noth- ing that could fall directly within that overall sub- ject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not ex- plain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regard- ing them. The "Invariant Sections" are certain Secondary Sec- tions whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a sec- tion does not ïŹt the above deïŹnition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none. The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words. A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose speciïŹcation is available to the general pub- lic, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing ed- itor, and that is suitable for input to text format- ters or for automatic translation to a variety of for- mats suitable for input to text formatters. A copy made in an otherwise Transparent ïŹle format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modiïŹcation by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque". Examples of suitable formats for Transparent copies include plain ASCII without markup, Tex- info input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard- conforming simple HTML, PostScript or PDF de- signed for human modiïŹcation. Examples of trans- parent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only. The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most promi- nent appearance of the workâs title, preceding the beginning of the body of the text. The "publisher" means any person or entity that distributes copies of the Document to the public. A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a speciïŹc section name mentioned below, such as "Acknowledgements", "Dedications", "En- dorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" ac- cording to this deïŹnition. The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Dis- claimers are considered to be included by reference in this License, but only as regards disclaiming war- ranties: any other implication that these Warranty Disclaimers may have is void and has no eïŹect on the meaning of this License. 2. VERBATIM COPY- ING You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical mea- sures to obstruct or control the reading or further copying of the copies you make or distribute. How- ever, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in sec- tion 3. You may also lend copies, under the same condi- tions stated above, and you may publicly display copies. 3. COPYING IN QUANTITY If you publish printed copies (or copies in media that commonly have printed covers) of the Doc- ument, numbering more than 100, and the Doc- umentâs license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Doc- ument and satisfy these conditions, can be treated as verbatim copying in other respects. If the required texts for either cover are too volu- minous to ïŹt legibly, you should put the ïŹrst ones listed (as many as ïŹt reasonably) on the actual cover, and continue the rest onto adjacent pages. If you publish or distribute Opaque copies of the Document numbering more than 100, you must ei- ther include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Doc- ument, free of added material. If you use the lat- ter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public. It is requested, but not required, that you con- tact the authors of the Document well before redis- tributing any large number of copies, to give them a chance to provide you with an updated version of the Document. 4. MODIFICATIONS You may copy and distribute a ModiïŹed Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modi- ïŹed Version under precisely this License, with the ModiïŹed Version ïŹlling the role of the Document, thus licensing distribution and modiïŹcation of the ModiïŹed Version to whoever possesses a copy of it. In addition, you must do these things in the Modi- ïŹed Version: * A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission. * B. List on the Title
- 90. Page, as authors, one or more persons or entities responsible for authorship of the modiïŹcations in the ModiïŹed Version, together with at least ïŹve of the principal authors of the Document (all of its principal authors, if it has fewer than ïŹve), unless they release you from this requirement. * C. State on the Title page the name of the publisher of the ModiïŹed Version, as the publisher. * D. Preserve all the copyright notices of the Document. * E. Add an appropriate copyright notice for your modiïŹca- tions adjacent to the other copyright notices. * F. Include, immediately after the copyright notices, a license notice giving the public permission to use the ModiïŹed Version under the terms of this Li- cense, in the form shown in the Addendum below. * G. Preserve in that license notice the full lists of In- variant Sections and required Cover Texts given in the Documentâs license notice. * H. Include an unal- tered copy of this License. * I. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the ModiïŹed Version as given on the Title Page. If there is no section Entitled "His- tory" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describ- ing the ModiïŹed Version as stated in the previous sentence. * J. Preserve the network location, if any, given in the Document for public access to a Trans- parent copy of the Document, and likewise the net- work locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network lo- cation for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission. * K. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein. * L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles. * M. Delete any section Entitled "Endorse- ments". Such a section may not be included in the ModiïŹed Version. * N. Do not retitle any existing section to be Entitled "Endorsements" or to conïŹict in title with any Invariant Section. * O. Preserve any Warranty Disclaimers. If the ModiïŹed Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the ModiïŹed Versionâs license notice. These titles must be distinct from any other section titles. You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your ModiïŹed Version by various partiesâfor ex- ample, statements of peer review or that the text has been approved by an organization as the au- thoritative deïŹnition of a standard. You may add a passage of up to ïŹve words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the ModiïŹed Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add an- other; but you may replace the old one, on explicit permission from the previous publisher that added the old one. The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply en- dorsement of any ModiïŹed Version. 5. COMBIN- ING DOCUMENTS You may combine the Document with other docu- ments released under this License, under the terms deïŹned in section 4 above for modiïŹed versions, provided that you include in the combination all of the Invariant Sections of all of the original doc- uments, unmodiïŹed, and list them all as Invariant Sections of your combined work in its license no- tice, and that you preserve all their Warranty Dis- claimers. The combined work need only contain one copy of this License, and multiple identical Invariant Sec- tions may be replaced with a single copy. If there are multiple Invariant Sections with the same name but diïŹerent contents, make the title of each such section unique by adding at the end of it, in paren- theses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work. In the combination, you must combine any sections Entitled "History" in the various original docu- ments, forming one section Entitled "History"; like- wise combine any sections Entitled "Acknowledge- ments", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorse- ments". 6. COLLECTIONS OF DOCUMENTS You may make a collection consisting of the Docu- ment and other documents released under this Li- cense, and replace the individual copies of this Li- cense in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects. You may extract a single document from such a col- lection, and distribute it individually under this Li- cense, provided you insert a copy of this License into the extracted document, and follow this Li- cense in all other respects regarding verbatim copy- ing of that document. 7. AGGREGATION WITH INDEPENDENT WORKS A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright re- sulting from the compilation is not used to limit the legal rights of the compilationâs users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is appli- cable to these copies of the Document, then if the Document is less than one half of the entire aggre- gate, the Documentâs Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate. 8. TRANSLATION Translation is considered a kind of modiïŹcation, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permis- sion from their copyright holders, but you may in- clude translations of some or all Invariant Sections in addition to the original versions of these Invari- ant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this Li- cense and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail. If a section in the Document is Entitled "Acknowl- edgements", "Dedications", or "History", the re- quirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title. 9. TERMINATION You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, mod- ify, sublicense, or distribute it is void, and will automatically terminate your rights under this Li- cense. However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and ïŹnally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessa- tion. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notiïŹes you of the violation by some reason- able means, this is the ïŹrst time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the vi- olation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have re- ceived copies or rights from you under this License. If your rights have been terminated and not perma- nently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it. 10. FUTURE REVISIONS OF THIS LI- CENSE The Free Software Foundation may publish new, re- vised versions of the GNU Free Documentation Li- cense from time to time. Such new versions will be similar in spirit to the present version, but may dif- fer in detail to address new problems or concerns. See http://www.gnu.org/copyleft/. Each version of the License is given a distinguish- ing version number. If the Document speciïŹes that a particular numbered version of this License "or any later version" applies to it, you have the op- tion of following the terms and conditions either of that speciïŹed version or of any later version that has been published (not as a draft) by the Free Soft- ware Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document speci- ïŹes that a proxy can decide which future versions of this License can be used, that proxyâs public state- ment of acceptance of a version permanently autho- rizes you to choose that version for the Document. 11. RELICENSING "Massive Multiauthor Collaboration Site" (or "MMC Site") means any World Wide Web server that publishes copyrightable works and also pro- vides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such a server. A "Massive Multiau- thor Collaboration" (or "MMC") contained in the site means any set of copyrightable works thus pub- lished on the MMC site. "CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-proïŹt corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization. "Incorporate" means to publish or republish a Doc- ument, in whole or in part, as part of another Doc- ument. An MMC is "eligible for relicensing" if it is licensed under this License, and if all works that were ïŹrst published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008. The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, pro- vided the MMC is eligible for relicensing. ADDEN- DUM: How to use this License for your documents To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page: Copyright (C) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this doc- ument under the terms of the GNU Free Documen- tation License, Version 1.3 or any later version pub- lished by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the "with . . . Texts." line with this: with the Invariant Sections being LIST THEIR TI- TLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation. If your document contains nontrivial examples of program code, we recommend releasing these exam- ples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software. 11.3 GNU Lesser General Public License GNU LESSER GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright © 2007 Free Software Foundation, Inc. <http://fsf.org/> Everyone is permitted to copy and distribute verba- tim copies of this license document, but changing it is not allowed. This version of the GNU Lesser General Public Li- cense incorporates the terms and conditions of ver- sion 3 of the GNU General Public License, supple- mented by the additional permissions listed below. 0. Additional DeïŹnitions. As used herein, âthis Licenseâ refers to version 3 of the GNU Lesser General Public License, and the âGNU GPLâ refers to version 3 of the GNU General Public License. âThe Libraryâ refers to a covered work governed by this License, other than an Application or a Com- bined Work as deïŹned below. An âApplicationâ is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. DeïŹning a subclass of a class deïŹned by the Library is deemed a mode of using an interface provided by the Library. A âCombined Workâ is a work produced by com- bining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the âLinked Versionâ. The âMinimal Corresponding Sourceâ for a Com- bined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version. The âCorresponding Application Codeâ for a Com- bined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Com- bined Work from the Application, but excluding the System Libraries of the Combined Work. 1. Excep- tion to Section 3 of the GNU GPL. You may convey a covered work under sections 3 and 4 of this License without being bound by sec- tion 3 of the GNU GPL. 2. Conveying ModiïŹed Versions. If you modify a copy of the Library, and, in your modiïŹcations, a facility refers to a function or data to be supplied by an Application that uses the fa- cility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modiïŹed version: * a) under this License, provided that you make a good faith eïŹort to ensure that, in the event an Ap- plication does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or * b) under the GNU GPL, with none of the additional permis- sions of this License applicable to that copy. 3. Object Code Incorporating Material from Li- brary Header Files. The object code form of an Application may incor- porate material from a header ïŹle that is part of the Library. You may convey such object code un- der terms of your choice, provided that, if the in- corporated material is not limited to numerical pa- rameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the follow- ing: * a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License. * b) Accompany the object code with a copy of the GNU GPL and this license document. 4. Combined Works. You may convey a Combined Work under terms of your choice that, taken together, eïŹectively do not restrict modiïŹcation of the portions of the Library contained in the Combined Work and reverse en- gineering for debugging such modiïŹcations, if you also do each of the following: * a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this Li- cense. * b) Accompany the Combined Work with a copy of the GNU GPL and this license document. * c) For a Combined Work that displays copyright no- tices during execution, include the copyright notice for the Library among these notices, as well as a ref- erence directing the user to the copies of the GNU GPL and this license document. * d) Do one of the following: o 0) Convey the Minimal Corresponding Source under the terms of this License, and the Cor- responding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modiïŹed version of the Linked Version to produce a modiïŹed Com- bined Work, in the manner speciïŹed by section 6 of the GNU GPL for conveying Corresponding Source. o 1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Li- brary already present on the userâs computer sys- tem, and (b) will operate properly with a modiïŹed version of the Library that is interface-compatible with the Linked Version. * e) Provide Installation Information, but only if you would otherwise be re- quired to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modiïŹed version of the Combined Work produced by recombining or relinking the Application with a modiïŹed version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner speciïŹed by section 6 of the GNU GPL for conveying Corresponding Source.) 5. Combined Libraries. You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following: * a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License. * b) Give prominent no- tice with the combined library that part of it is a work based on the Library, and explaining where to ïŹnd the accompanying uncombined form of the same work. 6. Revised Versions of the GNU Lesser General Public License. The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new ver- sions will be similar in spirit to the present version, but may diïŹer in detail to address new problems or concerns. Each version is given a distinguishing version num- ber. If the Library as you received it speciïŹes that a certain numbered version of the GNU Lesser Gen- eral Public License âor any later versionâ applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foun- dation. If the Library as you received it does not specify a version number of the GNU Lesser Gen- eral Public License, you may choose any version of the GNU Lesser General Public License ever pub- lished by the Free Software Foundation. If the Library as you received it speciïŹes that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxyâs public statement of acceptance of any version is permanent authorization for you to choose that version for the Library.