Slides [DAA] Unit 2 Ch 2.pdf
- 1. Chapter-2 Fundamentals of the Analysis of Algorithm Efficiency
- 2. Introduction • Analysis of algorithms” is usually used in a narrower, technical sense to mean an investigation of an algorithm’s efficiency with respect to two resources: 1. Running time 2. Memory space. • Unlike simplicity and generality, efficiency can be studied in precise quantitative terms. • The efficiency considerations are of primary importance from a practical point of view
- 3. The Analysis Framework • we outline a general framework for analyzing the efficiency of algorithms. • Time efficiency, also called time complexity, (indicates how fast an algorithm in question runs). • Space efficiency, also called space complexity, refers to the extra amount of memory units required by the algorithm. • We primarily concentrate on time efficiency, but the analytical framework introduced here is applicable to analyzing space efficiency as well
- 4. The Analysis Framework • Measuring an Input’s Size • Units for Measuring Running Time • Orders of Growth • Worst-Case, Best-Case, and Average-Case Efficiencies
- 5. Measuring an Input’s Size • First we observe that , it is logical to investigate an algorithm’s efficiency as a function of some parameter n indicating the algorithm’s input size. • In most cases, selecting such a parameter is quite straightforward. For example, problems of sorting, searching, finding the list’s smallest element. • For the problem of evaluating a polynomial it will be the polynomial’s degree or the number of its coefficients, which is larger by 1 than its degree. (this small difference does not matter on the efficiency analysis)
- 6. Measuring an Input’s Size • However , sometimes the choice of a parameter indicating an input size does matter. • One such example is computing the product of two n × n matrices. • There are two natural choices : matrix order n and total number of elements N in the matrices. • Here , the answer about an algorithm’s efficiency will be qualitatively different depending on which of these two choices we use
- 7. Measuring an Input’s Size • The choice of an appropriate size for n can be influenced by operations of the algorithm in question. • Example: Spell-checking algorithm • If the algorithm examines individual characters of its input, we should measure the size by the number of characters. • if it works by processing words, we should measure the size by the number of words.
- 8. Measuring an Input’s Size • Finally, we should make a special note about measuring input size for algorithms solving problems such as checking primality of a positive integer n. • Here, the input is just one number, and it is this number’s magnitude that determines the input size. • In such situations, it is preferable to measure size by the number b of bits in the n’s binary representation:
- 9. Units for Measuring Running Time • Which units for measuring an algorithm’s running time ? • There are obvious drawbacks if we use a second, or millisecond, and so on for measuring run time because of the following. 1. Dependence on the speed of a particular computer. 2. Dependence on the quality of a program implementing the algorithm. 3. Dependence on the quality of the compiler used in generating the machine code. 4. Difficulty of clocking the actual running time of the program.
- 10. Units for Measuring Running Time • Therefore, we would like to have a metric (measurement) that does not depend on these extraneous factors • One possible approach is to count the number of times the basic operation is executed. • The basic operation is the operation contributing the most to the total running time. • It is not difficult to identify the basic operation of an algorithm: it is usually the most time-consuming operation in the algorithm’s innermost loop.
- 11. Units for Measuring Running Time • For example, most sorting algorithms work by comparing elements. Here the basic operation is a comparison operation. • As another example, an operation that involves some or all of the four arithmetical operations: addition, subtraction, multiplication, and division. • Of the four, the most time-consuming operation is division, followed by multiplication and then addition and subtraction.
- 12. Units for Measuring Running Time • Thus, the established framework for the analysis of an algorithm’s is to count the number of times the basic operation is executed. • Let be the execution time of an algorithm’s basic operation on a particular computer, • Let C(n) be the number of times this operation needs to be executed for this algorithm. • Then we can estimate the running time T (n) of a program implementing this algorithm, with the formula,
- 13. Units for Measuring Running Time • We should be careful that the count C(n) does not contain any information about operations that are not basic. • Also we note that, and C(n) are often measured approximately. • Still, unless n (input size) is extremely large or very small, the formula can give a reasonable estimate often algorithm’s running time.
- 14. Units for Measuring Running Time • The formula also makes it possible to answer several questions. • For example: Assuming that • How much longer will the algorithm run if we double its input size? • The answer is about four times longer.
- 15. Orders of Growth • For large-size inputs, we concentrate on Orders of Growth to within a constant multiple. • Why? Because , small inputs no difference in running times for different algorithm. • For example the three approaches for finding GCD(m, n) have the same efficiency for small input numbers. • Thus, for large values of n (input size), it is the function’s order of growth that counts.
- 16. Orders of Growth • Following table contains values of a few functions particularly important for analysis of algorithms. Values (some approximate) of several functions important for analysis of algorithms
- 17. Orders of Growth • We observe from the table that the function growing the slowest among these is the logarithmic function. • Thus , we should expect a program implementing an algorithm with a logarithmic basic-operation count to run practically instantaneously on any input size. • On the other hand the exponential function and the factorial function n!, both these functions grow so fast that their values become enormously large even for rather small values of n.
- 18. Orders of Growth • There is a tremendous difference between the orders of growth of the functions and n!, yet both are often referred to as “exponential-growth functions” . • Algorithms that require an exponential number of operations are practical for solving only problems of very small sizes.
- 19. Worst-Case, Best-Case, and Average-Case Efficiencies • There are many algorithms for which running time depends not only on an input size but also on the specifics of a particular input. • Consider, as an example, sequential search.
- 20. Worst-Case, Best-Case, and Average-Case Efficiencies • In the worst case, when there are no matching elements or the first matching element happens to be the last one on the list, • In this case, the algorithm makes the largest number of key comparisons among all possible inputs of size n: • The worst-case efficiency of an algorithm is its efficiency for the worst-case input of size n, which is an input of size n for which the algorithm runs the longest among all possible inputs of that size.
- 21. Worst-Case, Best-Case, and Average-Case Efficiencies • To determine the worst-case , see what kind of inputs yield the largest value of the basic operation’s count C(n) and then compute the worst- case value. • Worst case t guarantees that for any instance of size n, the running time will not exceed
- 22. Worst-Case, Best-Case, and Average-Case Efficiencies • The best-case efficiency of an algorithm is its efficiency for the best-case input of size n, which is an input (or inputs) of size n for which the algorithm runs the fastest among all possible inputs of that size. • To find the best-case efficiency, determine the kind of inputs for which the count C(n) will be the smallest among all possible inputs of size n. • For example, the best-case inputs for sequential search are lists of size n with their first element equal to a search key; accordingly, for this algorithm.
- 23. Worst-Case, Best-Case, and Average-Case Efficiencies • The analysis of the best-case efficiency is not nearly as important as that of the worst-case efficiency. • But best-case efficiency is important for some algorithms • For example, there is a sorting algorithm (insertion sort) for which the best-case inputs are already sorted array. • Moreover, the best-case efficiency deteriorates only slightly for almost-sorted arrays. Therefore, such an algorithm might well be the method of choice for applications dealing with almost-sorted arrays.
- 24. Worst-Case, Best-Case, and Average-Case Efficiencies • The average-case efficiency seeks to provide the necessary information about an algorithm’s behavior on a “typical” or “random” input. • To analyze the algorithm’s average-case efficiency, we must make some assumptions about possible inputs of size n. • To determine average case for sequential search as follows.
- 25. Worst-Case, Best-Case, and Average-Case Efficiencies • The standard assumptions for sequential search are 1. The probability of a successful search is p 2. The probability of the first match occurring in the ith position of the list is the same for every i. • In the case of a successful search, the probability of the first match occurring in the ith position of the list is p/n every i. • In the case of an unsuccessful search, probability of such a search being (1− p).
- 26. Worst-Case, Best-Case, and Average-Case Efficiencies Therefore,
- 27. Worst-Case, Best-Case, and Average-Case Efficiencies • For example, if p = 1 (the search must be successful), the average number of key comparisons made by sequential search is (n + 1)/2; • If p = 0 (the search must be unsuccessful), the average number of key comparisons will be n • We observe that investigation of the average-case efficiency is considerably more difficult than investigation of the worst-case and best-case efficiencies.
- 28. Worst-Case, Best-Case, and Average-Case Efficiencies • Also note that the average-case efficiency cannot be obtained by taking the average of the worst-case and the best-case efficiencies. • Does one really need the average-case efficiency information? • The answer is unequivocally yes: there are many important algorithms for which the average case efficiency is much better than the worst-case efficiency would.
- 29. Worst-Case, Best-Case, and Average-Case Efficiencies • Amortized efficiency. • In some situations a single operation can be expensive, but the total time for an entire sequence of n such operations is always significantly better than the worst-case efficiency of that single operation multiplied by n. • So we can “amortize” the high cost of such a worst-case occurrence over the entire sequence. • Thus, it applies not to a single run of an algorithm but rather to a sequence of operations performed on the same data structure.
- 30. Worst-Case, Best-Case, and Average-Case Efficiencies • Amortized efficiency. • In some situations a single operation can be expensive, but the total time for an entire sequence of n such operations is always significantly better than the worst-case efficiency of that single operation multiplied by n. • So we can “amortize” the high cost of such a worst-case occurrence over the entire sequence. • Thus, it applies not to a single run of an algorithm but rather to a sequence of operations performed on the same data structure.