![Another example: increasing a binary counter
• Binary counter of length k, A[0..k-1] of bit array.
• INCREMENT(A)
1. i0
2. while i<k and A[i]=1
3. do A[i]0 (flip, reset)
4. ii+1
5. if i<k
6. then A[i]1 (flip, set)](https://image.slidesharecdn.com/aaunit1part-i-230522044237-b1e8035f/85/AA_Unit-1_part-I-pptx-38-320.jpg)
![Amortized (Aggregate) Analysis of INCREMENT(A)
Observation: The running time determined by #flips but not all bits flip each time INCREMENT is
called.
A[0] flips every time, total n times.
A[1] flips every other time, n/2 times.
A[2] flips every forth time, n/4 times.
….
for i=0,1,…,k-1, A[i] flips n/2i times.
Thus total #flips is
=2n.
𝑖=0
log 𝑛
𝑛
2𝑖 <
𝑖=0
∞
𝑛
2𝑖](https://image.slidesharecdn.com/aaunit1part-i-230522044237-b1e8035f/85/AA_Unit-1_part-I-pptx-39-320.jpg)
AA_Unit 1_part-I.pptx
- 1. Advanced Algorithms (PCCO6020T) Unit 1 Prof. Swapnil H. Chaudhari Computer Engineering Department, R.C.Patel Institute of Technology, Shirpur
- 9. Introduction Definition of algorithms: A set of instructions that solve a specific problem or accomplish a specific task. Importance of algorithms in computer science: Algorithms form the backbone of computer science and are essential in fields such as artificial intelligence, data science, cryptography, and more. Types of algorithms: There are many different types of algorithms, including search algorithms, sorting algorithms, graph algorithms, and dynamic programming algorithms, among others.
- 10. Introduction Time and space complexity: Two important factors in evaluating the efficiency of an algorithm are its time complexity, which refers to the amount of time it takes to run, and its space complexity, which refers to the amount of memory it requires. Big O notation: Big O notation is a mathematical notation used to describe the upper bound of an algorithm's time complexity. It provides a way to express the worst-case scenario for an algorithm's running time.
- 11. What is time complexity Time complexity is a function that describes the amount of time required to run an algorithm, as input size of the algorithm Calculating time complexity by the growth of the algorithm is the most reliable way of calculating the efficiency of an algorithm
- 12. Calculate the Time Complexity A common mistake with time complexity is to think of it as the running time(clock time) of an algorithm. The running time of the algorithm is how long it takes the computer to execute the lines of code to completion, usually measured in milliseconds or seconds. Using this method is not the most efficient way of calculating the running time of an algorithm, cause the running time of an algorithm depends on The speed of the computer (Hardware). The programming language used (Java, C++, Python). The compiler that translates our code into machine code (Clang, GCC, Min GW).
- 13. Consider a model machine Assigning values to variables. Making comparisons. Executing arithmetic operations. Accessing objects from memory.
- 14. Time Complexity: In the above code “Hello World” is printed only once on the screen. So, the time complexity is constant: O(1)
- 15. 1) Loop for( i = 1 ; i < = n ; i++) { x= y + z; }
- 17. Amortized Analysis Amortized analysis is applied on data structures that support many operations. The sequence of operations and the multiplicity of each operation is application specific or the associated algorithm specific. Classical asymptotic analysis gives worst case analysis of each operation without taking the effect of one operation on the other. Amortized analysis focuses on a sequence of operations, an interplay between operations, and thus yielding an analysis which is precise and depicts a micro-level analysis. 17
- 18. Amortized Analysis Purpose is to accurately compute the total time spent in executing a sequence of operations on a data structure Three different approaches: Aggregate method Accounting method Potential method 18
- 19. Aggregate method We determine an upper bound T(n) on Total cost of a sequence of n-operations. In the worst case : The average cost or amortized cost per operation is = 𝑇(𝑛) 𝑛 Note that this amortized cost applies to each operation, even when there is several types of operations in sequence. Note that this amortized cost applies to each operation, even when there is several types of operations in sequence. 19
- 20. How large should a hash table be? Goal: Make the table as small as possible, but large enough so that it won’t overflow (or otherwise become inefficient). Problem: What if we don’t know the proper size in advance? Solution: Dynamic tables. IDEA: Whenever the table overflows, “grow” it by allocating (via malloc or new) a new, larger table. Move all items from the old table into the new one, and free the storage for the old table.
- 21. Example of a dynamic table 1. INSERT 2. INSERT 1 overflow
- 22. 1 Example of a dynamic table 1. INSERT 2. INSERT overflow
- 23. 1 2 Example of a dynamic table 1. INSERT 2. INSERT
- 24. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 1 2 overflow
- 25. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 2 1 overflow
- 26. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 2 1
- 27. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 4 3 2 1
- 28. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 5. INSERT 4 3 2 1 overflow
- 29. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 5. INSERT 4 3 2 1 overflow
- 30. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 5. INSERT 4 3 2 1
- 31. Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 5. INSERT 6. INSERT 7. INSERT 6 5 4 3 2 1 7
- 32. Worst-case analysis Consider a sequence of n insertions. The worst-case time to execute one insertion is (n).Therefore, the worst-case time for n insertions is n · (n) = (n2). WRONG! In fact, the worst-case cost for n insertions is only (n) ≪ (n2). Let’s see why.
- 33. Tighter analysis i 1 2 3 4 5 6 7 8 9 10 sizei 1 2 4 4 8 8 8 8 16 16 ci 1 2 3 1 5 1 1 1 9 1 Let ci = the cost of the ith insertion = i if i – 1 is an exact power of 2, 1 otherwise.
- 34. Tighter analysis Let ci = the cost of the ith insertion = i if i – 1 is an exact power of 2, 1 otherwise. i 1 2 3 4 5 6 7 8 9 10 sizei 1 2 4 4 8 8 8 8 16 16 ci 1 1 1 1 2 1 1 4 1 1 1 1 8 1
- 35. Tighter analysis (continued) 2 j n n lg(n1) j0 Cost of n insertions ci i1 3n (n). Thus, the average cost of each dynamic-table operation is (n)/n = (1).
- 36. Example for amortized analysis • Amortized analysis can be used to show that the average cost of an operation is small , if one averages over a sequence of operations ,even though a single operation within the sequence might be expensive • Stack operations: – PUSH(S,x), O(1) – POP(S), O(1) – MULTIPOP(S,k), min(s,k) •while not STACK-EMPTY(S) and k>0 • do POP(S) • k=k-1 • Let us consider a sequence of n PUSH, POP, MULTIPOP. – The worst case cost for MULTIPOP in the sequence is O(n), since the stack size is at most n. – thus the cost of the sequence is O(n2). Correct, but not tight.
- 37. Aggregate Analysis • In fact, a sequence of n operations on an initially empty stack cost at most O(n). Why? Each object can be POP only once (including in MULTIPOP) for each time it is PUSHed. #POPs is at most #PUSHs, which is at most n. Thus the average cost of an operation is O(n)/n = O(1). Amortized cost in aggregate analysis is defined to be average cost.
- 38. Another example: increasing a binary counter • Binary counter of length k, A[0..k-1] of bit array. • INCREMENT(A) 1. i0 2. while i<k and A[i]=1 3. do A[i]0 (flip, reset) 4. ii+1 5. if i<k 6. then A[i]1 (flip, set)
- 39. Amortized (Aggregate) Analysis of INCREMENT(A) Observation: The running time determined by #flips but not all bits flip each time INCREMENT is called. A[0] flips every time, total n times. A[1] flips every other time, n/2 times. A[2] flips every forth time, n/4 times. …. for i=0,1,…,k-1, A[i] flips n/2i times. Thus total #flips is =2n. 𝑖=0 log 𝑛 𝑛 2𝑖 < 𝑖=0 ∞ 𝑛 2𝑖
- 40. Accounting Method In accounting method, we assign different charges to different operations. The amount we charge is called amortized cost 𝐶𝑖 = is the actual cost 𝐶𝑖 =is the amortized cost
- 41. Accounting method • Charge ith operation a fictitious amortized cost ĉi, where $1 pays for 1 unit of work (i.e., time). • This fee is consumed to perform the operation. • Any amount not immediately consumed is stored in the bank for use by subsequent operations. • The bank balance must not go negative! We must ensure that n n i1 i1 ci cˆi for all n. • Thus, the total amortized costs provide an upper bound on the total true costs.
- 42. Charge an amortized cost of ĉi = $3 for the ith insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item. Accounting analysis of dynamic tables
- 43. Accounting analysis (continued) Key invariant: Bank balance never drops below 0. Thus, the sum of the amortized costs provides an upper bound on the sum of the true costs. i 1 2 3 4 5 6 7 8 9 10 sizei 1 2 4 4 8 8 8 8 16 16 ci 1 2 3 1 5 1 1 1 9 1 ĉi 2* 3 3 3 3 3 3 3 3 3 banki 1 2 2 4 2 4 6 8 2 4
- 46. Potential method IDEA: View the bank account as the potential energy (à la physics) of the dynamic set. Framework: • Start with an initial data structure D0. • Operation i transforms Di–1 to Di. • The cost of operation i is ci. • Define a potential function : {Di} R, such that (D0 ) = 0 and (Di ) 0 for all i. • The amortized cost ĉi with respect to is defined to be ĉi = ci + (Di) – (Di–1).
- 47. Understanding potentials ĉi = ci + (Di) – (Di–1) potential difference i • If i > 0, then ĉi > ci. Operation i stores work in the data structure for later use. • If i < 0, then ĉi < ci. The data structure delivers up stored work to help pay for operation i.
- 48. The amortized costs bound the true costs The total amortized cost of n operations is n n ĉi ci (Di ) (Di1) i1 i1 Summing both sides.
- 49. The amortized costs bound the true costs
- 50. The amortized costs bound the true costs
- 61. Conclusions • Amortized costs can provide a clean abstraction of data-structure performance. • Any of the analysis methods can be used when an amortized analysis is called for, but each method has some situations where it is arguably the simplest. • Different schemes may work for assigning amortized costs in the accounting method, or potentials in the potential method, sometimes yielding radically different bounds.