Algorithms - Rocksolid Tour 2013
Download as PPTX, PDF0 likes857 views
This document summarizes a presentation on algorithm performance and asymptotic analysis. It introduces common algorithm analysis techniques like Big O notation and the master method for solving recurrences. Specific algorithms like insertion sort and merge sort are analyzed. Insertion sort is shown to have quadratic worst-case runtime while merge sort achieves linearithmic runtime through a divide and conquer approach. Overall the presentation aims to demonstrate how asymptotic analysis can provide insights into algorithm optimization.
![Analysis of Insertion Sort
InsertionSort(A)
for j = 2 to A.length
key=[Aj]
i=j-1
while i > 0 and A[i] > key
A[i+1] = A[i]
i=i-1
A[i+1] = key
9](https://image.slidesharecdn.com/algorithms-130123054628-phpapp02/85/Algorithms-Rocksolid-Tour-2013-9-320.jpg)
![Best Case Running Time
If the input (A) is already sorted then...
A[i] <= key when has initial value of j-1 thus tj=1.
And so...
T(n) = c1n+c2(n-1)+c3(n-1)+c4(n-1)+c7(n-1)
= (c1+c2+c3+c4+c7)n-(c2+c3+c4+c7)
Which can be expressed as an+b for constants a
and b that depend on ci
So T(n) is a linear function of n
13](https://image.slidesharecdn.com/algorithms-130123054628-phpapp02/85/Algorithms-Rocksolid-Tour-2013-13-320.jpg)
![Worst Case Scenario
If the input (n) is in reverse sort order then...
We have to compare each A[j] with each
element in the sub array A[1..j-1].
And so...
T(n) = (c4/2+c5/2+c6/2)n^2 +(c1 +c2+c3+c4/2-
c5/2-c6/2+c7)n-(c2+c3+c4+c7)
Which can be expressed as an^2 + bn + c
So T(n) is a quadratic function of n
16](https://image.slidesharecdn.com/algorithms-130123054628-phpapp02/85/Algorithms-Rocksolid-Tour-2013-16-320.jpg)
![Analysis of Merge Sort
MergeSort(A,p,r)
if(p<r)
q = [(p+r)/2]
MergeSort(A,p,q)
MergeSort(A,q+1,r)
Merge(A,p,q,r)
Initial call MergeSort(A,1,A.length)
29](https://image.slidesharecdn.com/algorithms-130123054628-phpapp02/85/Algorithms-Rocksolid-Tour-2013-29-320.jpg)
Algorithms - Rocksolid Tour 2013
- 1. Asymptotes and Algorithms By Gary Short Gibraltar Software 1
- 2. Agenda • Introduction • Performance, does it matter? • How do we measure performance? • Analysis of Insertion Sort • Simplifying things with asymptotic notation • Designing algorithms • Solving recurrences • Questions. 2
- 3. Introduction • Gary Short • Head of Gibraltar Labs • C# MVP • [email protected] • @garyshort • http://www.facebook.com/theothergaryshort 3
- 4. Performance – Does it Matter? Performance is the most important thing in software engineering today... 4
- 5. ... Apart from everything else! 5
- 6. So Why Bother About Performance? 6
- 7. How do we Measure Performance? • What do we care about? – Memory? – Bandwidth? – Computational time? 7
- 8. We Need a Model to Work With • RAM Model – Arithmetic – add, subtract, etc – Data movement – load, copy, store – Control – branching, subroutine call, return – Data – Integers, floats • Instruction are run in series – And take constant time • Not really, but shhh! –Ed. 8
- 9. Analysis of Insertion Sort InsertionSort(A) for j = 2 to A.length key=[Aj] i=j-1 while i > 0 and A[i] > key A[i+1] = A[i] i=i-1 A[i+1] = key 9
- 10. That Makes no Sense, Show me! 10
- 11. So What’s The Running Time? 11
- 12. Sum Running Time for Each Statement... T(n) = c1n+c2(n-1)+c3(n-1)+c4 sum(tj) j=2..n+c5 sum(tj-1) j=2..n+c6sum(tj-1) j=2..n+c7(n-1) 12
- 13. Best Case Running Time If the input (A) is already sorted then... A[i] <= key when has initial value of j-1 thus tj=1. And so... T(n) = c1n+c2(n-1)+c3(n-1)+c4(n-1)+c7(n-1) = (c1+c2+c3+c4+c7)n-(c2+c3+c4+c7) Which can be expressed as an+b for constants a and b that depend on ci So T(n) is a linear function of n 13
- 14. 14
- 15. Side Note: No One Cares About Best Case 15
- 16. Worst Case Scenario If the input (n) is in reverse sort order then... We have to compare each A[j] with each element in the sub array A[1..j-1]. And so... T(n) = (c4/2+c5/2+c6/2)n^2 +(c1 +c2+c3+c4/2- c5/2-c6/2+c7)n-(c2+c3+c4+c7) Which can be expressed as an^2 + bn + c So T(n) is a quadratic function of n 16
- 17. 17
- 18. In Short... In worst case insertion sort sucks! 18
- 19. Man That Was a Lot of Maths! 19
- 20. Simplifying Things With Asymptotic Notation • Asymptotic notation characterises functions by their growth rates • Functions with the same growth rates have the same Asymptotic notation 20
- 21. How Does That Help Us? Let’s say we have a function with running time T(n) = 4n^2 - 2n + 2 If n = 500 then 4n^2 is 1000 times bigger than 2n So... We can ignore smaller order terms and coefficients T(n) = 4n^2 -2n +2 can be written O(n) = n^2 21
- 22. A Short Note on The Abuse of “=“ If T(n) = 4n^2 -2n +2 Then saying T(n) = O(n^2) is not strictly correct Rather T(n) is in the set O(n^2) and the above should be read as T(n) is O(n^2) and not T(n) equals O(n^2) But really on Maths geeks care – Ed. 22
- 23. So Back to Insertion Sort So now we can say of Insertion Sort that... Best case it’s O(n) And worst case it’s O(n^2) And since we only care about worst case... We say that Insertion Sort has O(n^2) Which sucks! – Ed. 23
- 24. Designing Algorithms So can we do better? 24
- 25. Optimizing Algorithms is Child’s Play • Sit at table • Foreach item in itemsOnPlate – Eat item • Wait(MealComplete) • Foreach dish in dishesUsed – WashDish – DryDish • Resume Play 25
- 26. Child Will Optimize To… • Pause Game • Set Speed = MaxInt • Run to table • Take sliceBread(1) • Foreach item on Plate – Place item on bread • Take sliceBread(2) • Run Outside • Resume Game 26
- 27. Divide And Conquer • Divide – Divide the problem into sub problems • Conquer – Solve the sub problems recursively • Combine – Add the solutions to the sub problems into the solution for the original problem. 27
- 28. Merge Sort • Divide – Divide the n elements into two n/2 element arrays • Conquer – Sort the two arrays recursively • Combine – Merge the two sorted arrays to produce the answer. 28
- 29. Analysis of Merge Sort MergeSort(A,p,r) if(p<r) q = [(p+r)/2] MergeSort(A,p,q) MergeSort(A,q+1,r) Merge(A,p,q,r) Initial call MergeSort(A,1,A.length) 29
- 30. Dancers, or it Never Happened!! 30
- 31. So What’s The Running Time? In the general case... If the divide step yields ‘a’ sub problems Each 1/b the size of the original It takes T(n/b) time to solve one problem of n/b size So it takes aT(n/b) to solve ‘a’ of them Then, if it takes D(n) time to divide the problem And C(n) time to combine the results Then we get the recurrence... T(n) = aT(n/b) + D(n) + C(n). 31
- 32. Apply That to Merge Sort... • Divide – Computes the middle of the subarray, taking constant time so, D(n) = O(1) • Conquer – Recursively solve two sub problems each of size n/2 contributing 2T(n/2) to the running time • Combine – Merge procedure O(n) • Giving us a recurrence of 2T(n/2)+O(n) 32
- 33. Solve The Recurrence Using The Master Method For a Recurrence in the form T(n) = aT(n/b) + f(n) Then If f(n) = O(nlogba-k) then T(n) = O(nlogba) If f(n) = O(nlogba) then T(n) = O(nlogba log n) if f(n) = Omega(n log b a+k) and if af(n/b) <= cf(n) then T(n) = O(f(n)) 33
- 34. What?! • More simply we are comparing f(n) with the function n log ba and intuitively understanding that the bigger of the two determines the solution to the recurrence. 34
- 35. And So... • With Merge Sort we are in the third case of the Master Method thus... • T(n) = O(n log n) • Which is much better than the O(n^2) of Insertion Sort 35
- 36. 36
- 37. What We Learned • Performance is important • Therefore algorithmic optimization is too • We have a model to benchmark • And a syntax • Divide and conquer • Master Method • Other resources. 37
- 38. 38
- 39. Questions? 39