![Common Computational Complexities
Algorithm Python examples
Adding two numbers together x + y
Summing a vector of numbers sum(x)
Sorting a list v.sort()
Finding an item in an unsorted list x in v v.index(x)
Finding an item in a sorted list Complicated. See bisect library.
Finding an item in a hash table d[x] x in d
Matrix multiplication A.dot(B)
Matrix inversion np.linalg.inv(A)](https://image.slidesharecdn.com/algorithms101fordatascientists-chrispart-200214214652/85/Algorithms-101-for-Data-Scientists-8-320.jpg)
Algorithms 101 for Data Scientists
- 1. Algorithms 101 for Data Scientists Presented by Chris Conlan and Janice McMahon Bethesda Data Science Meetup
- 2. Disclaimer • This is a short presentation.
- 3. Story Time
- 4. Definitions The computational complexity, or simply complexity, of an algorithm describes the amount of resources required to run it relative to its inputs. The term complexity, when standing alone, typically refers to time complexity, or the relationship between the runtime of an algorithm and its inputs.
- 5. Big Oh Notation in Algorithms When we say … It is because … For some arbitrary constant 𝑀. 𝑔 𝑥 is essentially growth behavior of 𝑓 𝑥 . See Wikipedia for a mathematically rigorous definition. This is a simplification for use with algorithms. 𝑓 𝑥 = 𝑂 𝑔 𝑥 𝑓 𝑥 ≤ 𝑀𝑔 𝑥 as 𝑥 → ∞
- 6. Big Oh Notation in Algorithms If 𝑓 𝑥 describes the runtime of an algorithm for input of size 𝑥, then 𝑂 𝑔 𝑥 describes the growth behavior of the runtime. So, given an input of size 𝑛, we might say things like … • My algorithm here is 𝑂(1) • Your algorithm there is 𝑂(𝑛) • My neighbor’s algorithm is 𝑂(log(𝑛)) • My dog’s algorithm is 𝑂(𝑛2)
- 7. Common Computational Complexities Algorithm Input dimensions Complexity Adding two numbers together 2 𝑂 1 Summing a vector of numbers 𝑛 𝑂 𝑛 Sorting a list (fastest known method) 𝑛 𝑂 𝑛 ∗ 𝑙𝑜𝑔(𝑛) Finding an item in an unsorted list 𝑛 𝑂 𝑛 Finding an item in a sorted list (binary search) 𝑛 𝑂 𝑙𝑜𝑔(𝑛) Finding an item in a hash table 𝑛 𝑂 1 Matrix multiplication (fastest known method) two 𝑛 × 𝑛 matrices 𝑂 𝑛2.373 Matrix inversion (fastest known method) 𝑛 × 𝑛 matrix 𝑂 𝑛2.373
- 8. Common Computational Complexities Algorithm Python examples Adding two numbers together x + y Summing a vector of numbers sum(x) Sorting a list v.sort() Finding an item in an unsorted list x in v v.index(x) Finding an item in a sorted list Complicated. See bisect library. Finding an item in a hash table d[x] x in d Matrix multiplication A.dot(B) Matrix inversion np.linalg.inv(A)
- 9. Hidden Complexity • Computational complexity is very well-studied. • There are many optimal algorithms seamlessly integrated into modern programming languages. • It is still easy to accidentally write suboptimal code.
- 10. Compute the cumulative sum 𝑂 𝑛2
- 11. Compute the cumulative sum 𝑂 𝑛2
- 12. Compute the cumulative sum 𝑂 𝑛
- 13. Compute the cumulative sum 𝑂 𝑛
- 14. Don’t do this
- 15. Instead, do this
- 16. function n_values t_milliseconds values_per_ms 0 slow_cumulative_sum 10 0.040 252 1 slow_cumulative_sum 100 0.763 131 2 slow_cumulative_sum 1000 58.234 17 3 slow_cumulative_sum 10000 4846.866 2 4 slow_cumulative_sum_expanded 10 0.028 361 5 slow_cumulative_sum_expanded 100 0.991 101 6 slow_cumulative_sum_expanded 1000 93.545 11 7 slow_cumulative_sum_expanded 10000 9242.796 1 8 fast_cumulative_sum 10 0.016 610 9 fast_cumulative_sum 100 0.042 2358 10 fast_cumulative_sum 1000 0.311 3219 11 fast_cumulative_sum 10000 2.243 4459 12 fast_cumulative_sum 100000 18.902 5290 13 fast_cumulative_sum 1000000 197.117 5073 14 fast_cumulative_sum 10000000 1981.922 5046 15 fast_cumulative_sum 100000000 20699.219 4831 16 pandas_fast_cumulative_sum 10 0.409 24 17 pandas_fast_cumulative_sum 100 0.346 289 18 pandas_fast_cumulative_sum 1000 0.349 2863 19 pandas_fast_cumulative_sum 10000 0.336 29727 20 pandas_fast_cumulative_sum 100000 1.665 60049 21 pandas_fast_cumulative_sum 1000000 14.201 70420 22 pandas_fast_cumulative_sum 10000000 123.686 80850 23 pandas_fast_cumulative_sum 100000000 1031.409 96955 https://github.com/chrisconlan/complexity-studies/blob/master/cumulative_sum.py
- 19. There are countless examples of hidden complexity
- 20. Count the frequency of words in a book function n_values t_milliseconds values_per_ms 0 slow_count_occurences 10 0.010 1031 1 slow_count_occurences 100 0.675 148 2 slow_count_occurences 1000 39.999 25 3 slow_count_occurences 10000 3169.300 3 4 fast_count_occurences 10 0.011 893 5 fast_count_occurences 100 0.042 2387 6 fast_count_occurences 1000 0.218 4591 7 fast_count_occurences 10000 1.988 5031 8 fast_count_occurences 100000 26.933 3713 9 fast_count_occurences 1000000 373.540 2677 10 fast_count_occurences 10000000 4816.169 2076 11 pandas_fast_count_occurences 10 1.372 7 12 pandas_fast_count_occurences 100 0.954 105 13 pandas_fast_count_occurences 1000 1.530 654 14 pandas_fast_count_occurences 10000 4.976 2010 15 pandas_fast_count_occurences 100000 74.877 1336 16 pandas_fast_count_occurences 1000000 1123.875 890 17 pandas_fast_count_occurences 10000000 12356.014 809 https://github.com/chrisconlan/complexity-studies/blob/master/count_occurences.py
- 23. Find the words in a first list that are in a second list function n_values t_milliseconds values_per_ms 0 slow_match_within 10 0.006 1639 1 slow_match_within 100 0.192 520 2 slow_match_within 1000 16.793 60 3 slow_match_within 10000 1577.224 6 4 fast_match_within 10 0.010 1010 5 fast_match_within 100 0.027 3690 6 fast_match_within 1000 0.196 5110 7 fast_match_within 10000 1.931 5180 8 fast_match_within 100000 37.496 2667 9 fast_match_within 1000000 533.951 1873 10 fast_match_within 10000000 6079.538 1645 11 fast_intersection 10 0.013 769 12 fast_intersection 100 0.026 3774 13 fast_intersection 1000 0.159 6277 14 fast_intersection 10000 1.967 5083 15 fast_intersection 100000 27.306 3662 16 fast_intersection 1000000 408.962 2445 17 fast_intersection 10000000 4790.723 2087 https://github.com/chrisconlan/complexity-studies/blob/master/match_within.py