![Application to Parallel Reduction
Steps No. of Op. General
1 9 n-1
2 8 n-2
3 6 n-4
4 2 n-8
: : :
i n-2i
Sequential Algorithm
for i = 1 to n-1
a[i] = a[i] + a[i-1]
endfor](https://image.slidesharecdn.com/costoptimalalgorithm-200917050104/85/Cost-optimal-algorithm-14-320.jpg)
Cost optimal algorithm
- 2. Definition 2.1: Cost of an Algorithm The cost of a PRAM computation is the product of the parallel time complexity and the number of processors used. For example, a PRAM algorithm that has time complexity (log p) using p processors has cost (p log p).
- 3. Definition 2.2: Cost Optimal Algorithm A cost optimal parallel algorithm is an algorithm for which the cost is in the same complexity class as an optimal sequential algorithm.
- 4. Definition: Cost Optimal Problem Sequential Cost Parallel Algorithm Cost Optimal No. of PEs Complexity Cost Sum of n Numbers n n/2 log(n) nlog(n) No Prefix Sum (n) n n-1 log(n) nlog(n) No List Ranking n n log(n) nlog(n) No Pre-order Tree Traversal n 2(n-1) log(n) nlog(n) No Merging two sorted list n n log(n) nlog(n) No Graph Coloring (n) Polynomial Cn n2 Cn X n2 No
- 5. Cost Optimal: Reduction Algorithm
- 6. Cost Optimal: Reduction Algorithm The total number of operations performed by the parallel algorithm is of the same complexity class as an optimal sequential algorithm. The parallel reduction algorithm performs about n/2 additions in the first step, n/4 additions in the second step, n/8 additions in the third step, and so on, executing a total of n -1 additions over the log( 𝑛) iterations. Since both the sequential and the parallel algorithms perform n-1 additions, a cost-optimal variant of the parallel reduction algorithm may exists.
- 7. Cost Optimal: Reduction Algorithm Cost = P * Time Complexity = nlog(n) Cost optimal = (n-1) n-1 = P * log(n) P = (n-1)/ log(n) TC = log(n) Cost = (n-1) same as Sequential Once we have determined the appropriate number of processors, we need to verify that there is indeed a cost-optimal parallel-reduction algorithm with logarithmic time complexity.
- 11. Application to Parallel Reduction Parallel Algorithm: Sum of n Numbers • No. of Operations (M) = n-1 • Time Complexity (T) = log( 𝑛) • No. of PEs = 𝑛/2 Cost Optimal Parallel Algorithm • No. of Operations (M) = n-1 • Time Complexity (T) = log( 𝑛) • No. of PEs (P) = (n-1)/ log(n) = 𝑛/log( 𝑛)
- 12. Application to Parallel Reduction
- 13. Application to Parallel Reduction n = 16 p = (n/log n) = 4
- 14. Application to Parallel Reduction Steps No. of Op. General 1 9 n-1 2 8 n-2 3 6 n-4 4 2 n-8 : : : i n-2i Sequential Algorithm for i = 1 to n-1 a[i] = a[i] + a[i-1] endfor
- 15. Application to Parallel Reduction