Parallel Computing with R
1 like1,350 views
Parallel computing allows breaking problems into independent pieces that can be computed simultaneously across multiple processors. The document discusses using the snow package in R to set up a simple parallel cluster on a single machine and perform operations like bootstrapping in parallel. It also mentions more advanced high performance computing techniques for large memory, compiled code, profiling, and batch scheduling.
![Example Parallel Overview snow fork Summary
NegBin GLM with bootstrap
> library(MASS)
> m <- glm.nb(count ~ pforest, oven)
> fun1 <- function(i) {
+ id <- sample.int(nrow(oven), nrow(oven), replace = TRUE)
+ coef(glm.nb(count ~ pforest, oven[id, ]))
+ }
> B <- 199
> system.time(bm <- sapply(1:B, fun1))
user system elapsed
26.79 0.02 27.11
> bm <- cbind(coef(m), bm)
> cbind(coef(summary(m))[, 1:2], `Boot. SE` = apply(bm, 1, sd))
Estimate Std. Error Boot. SE
(Intercept) -2.177 0.1277 0.1229
pforest 2.674 0.1709 0.1553](https://image.slidesharecdn.com/parallelcomp-130527142226-phpapp02/85/Parallel-Computing-with-R-3-320.jpg)
![Example Parallel Overview snow fork Summary
Distribute stu, evaluate expressions
clusterExport(cl, oven)
clusterEvalQ(cl, library(MASS))
[[1]]
[1] MASS methods stats graphics
[5] grDevices utils datasets base
[[2]]
[1] MASS methods stats graphics
[5] grDevices utils datasets base
[[3]]
[1] MASS methods stats graphics
[5] grDevices utils datasets base](https://image.slidesharecdn.com/parallelcomp-130527142226-phpapp02/85/Parallel-Computing-with-R-9-320.jpg)
![Example Parallel Overview snow fork Summary
Random Number Generation (RNG)
library(rlecuyer)
tmp - clusterEvalQ(cl, set.seed(1234))
clusterEvalQ(cl, rnorm(5))
[[1]]
[1] -1.2071 0.2774 1.0844 -2.3457 0.4291
[[2]]
[1] -1.2071 0.2774 1.0844 -2.3457 0.4291
snow:::clusterSetupRNG(cl)
[1] RNGstream
clusterEvalQ(cl, rnorm(5))
[[1]]
[1] -1.14063 -0.49816 -0.76670 -0.04821 -1.09852
[[2]]
[1] 0.7050 0.4821 -1.2848 0.7198 0.7386
Important when calculating indices or doing simulations.](https://image.slidesharecdn.com/parallelcomp-130527142226-phpapp02/85/Parallel-Computing-with-R-10-320.jpg)
![Example Parallel Overview snow fork Summary
Apply operations: split
parallel:::parLapply
function (cl = NULL, X, fun, ...)
{
cl - defaultCluster(cl)
do.call(c, clusterApply(cl, x = splitList(X, length(cl)),
fun = lapply, fun, ...), quote = TRUE)
}
bytecode: 0x04c1eba8
environment: namespace:parallel
snow:::splitList(1:10, length(cl))
[[1]]
[1] 1 2 3 4 5
[[2]]
[1] 6 7 8 9 10](https://image.slidesharecdn.com/parallelcomp-130527142226-phpapp02/85/Parallel-Computing-with-R-11-320.jpg)
![Example Parallel Overview snow fork Summary
Apply operations: evaluate and combine
f - function(i) i * 2
(res - clusterApply(cl, snow:::splitList(1:10, length(cl)),
+ f))
[[1]]
[1] 2 4 6
[[2]]
[1] 8 10 12 14
[[3]]
[1] 16 18 20
do.call(c, res)
[1] 2 4 6 8 10 12 14 16 18 20](https://image.slidesharecdn.com/parallelcomp-130527142226-phpapp02/85/Parallel-Computing-with-R-12-320.jpg)
![Example Parallel Overview snow fork Summary
Apply operations: load balancing
f - function(i) i * 2
unlist(parallel:::parLapplyLB(cl, 1:10, f))
[1] 2 4 6 8 10 12 14 16 18 20](https://image.slidesharecdn.com/parallelcomp-130527142226-phpapp02/85/Parallel-Computing-with-R-13-320.jpg)
Parallel Computing with R
- 1. Example Parallel Overview snow fork Summary Parallel Computing with R Péter Sólymos Edmonton R User Group meeting, April 26, 2013
- 2. Example Parallel Overview snow fork Summary Ovenbird example from 'detect' package > str(oven) 'data.frame': 891 obs. of 11 variables: $ count : int 1 0 0 1 0 0 0 0 0 0 ... $ route : int 2 2 2 2 2 2 2 2 2 2 ... $ stop : int 2 4 6 8 10 12 14 16 18 20 ... $ pforest: num 0.947 0.903 0.814 0.89 0.542 ... $ pdecid : num 0.575 0.562 0.549 0.679 0.344 ... $ pagri : num 0 0 0 0 0.414 ... $ long : num 609343 608556 607738 607680 607944 ... $ lat : num 5949071 5947735 5946301 5944720 5943088 ... $ observ : Factor w/ 4 levels "ARS","DW","RDW",..: 4 4 4 4 4 4 4 4 4 4 ... $ julian : int 181 181 181 181 181 181 181 181 181 181 ... $ timeday: int 2 4 6 8 10 12 14 16 18 20 ...
- 3. Example Parallel Overview snow fork Summary NegBin GLM with bootstrap > library(MASS) > m <- glm.nb(count ~ pforest, oven) > fun1 <- function(i) { + id <- sample.int(nrow(oven), nrow(oven), replace = TRUE) + coef(glm.nb(count ~ pforest, oven[id, ])) + } > B <- 199 > system.time(bm <- sapply(1:B, fun1)) user system elapsed 26.79 0.02 27.11 > bm <- cbind(coef(m), bm) > cbind(coef(summary(m))[, 1:2], `Boot. SE` = apply(bm, 1, sd)) Estimate Std. Error Boot. SE (Intercept) -2.177 0.1277 0.1229 pforest 2.674 0.1709 0.1553
- 4. Example Parallel Overview snow fork Summary Parallel bootstrap > library(parallel) > (cl <- makePSOCKcluster(3)) socket cluster with 3 nodes on host 'localhost' > clusterExport(cl, "oven") > tmp <- clusterEvalQ(cl, library(MASS)) > t0 <- proc.time() > bm2 <- parSapply(cl, 1:B, fun1) > proc.time() - t0 user system elapsed 0.00 0.00 11.06 > stopCluster(cl)
- 5. Example Parallel Overview snow fork Summary High performance computing (HPC) ˆ Parallel computing, ˆ large memory and out-of-memory data, ˆ interfaces for compiled code, ˆ proling tools, ˆ batch scheduling. CRAN Task View: High-Performance and Parallel Computing with R
- 6. Example Parallel Overview snow fork Summary Parallel computing Embarassingly parallel problems: ˆ bootstrap, ˆ MCMC, ˆ simulations. Can be broken down into independent pieces.1 1Schmidberger et al. 2009 JSS: State of the Art in Parallel Computing with R
- 7. Example Parallel Overview snow fork Summary Parallel computing ˆ explicit (distributed memory), ˆ implicit (shared memory), ˆ grid, ˆ Hadoop, ˆ GPUs.
- 8. Example Parallel Overview snow fork Summary Starting a cluster library(snow) cl - makeCluster(3, type = SOCK) Cluster types: ˆ SOCK, multicore ˆ PVM, Parallel Virtual Machine ˆ MPI, Message Passing Interface ˆ NWS, NetWorkSpaces (multicore grid) Error: invalid connection
- 9. Example Parallel Overview snow fork Summary Distribute stu, evaluate expressions clusterExport(cl, oven) clusterEvalQ(cl, library(MASS)) [[1]] [1] MASS methods stats graphics [5] grDevices utils datasets base [[2]] [1] MASS methods stats graphics [5] grDevices utils datasets base [[3]] [1] MASS methods stats graphics [5] grDevices utils datasets base
- 10. Example Parallel Overview snow fork Summary Random Number Generation (RNG) library(rlecuyer) tmp - clusterEvalQ(cl, set.seed(1234)) clusterEvalQ(cl, rnorm(5)) [[1]] [1] -1.2071 0.2774 1.0844 -2.3457 0.4291 [[2]] [1] -1.2071 0.2774 1.0844 -2.3457 0.4291 snow:::clusterSetupRNG(cl) [1] RNGstream clusterEvalQ(cl, rnorm(5)) [[1]] [1] -1.14063 -0.49816 -0.76670 -0.04821 -1.09852 [[2]] [1] 0.7050 0.4821 -1.2848 0.7198 0.7386 Important when calculating indices or doing simulations.
- 11. Example Parallel Overview snow fork Summary Apply operations: split parallel:::parLapply function (cl = NULL, X, fun, ...) { cl - defaultCluster(cl) do.call(c, clusterApply(cl, x = splitList(X, length(cl)), fun = lapply, fun, ...), quote = TRUE) } bytecode: 0x04c1eba8 environment: namespace:parallel snow:::splitList(1:10, length(cl)) [[1]] [1] 1 2 3 4 5 [[2]] [1] 6 7 8 9 10
- 12. Example Parallel Overview snow fork Summary Apply operations: evaluate and combine f - function(i) i * 2 (res - clusterApply(cl, snow:::splitList(1:10, length(cl)), + f)) [[1]] [1] 2 4 6 [[2]] [1] 8 10 12 14 [[3]] [1] 16 18 20 do.call(c, res) [1] 2 4 6 8 10 12 14 16 18 20
- 13. Example Parallel Overview snow fork Summary Apply operations: load balancing f - function(i) i * 2 unlist(parallel:::parLapplyLB(cl, 1:10, f)) [1] 2 4 6 8 10 12 14 16 18 20
- 14. Example Parallel Overview snow fork Summary Implicit parallelism No need to distribute stu, only evaluate on child processes. mclapply(X, FUN, mc.cores)
- 15. Example Parallel Overview snow fork Summary Summary Parallel computing is not hard on a single computer. Diculty comes in when using large, shared, and heterogeneous resources. stopCluster(cl)