SlideShare a Scribd company logo

MPI Raspberry pi 3 cluster

Download as PPTX, PDF
A
Arafat Hussain

The document discusses setting up a 4-node MPI Raspberry Pi cluster and Hadoop cluster. It describes the hardware and software needed for the MPI cluster, including 4 Raspberry Pi 3 boards, Ethernet cables, micro SD cards, and MPI software. It also provides an overview of Hadoop, a framework for distributed storage and processing of big data, noting its origins from Google papers and use by companies like Amazon, Facebook, and Netflix.

1 of 16
Downloaded 12 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
MPI Raspberry pi 3 & Hadoop cluster 4 nodes
MPI Raspberry pi cluster  Why MPI  Hardware Needed  Software Needed  Machine File  MPI Code  Python Code  Output  Finished product
Why MPI?  Scientist during World War II used the same method of Parallel Computing to solve mathematical problems in the Manhattan Project.  reduce the amount of time it took to solve a large mathematical problem.  the people who performed these calculations were called computers.  How far can you go with MPI?
Hardware you need X4 Raspberry pi 3 X5 Ethernet cat 5 cables X4 micro USB cables X4 8 GB micro SD One 5 port Ethernet switch Option but recommended one 4 port USB power hub
Software Needed  MPICH  MPI4PY  Python to MPI interpreter
Machine File
MPI code
Python code
Output
Finished product
Hadoop  Hadoop is a framework of tools based on Java that supports running Big Data.  Inspired from Google’s GFS and MapReduce.  It is open source.  Relatively cheap to build.  Has Fault Tolerant System.
Parts of Hadoop  Hadoop consists of two main elements,  HDFS (Hadoop Distributed File System)  Name Node (Store the directories of all files in the cluster)  Data Node (Store Data)  MapReduce  Job Tracker(Responsible for taking requests from client and assign to TaskTracker)  Task Tracker(Perform MapReduce in DataNodes/Slave)
HDFS (Hadoop Distributed File System)
MapReduce
Use of Hadoop in Tech World  Amazon  Facebook  Netflix  eBay  Twitter etc.
Thank You

More Related Content

What's hot (20)

PDF
Understand and Harness the Capabilities of Intel® Xeon Phi™ Processors
Intel® Software
 
PDF
Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...
Intel® Software
 
PPTX
My ppt hpc u4
Vidyalankar Institute of Technology
 
PDF
Manycores for the Masses
Intel® Software
 
PPT
Multicore
Birgit Plötzeneder
 
PDF
Standardizing on a single N-dimensional array API for Python
Ralf Gommers
 
PPT
Migration To Multi Core - Parallel Programming Models
Zvi Avraham
 
PDF
Numba Overview
stan_seibert
 
PDF
Get Your Hands Dirty with Intel® Distribution for Python*
Intel® Software
 
PDF
Hetergeneous Compute with Standards Based OFI/MPI/OpenMP Programming
Intel® Software
 
PDF
Optimize Single Particle Orbital (SPO) Evaluations Based on B-splines
Intel® Software
 
PDF
SciPy 2019: How to Accelerate an Existing Codebase with Numba
stan_seibert
 
PDF
PyData NYC whatsnew NumPy-SciPy 2019
Ralf Gommers
 
PDF
High Performance Python - Marc Garcia
Marc Garcia
 
PDF
On the Capability and Achievable Performance of FPGAs for HPC Applications
Wim Vanderbauwhede
 
PDF
TinyML as-a-Service
Hiroshi Doyu
 
PDF
Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...
Intel® Software
 
PDF
Numba: Array-oriented Python Compiler for NumPy
Travis Oliphant
 
KEY
Numba
Travis Oliphant
 
PDF
Introduction to MPI
Akhila Prabhakaran
 
Understand and Harness the Capabilities of Intel® Xeon Phi™ Processors
Intel® Software
 
Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...
Intel® Software
 
My ppt hpc u4
Vidyalankar Institute of Technology
 
Manycores for the Masses
Intel® Software
 
Multicore
Birgit Plötzeneder
 
Standardizing on a single N-dimensional array API for Python
Ralf Gommers
 
Migration To Multi Core - Parallel Programming Models
Zvi Avraham
 
Numba Overview
stan_seibert
 
Get Your Hands Dirty with Intel® Distribution for Python*
Intel® Software
 
Hetergeneous Compute with Standards Based OFI/MPI/OpenMP Programming
Intel® Software
 
Optimize Single Particle Orbital (SPO) Evaluations Based on B-splines
Intel® Software
 
SciPy 2019: How to Accelerate an Existing Codebase with Numba
stan_seibert
 
PyData NYC whatsnew NumPy-SciPy 2019
Ralf Gommers
 
High Performance Python - Marc Garcia
Marc Garcia
 
On the Capability and Achievable Performance of FPGAs for HPC Applications
Wim Vanderbauwhede
 
TinyML as-a-Service
Hiroshi Doyu
 
Accelerate Your Python* Code through Profiling, Tuning, and Compilation Part ...
Intel® Software
 
Numba: Array-oriented Python Compiler for NumPy
Travis Oliphant
 
Numba
Travis Oliphant
 
Introduction to MPI
Akhila Prabhakaran
 

Viewers also liked (19)

PDF
MPI and Distributed Applications
Marcos Gonzalez
 
PDF
Open MPI State of the Union X SC'16 BOF
Jeff Squyres
 
PDF
Ejclase mpi
Fernanda Escobar
 
PDF
CUDA-Aware MPI
Eugene Kolesnikov
 
PPTX
MPI message passing interface
Mohit Raghuvanshi
 
PDF
FDA Focus on Design Controls
April Bright
 
PPTX
Présentation Raspberry Pi (cocoaheads remix)
Arnaud Boudou
 
PDF
Sophia conf 2013 - Le monde du Raspberry
Nicolas Hennion
 
PDF
Raspberry Pi - Lecture 3 Embedded Communication Protocols
Mohamed Abdallah
 
PDF
raspberry pi
TECOS
 
PDF
Introduction to Windows IoT via Raspberry Pi 3
Lee Richardson
 
DOCX
FINAL SEMINAR REPORT OF RASPBERRY PI
GANESH GOVIND BHOR
 
PPTX
Raspberry-Pi
Christophe Porchet
 
PDF
Introduction à l'IoT: du capteur à la donnée_Presentation Mix-IT2015
Sameh BEN FREDJ
 
PPTX
Présentation Projet de fin d'année
Yassine DAHMANE
 
PDF
Hec propriété intellectuelle, seance 1, 30 septembre 2015
Céline Bondard
 
PDF
Gateway d’un système de monitoring
Ghassen Chaieb
 
PDF
How to Make Awesome SlideShares: Tips & Tricks
SlideShare
 
PDF
Getting Started With SlideShare
SlideShare
 
MPI and Distributed Applications
Marcos Gonzalez
 
Open MPI State of the Union X SC'16 BOF
Jeff Squyres
 
Ejclase mpi
Fernanda Escobar
 
CUDA-Aware MPI
Eugene Kolesnikov
 
MPI message passing interface
Mohit Raghuvanshi
 
FDA Focus on Design Controls
April Bright
 
Présentation Raspberry Pi (cocoaheads remix)
Arnaud Boudou
 
Sophia conf 2013 - Le monde du Raspberry
Nicolas Hennion
 
Raspberry Pi - Lecture 3 Embedded Communication Protocols
Mohamed Abdallah
 
raspberry pi
TECOS
 
Introduction to Windows IoT via Raspberry Pi 3
Lee Richardson
 
FINAL SEMINAR REPORT OF RASPBERRY PI
GANESH GOVIND BHOR
 
Raspberry-Pi
Christophe Porchet
 
Introduction à l'IoT: du capteur à la donnée_Presentation Mix-IT2015
Sameh BEN FREDJ
 
Présentation Projet de fin d'année
Yassine DAHMANE
 
Hec propriété intellectuelle, seance 1, 30 septembre 2015
Céline Bondard
 
Gateway d’un système de monitoring
Ghassen Chaieb
 
How to Make Awesome SlideShares: Tips & Tricks
SlideShare
 
Getting Started With SlideShare
SlideShare
 
Ad

Similar to MPI Raspberry pi 3 cluster (20)

PPT
Raspberry Pi Cluster Test Bed
Bradford Bazemore
 
PPTX
Senior Design: Raspberry Pi Cluster Computing
Ralph Walker II
 
PPTX
Introduction to MPI Basics easy way.pptx
imareebkhan25
 
PDF
Cluster Setup Manual Using Ubuntu and MPICH
Misu Md Rakib Hossain
 
PDF
Elasticwulf Pycon Talk
Peter Skomoroch
 
PPTX
Presentation 2 Spring 2016 FINAL fat cut (1)
Michael Vistine
 
PDF
Mpi4py
A Jorge Garcia
 
PDF
Clustering Manual for Parallel Computing
Mamun Ahmed
 
PDF
Building SuperComputers @ Home
Abhishek Parolkar
 
PDF
Maxwell siuc hpc_description_tutorial
madhuinturi
 
PDF
Complete MPICH2 Clustering Manual in Ubuntu
Minhas Kamal
 
PDF
Introduction to GPUs in HPC
inside-BigData.com
 
PDF
Building A Linux Cluster Using Raspberry PI #2!
A Jorge Garcia
 
ODP
Parallel Programming on the ANDC cluster
Sudhang Shankar
 
PPT
Intermachine Parallelism
Sri Prasanna
 
PDF
Parallel Computing Example with Raspberry Pi Cluster
Herpiko Dwi Aguno
 
PPTX
Designing High-Performance and Scalable Middleware for HPC, AI and Data Science
Object Automation
 
PDF
Introduction to HPC Programming Models - EUDAT Summer School (Stefano Markidi...
EUDAT
 
PPTX
Bigdata workshop february 2015
clairvoyantllc
 
PPTX
High performance computing for research
Esteban Hernandez
 
Raspberry Pi Cluster Test Bed
Bradford Bazemore
 
Senior Design: Raspberry Pi Cluster Computing
Ralph Walker II
 
Introduction to MPI Basics easy way.pptx
imareebkhan25
 
Cluster Setup Manual Using Ubuntu and MPICH
Misu Md Rakib Hossain
 
Elasticwulf Pycon Talk
Peter Skomoroch
 
Presentation 2 Spring 2016 FINAL fat cut (1)
Michael Vistine
 
Mpi4py
A Jorge Garcia
 
Clustering Manual for Parallel Computing
Mamun Ahmed
 
Building SuperComputers @ Home
Abhishek Parolkar
 
Maxwell siuc hpc_description_tutorial
madhuinturi
 
Complete MPICH2 Clustering Manual in Ubuntu
Minhas Kamal
 
Introduction to GPUs in HPC
inside-BigData.com
 
Building A Linux Cluster Using Raspberry PI #2!
A Jorge Garcia
 
Parallel Programming on the ANDC cluster
Sudhang Shankar
 
Intermachine Parallelism
Sri Prasanna
 
Parallel Computing Example with Raspberry Pi Cluster
Herpiko Dwi Aguno
 
Designing High-Performance and Scalable Middleware for HPC, AI and Data Science
Object Automation
 
Introduction to HPC Programming Models - EUDAT Summer School (Stefano Markidi...
EUDAT
 
Bigdata workshop february 2015
clairvoyantllc
 
High performance computing for research
Esteban Hernandez
 
Ad

MPI Raspberry pi 3 cluster

  • 1. MPI Raspberry pi 3 & Hadoop cluster 4 nodes
  • 2. MPI Raspberry pi cluster  Why MPI  Hardware Needed  Software Needed  Machine File  MPI Code  Python Code  Output  Finished product
  • 3. Why MPI?  Scientist during World War II used the same method of Parallel Computing to solve mathematical problems in the Manhattan Project.  reduce the amount of time it took to solve a large mathematical problem.  the people who performed these calculations were called computers.  How far can you go with MPI?
  • 4. Hardware you need X4 Raspberry pi 3 X5 Ethernet cat 5 cables X4 micro USB cables X4 8 GB micro SD One 5 port Ethernet switch Option but recommended one 4 port USB power hub
  • 5. Software Needed  MPICH  MPI4PY  Python to MPI interpreter
  • 11. Hadoop  Hadoop is a framework of tools based on Java that supports running Big Data.  Inspired from Google’s GFS and MapReduce.  It is open source.  Relatively cheap to build.  Has Fault Tolerant System.
  • 12. Parts of Hadoop  Hadoop consists of two main elements,  HDFS (Hadoop Distributed File System)  Name Node (Store the directories of all files in the cluster)  Data Node (Store Data)  MapReduce  Job Tracker(Responsible for taking requests from client and assign to TaskTracker)  Task Tracker(Perform MapReduce in DataNodes/Slave)
  • 13. HDFS (Hadoop Distributed File System)
  • 15. Use of Hadoop in Tech World  Amazon  Facebook  Netflix  eBay  Twitter etc.