Open MPI 2
Download as PPT, PDF2 likes1,557 views
The document discusses different types of message passing routines in MPI including collective, synchronous, non-blocking, and asynchronous routines. Collective routines involve sending/receiving messages to/from a group of processes and include operations like broadcast, gather, scatter, reduce, and barrier. Synchronous routines return only after message transfer completes while non-blocking routines return immediately without waiting for completion. Asynchronous routines return before transfer finishes by using message buffers to store messages.
![Example
In the following code, size of send buffer is given by 100 *
<number of processes> and 100 contiguous elements are
send to each process:
main (int argc, char *argv[]) {
int size, *sendbuf, recvbuf[100]; /* for each process */
MPI_Init(&argc, &argv); /* initialize MPI */
MPI_Comm_size(MPI_COMM_WORLD, &size);
sendbuf = (int *)malloc(size*100*sizeof(int));
.
MPI_Scatter(sendbuf,100,MPI_INT,recvbuf,100,MPI_INT,0,
MPI_COMM_WORLD);
.
MPI_Finalize(); /* terminate MPI */
}
2a.10](https://image.slidesharecdn.com/slides2a-130504060132-phpapp01/85/Open-MPI-2-10-320.jpg)
![2a.13
Gather Example
To gather items from group of processes into process 0, using
dynamically allocated memory in root process:
int data[10]; /*data to be gathered from
processes*/
MPI_Comm_rank(MPI_COMM_WORLD, &myrank); /* find rank */
if (myrank == 0) {
MPI_Comm_size(MPI_COMM_WORLD, &grp_size); /*find group
size*/
buf = (int *)malloc(grp_size*10*sizeof (int)); /*alloc.
mem*/
}
MPI_Gather(data,10,MPI_INT,buf,grp_size*10,MPI_INT,0,MPI_COMM_
WORLD) ;
…
MPI_Gather() gathers from all processes, including root.](https://image.slidesharecdn.com/slides2a-130504060132-phpapp01/85/Open-MPI-2-13-320.jpg)
![2a.17
Sample MPI program with
collective routines
#include “mpi.h”
#include <stdio.h>
#include <math.h>
#define MAXSIZE 1000
void main(int argc, char *argv) {
int myid, numprocs, data[MAXSIZE], i, x, low, high, myresult, result;
char fn[255];
char *fp;
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
MPI_Comm_rank(MPI_COMM_WORLD,&myid);
if (myid == 0) { /* Open input file and initialize data */
strcpy(fn,getenv(“HOME”));
strcat(fn,”/MPI/rand_data.txt”);
if ((fp = fopen(fn,”r”)) == NULL) {
printf(“Can’t open the input file: %snn”, fn);
exit(1);
}
for(i = 0; i < MAXSIZE; i++) fscanf(fp,”%d”, &data[i]);
}
MPI_Bcast(data, MAXSIZE, MPI_INT, 0, MPI_COMM_WORLD); /* broadcast data */
x = n/nproc; /* Add my portion Of data */
low = myid * x;
high = low + x;
for(i = low; i < high; i++)
myresult += data[i];
printf(“I got %d from %dn”, myresult, myid); /* Compute global sum */
MPI_Reduce(&myresult, &result, 1, MPI_INT, MPI_SUM, 0, MPI_COMM_WORLD);
if (myid == 0) printf(“The sum is %d.n”, result);
MPI_Finalize();
}](https://image.slidesharecdn.com/slides2a-130504060132-phpapp01/85/Open-MPI-2-17-320.jpg)
Open MPI 2
- 1. 2a.1 Message-Passing Computing More MPI routines: Collective routines Synchronous routines Non-blocking routines ITCS 4/5145 Parallel Computing, UNC-Charlotte, B. Wilkinson, 2009.
- 2. 2a.2 Collective message-passing routines Have routines that send message(s) to a group of processes or receive message(s) from a group of processes Higher efficiency than separate point-to-point routines although routines not absolutely necessary.
- 3. 2a.3 Collective Communication Involves set of processes, defined by an intra-communicator. Message tags not present. Principal collective operations: • MPI_Bcast() - Broadcast from root to all other processes • MPI_Gather() - Gather values for group of processes • MPI_Scatter() - Scatters buffer in parts to group of processes • MPI_Alltoall() - Sends data from all processes to all processes • MPI_Reduce() - Combine values on all processes to single value • MPI_Reduce_scatter() - Combine values and scatter results • MPI_Scan()- Compute prefix reductions of data on processes • MPI_Barrier() - A means of synchronizing processes by stopping each one until they all have reached a specific “barrier” call.
- 4. MPI broadcast operation 2a.4 Sending same message to all processes in communicator. Multicast - sending same message to defined group of processes.
- 6. Basic MPI scatter operation 2a.6 Sending each element of an array in root process to a separate process. Contents of ith location of array sent to ith process.
- 8. • Simplest scatter would be as illustrated which one element of an array is sent to different processes. • Extension provided in the MPI_Scatter() routine is to send a fixed number of contiguous elements to each process. 2a.8
- 9. Scattering contiguous groups of elements to each process 2a.9
- 10. Example In the following code, size of send buffer is given by 100 * <number of processes> and 100 contiguous elements are send to each process: main (int argc, char *argv[]) { int size, *sendbuf, recvbuf[100]; /* for each process */ MPI_Init(&argc, &argv); /* initialize MPI */ MPI_Comm_size(MPI_COMM_WORLD, &size); sendbuf = (int *)malloc(size*100*sizeof(int)); . MPI_Scatter(sendbuf,100,MPI_INT,recvbuf,100,MPI_INT,0, MPI_COMM_WORLD); . MPI_Finalize(); /* terminate MPI */ } 2a.10
- 11. Gather 2.11 Having one process collect individual values from set of processes.
- 13. 2a.13 Gather Example To gather items from group of processes into process 0, using dynamically allocated memory in root process: int data[10]; /*data to be gathered from processes*/ MPI_Comm_rank(MPI_COMM_WORLD, &myrank); /* find rank */ if (myrank == 0) { MPI_Comm_size(MPI_COMM_WORLD, &grp_size); /*find group size*/ buf = (int *)malloc(grp_size*10*sizeof (int)); /*alloc. mem*/ } MPI_Gather(data,10,MPI_INT,buf,grp_size*10,MPI_INT,0,MPI_COMM_ WORLD) ; … MPI_Gather() gathers from all processes, including root.
- 14. 2a.14 Reduce Gather operation combined with specified arithmetic/logical operation. Example: Values could be gathered and then added together by root:
- 16. 2a.16 Reduce - operations MPI_Reduce(*sendbuf,*recvbuf,count,datatype,op,root,comm) Parameters: *sendbuf send buffer address *recvbuf receive buffer address count number of send buffer elements datatype data type of send elements op reduce operation. Several operations, including MPI_MAX Maximum MPI_MIN Minimum MPI_SUM Sum MPI_PROD Product root root process rank for result comm communicator
- 17. 2a.17 Sample MPI program with collective routines #include “mpi.h” #include <stdio.h> #include <math.h> #define MAXSIZE 1000 void main(int argc, char *argv) { int myid, numprocs, data[MAXSIZE], i, x, low, high, myresult, result; char fn[255]; char *fp; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); if (myid == 0) { /* Open input file and initialize data */ strcpy(fn,getenv(“HOME”)); strcat(fn,”/MPI/rand_data.txt”); if ((fp = fopen(fn,”r”)) == NULL) { printf(“Can’t open the input file: %snn”, fn); exit(1); } for(i = 0; i < MAXSIZE; i++) fscanf(fp,”%d”, &data[i]); } MPI_Bcast(data, MAXSIZE, MPI_INT, 0, MPI_COMM_WORLD); /* broadcast data */ x = n/nproc; /* Add my portion Of data */ low = myid * x; high = low + x; for(i = low; i < high; i++) myresult += data[i]; printf(“I got %d from %dn”, myresult, myid); /* Compute global sum */ MPI_Reduce(&myresult, &result, 1, MPI_INT, MPI_SUM, 0, MPI_COMM_WORLD); if (myid == 0) printf(“The sum is %d.n”, result); MPI_Finalize(); }
- 18. Collective routines General features • Performed on a group of processes, identified by a communicator • Substitute for a sequence of point-to-point calls • Communications are locally blocking • Synchronization is not guaranteed (implementation dependent) • Some routines use a root process to originate or receive all data • Data amounts must exactly match • Many variations to basic categories • No message tags are needed 2a.18
- 19. 2a.19 Barrier Block process until all processes have called it. Synchronous operation. MPI_Barrier(comm) Communicator
- 20. 2a.20 Synchronous Message Passing Routines that return when message transfer completed. Synchronous send routine • Waits until complete message can be accepted by the receiving process before sending the message. In MPI, MPI_SSend() routine. Synchronous receive routine • Waits until the message it is expecting arrives. In MPI, actually the regular MPI_recv() routine.
- 21. 2a.21 Synchronous Message Passing Synchronous message-passing routines intrinsically perform two actions: • They transfer data and • They synchronize processes.
- 22. 2a.22 Synchronous Ssend() and recv() using 3-way protocol Process 1 Process 2 Ssend(); recv(); Suspend Time process Acknowledgment MessageBoth processes continue (a) When send() occurs before recv() Process 1 Process 2 recv(); Ssend(); Suspend Time process Acknowledgment MessageBoth processes continue (b) When recv() occurs before send() Request to send Request to send
- 23. 2a.23 Parameters of synchronous send (same as blocking send) MPI_Ssend(buf, count, datatype, dest, tag, comm) Address of Number of items Datatype of Rank of destination Message tag Communicator send buffer to send each item process
- 24. 2a.24 Asynchronous Message Passing • Routines that do not wait for actions to complete before returning. Usually require local storage for messages. • More than one version depending upon the actual semantics for returning. • In general, they do not synchronize processes but allow processes to move forward sooner. • Must be used with care.
- 25. 2a.25 MPI Definitions of Blocking and Non- Blocking • Blocking - return after their local actions complete, though the message transfer may not have been completed. Sometimes called locally blocking. • Non-blocking - return immediately (asynchronous) Non-blocking assumes that data storage used for transfer not modified by subsequent statements prior to being used for transfer, and it is left to the programmer to ensure this. Blocking/non-blocking terms may have different interpretations in other systems.
- 26. 2a.26 MPI Nonblocking Routines • Non-blocking send - MPI_Isend() - will return “immediately” even before source location is safe to be altered. • Non-blocking receive - MPI_Irecv() - will return even if no message to accept.
- 27. 2a.27 Nonblocking Routine Formats MPI_Isend(buf,count,datatype,dest,tag,comm,request) MPI_Irecv(buf,count,datatype,source,tag,comm, request) Completion detected by MPI_Wait() and MPI_Test(). MPI_Wait() waits until operation completed and returns then. MPI_Test() returns with flag set indicating whether operation completed at that time. Need to know whether particular operation completed. Determined by accessing request parameter.
- 28. 2a.28 Example To send an integer x from process 0 to process 1 and allow process 0 to continue: MPI_Comm_rank(MPI_COMM_WORLD, &myrank); /* find rank */ if (myrank == 0) { int x; MPI_Isend(&x,1,MPI_INT, 1, msgtag, MPI_COMM_WORLD, req1); compute(); MPI_Wait(req1, status); } else if (myrank == 1) { int x; MPI_Recv(&x,1,MPI_INT,0,msgtag, MPI_COMM_WORLD, status); }
- 29. 2a.29 How message-passing routines return before message transfer completed Process 1 Process 2 send(); recv(); Message buffer Read message buffer Continue process Time Message buffer needed between source and destination to hold message:
- 30. 2a.30 Asynchronous (blocking) routines changing to synchronous routines • Message buffers only of finite length • A point could be reached when send routine held up because all available buffer space exhausted. • Then, send routine will wait until storage becomes re-available - i.e. routine will behave as a synchronous routine.
- 31. 2a.31 Next topic • Parallel Algorithms Other MPI features will be introduced as we need them.