Slide05 Message Passing Architecture
2 likes5,551 views
This document discusses message passing architectures. The key points are: 1) Message passing architectures allow processors to communicate data without a global memory by sending messages. Each processor has local memory and communicates via messages. 2) Important factors in message passing networks are link bandwidth and network latency. 3) Processes running on different processors use external channels to exchange messages, while processes on the same processor use internal channels. This avoids the need for synchronization.
Slide05 Message Passing Architecture
- 2. Introduction • The architecture is used to communicate data among a set of processors without the need for a global memory. • Each PE has its own local memory and communicates with other PEs using message.
- 3. Message Passing Systems 104 MESSAGE PASSING ARCHITECTURE P1 M1 P2 M2 Pn Mn Link 1 Link 2 Link n Interconnection Network Figure 5.1 Message passing systems. networks have received considerable attention over the years. As shown in Chapter 2, two important factors must be considered in designing message passing interconnec- tion networks: link bandwidth and the network latency. The link bandwidth is defined as the number of bits that can be transmitted per unit of time (bits/s). Network latency is defined as the time to complete a message transfer through the network.
- 4. Message Passing Systems 104 MESSAGE PASSING ARCHITECTURE P1 M1 P2 M2 Pn Mn Link 1 Link 2 Link n Interconnection Network Figure 5.1 Message passing systems. networks have received considerable attention over the years. As shown in Chapter 2, two important factors must be considered in designing message passing interconnec- external channel tion networks: link bandwidth and the network latency. The link bandwidth is defined as the number of bits that can be transmitted per unit of time (bits/s). Network latency is defined as the time to complete a message transfer through the network.
- 5. MP Network • Normally, a static-type network • hypercubes • nearest-neighbor 2D & 3D mesh • Two important factors must be considered: • link bandwidth - the number of bits that can be transmitted per unit of times (bits/s) • network latency - the time to complete a message transfer through the network
- 6. Executing Program • The program is divided into concurrent processes; each is executed on a separate processor. • If the no. of processes is larger than the number of processors, then more than one process will have to be executed on a processor in a time-shared fashion. 104 MESSAGE PASSING ARCHITECTURE P1 M1 P2 M2 Pn Mn Link 1 Link 2 Link n Interconnection Network Figure 5.1 Message passing systems. networks have received considerable attention over the years. As shown in Chapter 2, two important factors must be considered in designing message passing interconnec- tion networks: link bandwidth and the network latency. The link bandwidth is defined as the number of bits that can be transmitted per unit of time (bits/s). Network latency
- 7. Process Communication • Processes running on a given processor use what is called internal channels to exchange messages among themselves. • Processes running on different processors use the external channels to exchange messages.
- 8. Data Exchanged • Data exchanged among processors cannot be shared; it is rather copied (using send/ receive messages). • An important advantage of this form of data exchange is the elimination of the need for synchronization constructs, such as semaphores, which results in performance improvement.
- 9. More advantages • A message passing scheme offers flexibility in accommodating a large number of processors in addition to being readily scalable. • A given node can execute more than one process, each at a given time.
- 10. 5.2 ROUTING IN MESSAGE PASSING NETWORKS 105 Outside World P1 m1 P2 m4 P3 m2 m3 P4 Figure 5.2 An example of a message passing system. Three types of granularity can be distinguished. These are: 1. Coarse granularity: Each process holds a large number of sequential instruc-
- 11. Process Granularity • It’s a parameter which describes the size of a process in a message passing system. computation time Process Granularity = communication time
- 12. Granularity Types • There are 3 types of granularity: • Coarse - each process holds a large number of sequential instructions and takes a substantial amount of time to execute. • Medium - a middle ground where communication overhead is reduced • Fine - each process contains a few sequential instructions • Use mostly medium or coarse granularity
- 13. Routing in MP Networks • Involves the identification of a set of permissible paths that may be used by a message to reach its destination, and a function (η) that selects one path from the set of permissible paths.
- 14. Routing Techniques • Adaptive • the path taken by the message depends on network conditions for a given source and destination pair • Deterministic (oblivious) • determines the path using only the source and destination regardless of the network conditions - simple but bandwidth inefficient
- 15. Routing for Broadcasting & Multicasting • Communication operations: unicast and collective • Unicast - communicate to only a single destination • Collective - several routing operations are defined, broadcast & multicast are the most widely used.
- 16. Broadcast Multicast one-to-all operation one-to-many operation distribute data during uses in large-scale computation of a multiprocessors, inc. distributed memory parallel search algorithm program routing algorithms must be deadlock-free little time and short path is desirable
- 17. of these resources in a strict monotonic order. This restricted way for using network resources prevents the occurrence of circular wait, and hence prevents the occur- rence of deadlock. The channel dependency graph (CDG) is a technique used to develop a deadlock-free routing algorithm. A CDG is a directed graph D ¼ G(C, E ), where the vertex set C consists of all the unidirectional channels in the network and the set of edges E includes all the pairs of connected channels, 000 110 111 001 010 100 011 101 110 011 101 110 100 101 111 010 100 111 100 111 001 010 011 001 110 101 011 010 000 001 Figure 5.3 Hypercube broadcast tree-based communication.
- 18. Routing Potential Problems • Deadlock • When two messages each hold the resources required by the other in order to move, both message will be blocked. • Livelock • A situation in which a message keeps going around the network and never reaches its destination. • Starvation • A node is said to suffer from starvation if it has a message to inject into the network but is never allowed to do so.
- 19. Message Passing vs Shared Memory • Shared memory systems may be easier to program, but the difficult to scale up to a large number of processors. • If scalability to larger and larger systems was to continue, systems had to use message passing techniques. • However, a number of problems associated with message passing systems: communication overhead and difficulty of programming.