CloudComputing_UNIT1.pdf
- 1. Cloud Computing Page 5 UNIT -1 Scalable Computing over the Internet High-Throughput Computing-HTC HTC paradigm pays more attention to high-flux computing. The main application for high-flux computing is in Internet searches and web services by millions or more users simultaneously. The performance measures high throughput or the number of tasks completed per unit of time. HTC technology needs to improve batch processing speed, and also address the acute problems of cost, energy savings, security, and reliability at many data and enterprise computing centers Computing Paradigm Distinctions Centralized computing o This is a computing paradigm by which all computer resources are centralized in one physical system. o All resources (processors, memory, and storage) are fully shared and tightly coupled within one integrated OS. o Many data centers and supercomputers are centralized systems, but they are used in parallel, distributed, and cloud computing applications. • Parallel computing In parallel computing, all processors are either tightly coupled with centralized shared memory or loosely coupled with distributed memory . Interprocessor communication is accomplished through shared memory or via message passing. A computer system capable of parallel computing is commonly known as a parallel computer Programs running in a parallel computer are called parallel programs. The process of writing parallel programs is often referred to as parallel programming • Distributed computing A distributed system consists of multiple autonomous computers, each having its own private memory, communicating through a computer network. Information exchange in a distributed system is accomplished through message passing. A computer program that runs in a distributed system is known as a distributed program. The process of writing distributed programs is referred to as distributed programming. Distributed computing system uses multiple computers to solve large-scale problems over the Internet using a centralized computer to solve computational problems. • Cloud computing An Internet cloud of resources can be either a centralized or a distributed computing system. The cloud applies parallel or distributed computing, or both. Clouds can be built with physical or virtualized resources over large data centers that are centralized or distributed. Cloud computing can also be a form of utility computing or service computing
- 2. Cloud Computing Page 6 Degrees of Parallelism Bit-level parallelism (BLP) : o converts bit-serial processing toword-level processing gradually. Instruction-levelparallelism (ILP) o the processor executes multiple instructions simultaneously rather thanonly one instruction at a time. o ILP is executed through pipelining, superscalarcomputing, VLIW (very long instruction word) architectures, and multithreading. o ILP requiresbranch prediction, dynamic scheduling, speculation, and compiler support to work efficiently. Data-level parallelism (DLP) o DLP through SIMD (single instruction, multipledata) and vector machines using vector or array types of instructions. o DLP requires even more hardwaresupport and compiler assistance to work properly. Task-level parallelism (TLP): o Ever since the introduction of multicoreprocessors and chip multiprocessors (CMPs), we have been exploring TLP o TLP is far from beingvery successful due to difficulty in programming and compilation of code for efficient execution onmulticore CMPs. Utility Computing o Utility computing focuses on a business model in which customers receive computing resources from a paid service provider. All grid/cloud platforms are regarded as utility service providers. The Internet of Things (IoT) o Traditional Internet connects machines to machines or web pages to web pages. o IoT was introduced in 1999 at MIT o networked interconnection of everyday objects, tools, devices, or computers o a wireless network of sensors that interconnect all things in our daily life. o Three communication patterns co-exist: namely H2H (human-to-human), H2T (human- tothing),and T2T (thing-to-thing). o connect things (including human and machine objects) at any time and any place intelligently with low cost o IPv6 protocol, 2128 IP addresses are available to distinguish all the objects on Earth, including all computers and pervasive devices o IoT needs to be designed to track 100 trillion static or moving objects simultaneously. o IoT demands universal addressability of all of the objects or things. o The dynamic connections will grow exponentially into a new dynamic network of networks, called the Internet of Things (IoT). Cyber-Physical Systems o A cyber-physical system (CPS) is the result of interaction between computational processes and the physical world. o CPS integrates “cyber” (heterogeneous, asynchronous) with “physical” (concurrent and information-dense) objects
- 3. Cloud Computing Page 7 o CPS merges the “3C” technologies of computation, communication, and control into an intelligent closed feedback system o IoT emphasizes various networking connections among physical objects, while the CPS emphasizes exploration of virtual reality (VR) applications in the physical world SYSTEM MODELS FOR DISTRIBUTED AND CLOUD COMPUTING o Distributed and cloud computing systems are built over a large number of autonomous computer nodes. These node machines are interconnected by SANs, LANs, or WANs o A massive system is with millions of computers connected to edge networks. o Massive systems are considered highly scalable o massive systems are classified into four groups: clusters, P2P networks, computing grids, and Internet clouds Computing cluster o A computing cluster consists of interconnected stand-alone computers which work cooperatively as a single integrated computing resource. Cluster Architecture o the architecture consists of a typical server cluster built around a low-latency, high bandwidth interconnection network. o build a larger cluster with more nodes, the interconnection network can be built with multiple levels of Gigabit Ethernet, Myrinet, or InfiniBand switches. o Through hierarchical construction using a SAN, LAN, or WAN, one can build scalable clusters with an increasing number of nodes o cluster is connected to the Internet via a virtual private network (VPN) gateway. o gateway IP address locates the cluster
- 4. Cloud Computing Page 8 o Clusters have loosely coupled node computers. o All resources of a server node are managed by their own OS. o Most clusters have multiple system images as a result of having many autonomous nodes under different OS control Single-System Image -Cluster o an ideal cluster should merge multiple system images intoa single-system image (SSI) o acluster operating system or some middleware have to support SSI at various levels, including the sharing of CPUs, memory, and I/O across all cluster nodes. o illusion created by software or hardware that presents a collection of resources as one integrated, powerful resource o SSI makes the cluster appear like a single machine to the user. o A cluster with multiple system images is nothing but a collection of independent computers. Hardware, Software, and Middleware Support –Cluster o Clusters exploring massive parallelism are commonly known as MPPs –Massive Parallel Processing o The building blocks are computer nodes (PCs, workstations, servers, or SMP), special communication software such as PVM or MPI, and a network interface card in each computer node. o Most clusters run under the Linux OS. o nodes are interconnected by a high-bandwidth network o Special cluster middleware supports are needed to create SSI or high availability (HA). o all distributed memory to be shared by all servers by forming distributed shared memory (DSM). o SSI features are expensive o achieving SSI, many clusters are loosely coupled machines o virtual clusters are created dynamically, upon user demand Grid Computing A web service such as HTTP enables remote access of remote web pages computing grid offers an infrastructure that couples computers, software/middleware, special instruments, and people and sensors together Enterprises or organizations present grids as integrated computing resources. They can also beviewed as virtual platforms to support virtual organizations. The computers used in a grid are primarilyworkstations, servers, clusters, and supercomputers Peer-to-Peer Network-P2P P2P architecture offers a distributed model of networked systems. P2P network is client-oriented instead of server-oriented In a P2P system, every node acts as both a client and a server
- 5. Cloud Computing Page 9 Peer machines are simply client computers connected to the Internet. All client machines act autonomously to join or leave the system freely. This implies that no master-slave relationship exists among the peers. No central coordination or central database is needed. The system is self-organizing with distributed control. P2P two layer of abstractions as given in the figure Each peer machine joins or leaves the P2P network voluntarily Only the participatingpeers form the physical network at any time. Physical network is simply an ad hoc networkformed at various Internet domains randomly using the TCP/IP and NAI protocols. Peer-to-Peer Network-Overlay network Data items or files are distributed in the participating peers. Based on communication or file-sharing needs, the peer IDs form an overlay network at the logical level. When a new peer joins the system, its peer ID is added as a node in the overlay network. When an existing peer leaves the system, its peer ID is removed from the overlay network automatically. An unstructured overlay network is characterized by a random graph. There is no fixed route to send messages or files among the nodes. Often, flooding is applied to send a query to all nodes in an unstructured overlay, thus resulting in heavy network traffic and nondeterministic search results. Structured overlay networks follow certain connectivity topology and rules for inserting and removing nodes (peer IDs) from the overlay graph Cloud Computing A cloud is a pool of virtualized computer resources. A cloud can host a variety of different workloads, including batch-style backend jobs and interactive and user-facing applications.” Cloud computing applies a virtualized platform with elastic resources on demand by provisioning hardware, software, and data sets dynamically The Cloud Landscape Infrastructure as a Service (IaaS) This model puts together infrastructures demanded by users—namely servers, storage, networks, and the data center fabric.
- 6. Cloud Computing Page 10 The user can deploy and run on multiple VMs running guest OSes on specific applications. The user does not manage or control the underlying cloud infrastructure, but can specify when to request and release the needed resources. Platform as a Service (PaaS) This model enables the user to deploy user-built applications onto a virtualized cloud platform. PaaS includes middleware, databases, development tools, andsome runtime support such as Web 2.0 and Java. The platform includes both hardware andsoftware integrated with specific programming interfaces. The provider supplies the API andsoftware tools (e.g., Java, Python, Web 2.0, .NET). The user is freed from managing the cloudinfrastructure. Software as a Service (SaaS) This refers to browser-initiated application software overthousands of paid cloud customers. The SaaS model applies to business processes, industryapplications, consumer relationship management (CRM), enterprise resources planning (ERP),human resources (HR), and collaborative applications. On the customer side, there is no upfrontinvestment in servers or software licensing. On the provider side, costs are rather low, comparedwith conventional hosting of user applications Internet clouds offer four deployment modes: private, public, managed, and hybrid SOFTWARE ENVIRONMENTS FOR DISTRIBUTED SYSTEMSAND CLOUDS Service-Oriented Architecture (SOA) In grids/web services, Java, and CORBA, an entity is, respectively, a service, a Java object, and a CORBA distributed object in a variety of languages. These architectures build on the traditional seven Open Systems Interconnection (OSI) layers that provide the base networking abstractions. On top of this we have a base software environment, which would be o .NET or Apache Axis for web services, o the Java Virtual Machine for Java, and a broker network for CORBA On top of this base environment one would build a higher level environment reflecting the special features of the distributed computing environment. SOAapplies to building grids, clouds, grids of clouds, clouds of grids, clouds of clouds (also known asinterclouds), SS (sensor service : A large number of sensors provide data-collectionservices (ZigBee device, a Bluetoothdevice, WiFi access point, a personal computer, a GPA, or a wireless phoneetc Filter services : to eliminate unwanted raw data, in orderto respond to specific requests from the web, the grid, or web services Layered Architecture for Web Services and Grids Entity Interfaces Java methodinterfaces correspond to the Web Services Description Language (WSDL),
- 7. Cloud Computing Page 11 CORBA interface - definition language (IDL) specifications These interfaces are linked with customized, high-level communication systems: SOAP, RMI, and IIOP These communication systems support features including particular message patterns (such as Remote Procedure Call or RPC), fault recovery, and specialized routing. Communication systems are built on message-oriented middleware (enterprise bus) infrastructure such as Web-Sphere MQ or Java Message Service (JMS) Cases of fault tolerance- the features in the Web Services Reliable Messaging (WSRM) Security -reimplements the capabilities seen in concepts such as Internet Protocol Security (IPsec) Several models with, for example, JNDI (Jini and Java Naming and DirectoryInterface) illustrating different approaches within the Java distributed object model. The CORBA TradingService, UDDI (Universal Description, Discovery, and Integration), LDAP (Lightweight Directory Access Protocol), and ebXML (Electronic Business using eXtensibleMarkup Language earlier years, CORBA and Java approaches were used in distributed systems rather than today’sSOAP, XML, or REST (Representational State Transfer). Web Services and Tools REST approach: delegates most ofthe difficult problems to application (implementation-specific) software. In a web services language minimal information in the header, and the message body (that is opaque to genericmessage processing) carries all the needed information. architectures are clearly more appropriatefor rapid technology environments. REST can use XML schemas but not those that are part of SOAP; “XML overHTTP” is a popular design choice in this regard. Above the communication and managementlayers, we have the ability to compose new entities or distributed programs by integrating severalentities together. CORBA and Java: the distributed entities are linked with RPCs, and the simplest way to buildcomposite applications is to view the entities as objects and use the traditional ways of linking themtogether. For Java, this could be as simple as writing a Java program with method calls replaced byRemote Method Invocation (RMI), CORBA supports a similar model with a syntax reflecting theC++ style of its entity (object) interfaces.
- 8. Cloud Computing Page 12 Parallel and Distributed Programming Models PERFORMANCE, SECURITY, AND ENERGY EFFICIENCY Performance Metrics: In a distributed system, performance is attributed to a large numberof factors. System throughput is often measured in MIPS, Tflops (tera floating-point operations persecond), or TPS (transactions per second). Systemoverhead is often attributed to OS boot time, compile time, I/O data rate, and the runtime support systemused. Other performance-related metrics include the QoS for Internet and web services; systemavailability and dependability; and security resilience for system defense against network attacks Dimensions of Scalability Any resource upgrade ina system should be backward compatible with existing hardware and software resources. System scaling can increase or decrease resources depending on many practicalfactors Size scalability This refers to achieving higher performance or more functionality by increasingthe machine size. The word “size” refers to adding processors, cache, memory, storage, or I/Ochannels. The most obvious way to determine size scalability is to simply count the number ofprocessors installed.
- 9. Cloud Computing Page 13 Not all parallel computer or distributed architectures are equally sizescalable. For example, the IBM S2 was scaled up to 512 processors in 1997. But in 2008, theIBMBlueGene/L system scaled up to 65,000 processors. • Software scalability This refers to upgrades in the OS or compilers, adding mathematical andengineering libraries, porting new application software, and installing more user- friendlyprogramming environments. Some software upgrades may not work with large systemconfigurations. Testing and fine-tuning of new software on larger systems is a nontrivial job. • Application scalability This refers to matching problem size scalability with machine sizescalability. Problem size affects the size of the data set or the workload increase. Instead of increasingmachine size, users can enlarge the problem size to enhance system efficiency or cost-effectiveness. • Technology scalability This refers to a system that can adapt to changes in building technologies,such as the component and networking technologies Whenscaling a system design with new technology one must consider three aspects: time, space, andheterogeneity. (1) Time refers to generation scalability. When changing to new-generation processors,one must consider the impact to the motherboard, power supply, packaging and cooling,and so forth. Based on past experience, most systems upgrade their commodity processors everythree to five years. (2) Space is related to packaging and energy concerns. Technology scalabilitydemands harmony and portability among suppliers. (3) Heterogeneity refers to the use ofhardware components or software packages from different vendors. Heterogeneity may limit thescalability. Amdahl’s Law Let the program has been parallelized or partitioned for parallelexecution on a cluster of many processing nodes. Assume that a fraction α of the code must be executedsequentially, called the sequential bottleneck. Therefore, (1 − α) of the code can be compiledfor parallel execution by n processors. The total execution time of the program is calculated byα T + (1 − α)T/n, where the first term is the sequential execution time on a single processor and thesecond term is the parallel execution time on n processing nodes. I/O time or exception handling timeis also not included in the following speedup analysis. Amdahl’s Law states that the speedup factorof using the n-processor system over the use of a single processor is expressed by:
- 10. Cloud Computing Page 14 the code is fully parallelizable with α = 0. As the cluster becomes sufficiently large, that is, n →∞, S approaches 1/α, an upper bound on the speedup S. this upper bound is independentof the cluster size n. The sequential bottleneck is the portion of the code that cannot be parallelized. Gustafson’s Law To achieve higher efficiency when using a large cluster, we must consider scaling the problem sizeto match the cluster capability. This leads to the following speedup law proposed by John Gustafson(1988), referred as scaled-workload speedup. Let W be the workload in a given program. When using an n-processor system, the user scales the workload to W′ = αW + (1 − α)nW.Scaled workload W′ is essentially the sequential execution time on a single processor. The parallelexecution time of a scaled workload W′ on n processors is defined by a scaled-workload speedupas follows: Network Threats and Data Integrity ENERGY EFFICIENCY IN DISTRIBUTED COMPUTING Primary performance goals in conventional parallel and distributed computing systems are highperformance and high throughput, considering some form of performance reliability (e.g.,
- 11. Cloud Computing Page 15 fault toleranceand security). However, these systems recently encountered new challenging issues includingenergy efficiency, and workload and resource outsourcing Energy Consumption of Unused Servers: To run a server farm (data center) a company has to spend a huge amount of money for hardware,software, operational support, and energy every year. Therefore, companies should thoroughlyidentify whether their installed server farm (more specifically, the volume of provisioned resources)is at an appropriate level, particularly in terms of utilization. Reducing Energy in Active Servers: In addition to identifying unused/underutilized servers for energy savings, it is also necessary toapply appropriate techniques to decrease energy consumption in active distributed systems with negligibleinfluence on their performance. Application Layer: Until now, most user applications in science, business, engineering, and financial areas tend toincrease a system’s speed or quality. By introducing energy-aware applications, the challenge is todesign sophisticated multilevel and multi-domain energy management applications without hurtingperformance. Middleware Layer: The middleware layer acts as a bridge between the application layer and the resource layer. Thislayer provides resource broker, communication service, task analyzer, task scheduler, securityaccess, reliability control, and information service capabilities. It is also responsible for applyingenergy-efficient techniques, particularly in task scheduling. Resource Layer: The resource layer consists of a wide range of resources including computing nodes and storageunits. This layer generally interacts with hardware devices and the operating system; therefore, itis responsible for controlling all distributed resources in distributed computing systems. Dynamic power management (DPM) and dynamic voltage-frequency scaling (DVFS) are two popular methods incorporated into recent computer hardware systems. In DPM, hardware devices, such as the CPU, have the capability to switch from idle mode to one or more lower power modes. In DVFS, energy savings are achieved based on the fact that the power consumptionin CMOS circuits has a direct relationship with frequency and the square of the voltage supply. Network Layer: Routing and transferring packets and enabling network services to the resource layer are the mainresponsibility of the network layer in distributed computing systems. The major challenge to buildenergy-efficient networks is, again, determining how to measure, predict, and create a balancebetween energy consumption and performance. Clustering for Massive Parallelism. A computer cluster is a collection of interconnected stand-alone computers which can work together collectively and cooperatively as a single integrated computing resource pool. Clustering explores massive parallelism at the job level and achieves high availability (HA) through stand-alone operations. Benefits of computer clusters and massively parallel processors (MPPs) include
- 12. Cloud Computing Page 16 Scalable performance, HA, fault tolerance, modular growth, and use of commodity components. These features can sustain the generation changes experienced in hardware, software, and network components. Design Objectives of Computer Clusters Scalability: Clustering of computers is based on the concept of modular growth. To scale a cluster from hundreds of uniprocessor nodes to a supercluster with 10,000 multicore nodes is a nontrivial task. The scalability could be limited by a number of factors, such as the multicore chip technology, cluster topology, packaging method, power consumption, and cooling scheme applied. Packaging Cluster nodes can be packaged in a compact or a slack fashion. In a compact cluster, the nodes are closely packaged in one or more racks sitting in a room, and the nodes are not attached to peripherals (monitors, keyboards, mice, etc.). In a slack cluster, the nodes are attached to their usual peripherals (i.e., they are complete SMPs, workstations, and PCs), and they may be located in different rooms, different buildings, or even remote regions. Packaging directly affects communication wire length, and thus the selection of interconnection technology used. While a compact cluster can utilize a high-bandwidth, low-latency communication network that is often proprietary, nodes of a slack cluster are normally connected through standard LANs or WANs. Control A cluster can be either controlled or managed in a centralized or decentralized fashion. A compact cluster normally has centralized control, while a slack cluster can be controlled either way. In a centralized cluster, all the nodes are owned, controlled, managed, and administered by a central operator. In a decentralized cluster, the nodes have individual owners. This lack of a single point of control makes system administration of such a cluster very difficult. It also calls for special techniques for process scheduling, workload migration, checkpointing, accounting, and other similar tasks. Homogeneity A homogeneous cluster uses nodes from the same platform, that is, the same processor architecture and the same operating system; often, the nodes are from the same vendors. A heterogeneous cluster uses nodes of different platforms. Interoperability is an important issue in heterogeneous clusters. In a homogeneous cluster, a binary process image can migrate to another node and continue execution.
- 13. Cloud Computing Page 17 This is not feasible in a heterogeneous cluster, as the binary code will not be executable when the process migrates to a node of a different platform. Security Intracluster communication can be either exposed or enclosed. In an exposed cluster, the communication paths among the nodes are exposed to the outside world. An outside machine can access the communication paths, and thus individual nodes, using standard protocols (e.g., TCP/IP). Such exposed clusters are easy to implement, but have several disadvantages: • Being exposed, intracluster communication is not secure, unless the communication subsystemperforms additional work to ensure privacy and security. Outside communications may disrupt intracluster communications in an unpredictable fashion. Standard communication protocols tend to have high overhead. In an enclosed cluster, intracluster communication is shielded from the outside world, which alleviates the aforementioned problems. A disadvantage is that there is currently no standard for efficient, enclosed intracluster communication. Consequently, most commercial or academic clusters realize fast communications through one-of-a-kind protocols Fundamental Cluster Design Issues 1. Scalable Performance: Scaling of resources (cluster nodes, memory capacity, I/O bandwidth, etc.) leads to a proportional increase in performance. Both scale-up and scale- down capabilities are needed, depending on application demand or cost-effectiveness considerations. Clustering is driven by scalability 2. Single-System Image (SSI): A set of workstations connected by an Ethernet network is not necessarily a cluster. A cluster is a single system. 3. Availability Support: Clusters can provide cost-effective HA capability with lots of redundancy in processors, memory, disks, I/O devices, networks, and operating system images 4. Cluster Job Management: Clusters try to achieve high system utilization from traditional workstations or PC nodes that are normally not highly utilized. Job management software is required to provide batching, load balancing, parallel processing, and other functionality 5. Inter node Communication: The inter node physical wire lengths are longer in a cluster than in an MPP. A long wire implies greater interconnect network latency. But, longer
- 14. Cloud Computing Page 18 wires have more problems in terms of reliability, clock skew, and cross talking. These problems call for reliable and secure communication protocols, which increase overhead. Clusters often use commodity networks (e.g., Ethernet) with standard protocols such as TCP/IP. 6. Fault Tolerance and Recovery: Clusters of machines can be designed to eliminate all single points of failure. Through redundancy, a cluster can tolerate faulty conditions up to a certain extent. Heartbeat mechanisms can be installed to monitor the running condition of all nodes. In case of a node failure, critical jobs running on the failing nodes can be saved by failing over to the surviving node machines. Rollback recovery schemes restore the computing results through periodic checkpointing. 7. Cluster Family Classification:computer clusters are divided into three classes Compute clusters: o These are clusters designed mainly for collective computationover a single large job. The compute clusters do not handle many I/O operations, such as database services. When a single compute job requires frequent communication among the cluster nodes, the cluster must share a dedicated network, and thus the nodes are mostly homogeneous and tightly coupled. This type of clusters is also known as a Beowulf cluster High-Availability clusters HA (high-availability) o clusters are designed to be fault-tolerant and achieve HA of services. HA clusters operate with many redundant nodes to sustain faults or failures. Load-balancing clusters o These clusters shoot for higher resource utilization through load balancing among all participating nodes in the cluster. All nodes share the workload or function as a single virtual machine (VM). o Requests initiated from the user are distributed to all node computers to form a cluster. This results in a balanced workload among different machines, and thus higher resource utilization or higher performance. Middleware is needed to achieve dynamic load balancing by job or process migration among all the cluster nodes Basic Cluster Architecture simple cluster of computers built with commodity components supported with desired SSI features and HA capability commodity nodes are easy to replace or upgrade with new generations of hardware node operating systems should be designed for multiuser, multitasking, and multithreaded applications. nodes are interconnected by one or more fast commodity networks and use standard communication protocols network interface card is connected to the node’s standard I/O bus
- 15. Cloud Computing Page 19 When the processor or the operating system is changed, only the driver software needs to change. a cluster OS is not commercially available. Instead, we can deploy some cluster middleware to glue together all node platforms at the user space middleware offers HA services An SSI layer provides a single entry point, a single file hierarchy, a single point of control idealized cluster is supported by three subsystems conventional databasesand OLTP monitors A user interface subsystemis needed to combine the advantages of the web interface and the Windows GUI. cluster supports parallel programming based on standard languages and communication libraries using PVM, MPI, or OpenMP. The programming environment also includes tools for debugging, profiling, monitoring Resource Sharing in Clusters Clustering improves both availability and performance. Some HA clusters use hardware redundancy for scalable performance. The nodes of a cluster can be connected in one of three ways
- 16. Cloud Computing Page 20 Part (a) simply connects two or more autonomous computers via a LAN such as Ethernet. Part (b): shared-disk cluster Part (c) shared-memory cluster DESIGN PRINCIPLES OF COMPUTER CLUSTERS General-purpose computers and clusters of cooperative computers should be designed for scalability, availability, Single System Image, High Availability, Fault tolerance, and Rollback recovery Single System Image: A single system image is the illusion, created by software or hardware, that presents a collection of resources as an integrated powerful resource. SSI makes the cluster appear like a single machine to the user, applications, and network. A cluster with multiple system images is nothing but a collection of independent computers Single-System- Image Features Single System: The entire cluster is viewed by the users as one system, which has multiple processors. Single Control: Logically, an end user or system user utilizes services from one place with a single interface. Symmetry: A user can use a cluster service from any node. All cluster services and functionalities are symmetric to all nodes and all users, except those protected by access rights. Location Transparent: The user is not aware of the whereabouts of the physical device that eventually provides a service. Basic SSI Services A.Single Entry Point telnet cluster.usc.edu telnet node1.cluster.usc.edu
- 17. Cloud Computing Page 21 1. Four nodes of a cluster are used as host nodes to receive users’ login requests. 2. To log into the cluster a standard Unix command such as “telnet cluster.cs.hku.hk”, using the symbolic name of the cluster system is issued. 3. The symbolic name is translated by the DNS, which returns with the IP address 159.226.41.150 of the least-loaded node, which happens to be node Host1. 4. The user then logs in using this IP address. 5. The DNS periodically receives load information from the host nodes to make load- balancing translation decisions. B. Single File Hierarchy: xFS, AFS, Solaris MC Proxy The illusion of a single, huge file system image that transparently integrates local and global disks and other file devices (e.g., tapes). Files can reside on 3 types of locations in a cluster: Local storage - disk on the local node. Remote storage - disks on remote nodes. Stable storage - Persistent - data, once written to the stable storage, will stay there at least for a period of time (e.g., a week), even after the cluster shuts down. Fault tolerant - to some degree, by using redundancy and periodical backup to tapes. Three types of storage in a single file hierarchy. Solid lines show what process P can access and thedashed line shows what P may be able to access C. Single I/O, Networking, and Memory Space: To achieve SSI, we need a:
- 18. Cloud Computing Page 22 • single control point • single address space • single job management system • single user interface • single process control Single Networking: A properly designed cluster should behave as one system. Any process on any node can use any network and I/O device as though it were attached to the local node. Single networking means any node can access any network connection. Single Point of Control: The system administrator should be able to configure, monitor, test, and control the entire cluster and each individual node from a single point. Many clusters help with this through a system console that is connected to all nodes of the cluster Single Memory Space: Single memory space gives users the illusion of a big, centralized main memory, which in reality may be a set of distributed local memory spaces. Single I/O Address Space: A single I/O space implies that any node can access the RAIDs A cluster with single networking, single I/O space, single memory, and single point of control Other Services Single Job Management: All cluster jobs can be submitted from any node to a single job management system. GlUnix, Codine, LSF, etc. Single User Interface: The users use the cluster through a single graphical interface. Such an interface is available for workstations and PCs like CDE in Solaris/NT Single process spaceAll user processes created on various nodes form a single process space and share a uniform process identification scheme. A process on any node can create(e.g., through a UNIX fork) or communicate with (e.g., through signals, pipes, etc.) processes on remote nodes.
- 19. Cloud Computing Page 23 Middleware support for SSI clusteringSSI features aresupported by middleware developed at three cluster application levels: • Management level This level handles user applications and provides a job management system such as GLUnix, MOSIX, Load Sharing Facility (LSF), or Codine. • Programming levelThis level provides single file hierarchy (NFS, xFS, AFS, Proxy) and distributed shared memory (TreadMark, Wind Tunnel). • Implementation level This level supports a single process space, checkpointing, process migration, and a single I/O space. These features must interface with the cluster hardware and OS platform. Relationship among clustering middleware at the job management, programming, and implementation levels. High Availability through Redundancy: • Reliability measures how long a system can operate without a breakdown. • Availability indicates the percentage of time that a system is available to the user, that is, thepercentage of system uptime. • Serviceability refers to how easy it is to service the system, including hardware and softwaremaintenance, repair, upgrades, and so on. A system’s reliability is measured by the mean time to failure (MTTF), which is the average time of normal operation before the system (or a component of the system) fails. The metricfor serviceability is the mean time to repair (MTTR), which is the average time it takes to repair thesystem and restore it to working condition after it fails.
- 20. Cloud Computing Page 24 The availability of a system is defined by: Availability = MTTF/ (MTTF +MTTR) Failure is any event that prevents the system from normal operation • Unplanned failures The system breaks, due to an operating system crash, a hardware failure, anetwork disconnection, human operation errors, a power outage, and so on. All these are simplycalled failures. The system must be repaired to correct the failure. •Planned shutdownsThe system is not broken, but is periodically taken off normal operationfor upgrades, reconfiguration, and maintenance. Transient versus Permanent Failures A lot of failures are transient in that they occur temporarily and then disappear. They can be dealtwith without replacing any components. A standard approach is to roll back the system to a known state and start over. Permanent failures cannot be corrected by rebooting. Some hardwareor software component must be repaired or replaced. For instance, rebooting will not work ifthe system hard disk is broken. Partial versus Total Failures A failure that renders the entire system unusable is called a total failure. A failure that only affectspart of the system is called a partial failure if the system is still usable, even at a reduced capacity Redundancy Techniques Isolated Redundancy: A key technique to improve availability in any system is to use redundant components. When acomponent (the primary component) fails, the service it provided is taken over by another component(the backup component). Furthermore, the primary and the backup components should be isolatedfrom each other, meaning they should not be subject to the same cause of failure. Clustersprovide HA with redundancy in power supplies, fans, processors, memories, disks, I/O devices, networks,operating system images, and so on. In a carefully designed cluster, redundancy is also isolated. N-Version Programming to Enhance Software Reliability A common isolated-redundancy approach to constructing a mission-critical software system is calledN-version programming. The software is implemented by N isolated teams who may not even knowthe others exist. Different teams are asked to implement the software using different algorithms,programming languages, environment tools, and even platforms In a fault-tolerant system, the N versions all run simultaneously and their results are constantly compared. If the results differ, thesystem is notified that a fault has occurred. Fault-Tolerant Cluster Configurations: The cluster solution was targeted to provide availability support for two server nodes with threeascending levels of availability: hot
- 21. Cloud Computing Page 25 standby, active takeover, and fault-tolerant. The level of availabilityincreases from standby to active and fault-tolerant cluster configurations. The shorter is the recoverytime, the higher is the cluster availability. Failback refers to the ability of a recovered node returningto normal operation after repair or maintenance. Activeness refers to whether the node is used inactive work during normal operation. • Hot standby server clusters: In a hot standby cluster, only the primary node is actively doing all the useful work normally. The standby node is powered on (hot) and running some monitoring programs to communicate heartbeat signals to check the status of the primary node, but is not actively running other useful workloads. The primary node must mirror any data to shared disk storage, which is accessible by the standby node. The standby node requires a second copy of data. • Active-takeover clusters: In this case, the architecture is symmetric among multiple server nodes. Both servers are primary, doing useful work normally. Both failover and failback are often supported on both server nodes. When a node fails, the user applications fail over to the available node in the cluster. Depending on the time required to implement the failover, users may experience some delays or may lose some data that was not saved in the last checkpoint. • Failover cluster: When a component fails, this technique allows the remaining system to take over the services originally provided by the failed component. A failover mechanism mustprovide several functions, such as failure diagnosis, failure notification, and failure recovery.Failure diagnosis refers to the detection of a failure and the location of the failed componentthat caused the failure. A commonly used technique is heartbeat, whereby the cluster nodessend out a stream of heartbeat messages to one another. If the system does not receive thestream of heartbeat messages from a node, it can conclude that either the node or the networkconnection has failed. Recovery Schemes Failure recovery refers to the actions needed to take over the workload of a failed component. Thereare two types of recovery techniques. In backward recovery, the processes running on a cluster periodicallysave a consistent state (called a checkpoint) to a stable storage. After a failure, the systemis reconfigured to isolate the failed component, restores the previous checkpoint, and resumes normaloperation. This is called rollback. Backward recovery is relatively easy to implement in an application-independent, portable fashion If execution time is crucial,such as in real-time systems where the rollback time cannot be tolerated, a forward recovery schemeshould be used. With such a scheme, the system is not rolled back to the previous checkpoint upon afailure. Instead, the system utilizes the failure diagnosis information to reconstruct a valid system stateand continues execution. Forward recovery is application-dependent and may need extra hardware Checkpointing and Recovery Techniques
- 22. Cloud Computing Page 26 Checkpointingisthe process of periodically saving the state of an executing program to stable storage, from whichthe system can recover after a failure. Each program state saved is called a checkpoint. The disk filethat contains the saved state is called the checkpoint file. Checkpointing techniques are useful not only for availability, but also for program debugging, process migration, and load balancing Checkpointing can be realized by the operating system at the kernel level, where the OS transparentlycheckpoints and restarts processes A less transparent approach linksthe user code with a checkpointinglibrary in the user space. Checkpointing and restarting are handled by this runtime support. This approach is used widely because it has the advantage thatuser applications do not have to be modified. A third approach requires the user (or the compiler) to insert checkpointingfunctions in the application; thus, the application has to be modified, and the transparencyis lost. However, it has the advantage that the user can specify where to checkpoint. This is helpful to reduce checkpointing overhead. Checkpointing incurs both time and storage overheads. Checkpoint Overheads During a program’s execution, its states may be saved many times. This is denoted by the time consumedto save one checkpoint. The storage overhead is the extra memory and disk space requiredfor checkpointing. Both time and storage overheads depend on the size of the checkpoint file. Choosing an Optimal Checkpoint Interval The time period between two checkpoints is called the checkpoint interval. Making the interval larger can reduce checkpoint time overhead. Wong and Franklin derived an expression for optimal checkpoint interval Optimal checkpoint interval = Square root (MTTF x tc)/h MTTF is the system’s mean time to failure. This MTTF accounts the time consumed to save one checkpoint, and h is the average percentage of normal computation performed in a checkpointinterval before the system fails. The parameter h is always in the range. After a system isrestored, it needs to spend h × (checkpoint interval) time to recompute. Incremental Checkpoint Instead of saving the full state at each checkpoint, an incremental checkpoint scheme saves only theportion of the state that is changed from the previous checkpoint In full-state checkpointing, only one checkpoint file needs to be kepton disk. Subsequent checkpoints simply overwrite this file. With incremental checkpointing, old filesneeded to be kept, because a state may span many files. Thus, the total storage requirement is larger Forked Checkpointing Most checkpoint schemes are blocking in that the normal computation is stopped while checkpointingis in progress. With enough memory, checkpoint overhead can be reduced by
- 23. Cloud Computing Page 27 making a copy ofthe program state in memory and invoking another asynchronous thread to perform the checkpointingconcurrently. A simple way to overlap checkpointing with computation is to use the UNIXfork( ) system call. The forked child process duplicates the parent process’s address space andcheckpoints it. Meanwhile, the parent process continues execution. Overlapping is achieved sincecheckpointing is disk-I/O intensive User-Directed Checkpointing The checkpoint overheads can sometimes be substantially reduced if the user inserts code (e.g., library or system calls) to tell the system when to save, what to save, and what not to save. What should be the exact contents of a checkpoint? It should contain just enough information to allow asystem to recover. The state of a process includes its data state and control state Checkpointing Parallel Programs The state of a parallel program is usually much larger than that of a sequential program, as it consists of the set of the states of individual processes, plus thestate of the communication network. Parallelism also introduces various timing and consistency problems Consistent Snapshot A global snapshot is called consistent if there is no message that is received by the checkpoint of one process, but not yet sent by another process. Graphically, this corresponds to the case that no arrow crosses a snapshot line from right to left Coordinated versus Independent Checkpointing Checkpointing schemes for parallel programs can be classified into two types. In coordinated checkpointing(also called consistent checkpointing), the parallel program is frozen, and all processes arecheckpointed at the same time. In independent checkpointing, the processes are checkpointed independentof one another. Cluster Job Scheduling and Management A Job Management System (JMS) should have three parts: A user server lets the user submit jobs to one or more queues, specify resource requirements for each job, delete a job from a queue, inquire about the status of a job or a queue. A job scheduler that performs job scheduling and queuing according to job types, resource requirements, resource availability, and scheduling policies. A resource manager that allocates and monitors resources, enforces scheduling policies, and collects accounting information. JMS Administration JMS should be able to dynamically reconfigure the cluster with minimal impact on the running jobs. The administrator’s prologue and epilogue scripts should be able to run before and after each job for security checking, accounting, and cleanup.
- 24. Cloud Computing Page 28 Users should be able to cleanly kill their own jobs. The administrator or the JMS should be able to cleanly suspend or kill any job. Clean means that when a job is suspended or killed, all its processes must be included. Otherwise some “orphan” processes are left in the system, wasting cluster resources and may eventually render the system unusable. Several types of jobs execute on a cluster. Serial jobs run on a single node. Parallel jobs use multiple nodes. Interactive jobs are those that require fast turnaround time, and their input/output is directed to a terminal. These jobs do not need large resources, and the users expect them to execute immediately, not made to wait in a queue. Batch jobs normally need more resources, such as large memory space and long CPU time. But they do not need immediate response. They are submitted to a job queue to be scheduled to run when the resource becomes available (e.g., during off hours). Multi-Job Scheduling Schemes Cluster jobs may be scheduled to run at a specific time (calendar scheduling) or when a particular event happens (event scheduling). Jobs are scheduled according to priorities based on submission time, resource nodes, execution time, memory, disk, job type, and user identity. With static priority, jobs are assigned priorities according to a predetermined, fixed scheme. A simple scheme is to schedule jobs in a first-come, first-serve fashion. Another scheme is to assign different priorities to users. With dynamic priority, the priority of a job may change over time.
- 25. Cloud Computing Page 29 Job Scheduling Issues and Schemes for Cluster Nodes Scheduling Modes Dedicated Mode: Only one job runs in the cluster at a time, and at most one process of the job is assigned to a node at a time. The single job runs until completion before it releases the cluster to run other jobs. Space Sharing: Multiple jobs can run on disjoint partitions (groups) of nodes simultaneously. At most one process is assigned to a node at a time. Although a partition of nodes is dedicated to a job, the interconnect and the I/O subsystem may be shared by all jobs. Time sharing : Multiple user processes are assigned to the same node. Time-sharing introduces the following parallel scheduling policies: Independent Scheduling (Independent): Uses the operating system of each cluster node to schedule different processes as in a traditional workstation. Gang Scheduling: Schedules all processes of a parallel job together. When one process is active, all processes are active.