![TERAGRID RESOURCE PROVIDERS
• Indiana University - Big Red - IBM BladeCenter JS21 Cluster[7]
• Louisiana Optical Network Initiative (LONI)[8]
• National Center for Atmospheric Research (NCAR)
• National Center for Supercomputing Applications (NCSA)
• Pittsburgh Supercomputing Center (PSC) operated by University of
Pittsburgh and Carnegie Mellon University.
• San Diego Supercomputer Center (SDSC)
• Texas Advanced Computing Center (TACC)
11-03-2018Grid Computing Systems and Resource Management
33](https://image.slidesharecdn.com/finalppt2-180311064557/85/Grid-Computing-Systems-and-Resource-Management-33-320.jpg)
![REFERENCES
[1] Kai Hwang, Jack Dongarra Geoffrey Fox, Distributed and Cloud Computing: From Parallel
Processing to the Internet of Things, Morgan Kaufmann Publishers Inc. San Francisco,
CA, USA,1st Edition, eBook ISBN:9780128002049, Paperback ISBN: 9780123858801.
[2] https://www.nsf.gov/
[3] http://www.garudaindia.in/
[4] http://www.eu-datagrid.org/
[5] http://www.computerworld.com/article/2552339/networking/open-grid-
servicesarchitecture.html
[6] Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in China”, IEEE,
Dec. 2005
11-03-2018Grid Computing Systems and Resource Management
43](https://image.slidesharecdn.com/finalppt2-180311064557/85/Grid-Computing-Systems-and-Resource-Management-43-320.jpg)
Grid Computing Systems and Resource Management
- 1. GRID COMPUTING SYSTEMS AND RESOURCE MANAGEMENT Presented by: Souparnika Padaki Patil 11-03-2018 1 Grid Computing Systems and Resource Management
- 2. CONTENTS • Introduction • CPU Scavenging and Virtual Supercomputers. • Open Grid Services Architecture (OGSA). • Data-Intensive Grid Service Models. • National Grids and International Projects. • NSF TeraGrid in the United States. • DataGrid in the European Union. • The ChinaGrid Design Experiences. • Resource Management and Job Scheduling. • Grid Resource Monitoring with CGSP • Resource Brokering with Gridbus 11-03-2018Grid Computing Systems and Resource Management 2
- 3. INTRODUCTION • The idea of the grid was pioneered by Ian Foster, Carl Kesselman and Steve Tuecke in 2001. • The grid is a metacomputing infrastructure that brings together computers to form a large collection of compute, storage, and network resources. • The goal of grid computing is to explore fast solutions for large- scale computing problems. 11-03-2018Grid Computing Systems and Resource Management 3
- 4. CPU SCAVAGING AND VIRTUAL SUPER COMPUTERS It is the process of grid resource aggregation from local and remote sources. It consists of two parts: 1. CPU Scavenging and Virtual Supercomputers. 2. Virtual Organization 11-03-2018Grid Computing Systems and Resource Management 4
- 5. CPU SCAVAGING • The concept of creating a “grid” from the unused resources in a network of computers is known as CPU scavenging. • At present, many volunteer computing grids are built using the CPU scavenging model. • The most famous example is the SETI@Home, which applied over 3 million computers to achieve 23.37 TFlpos as of Sept. 2001 • In practice, these virtual grids can be viewed as virtual supercomputers. 11-03-2018Grid Computing Systems and Resource Management 5
- 6. EXAMPLES OF CPU SCAVAGING Fastest virtual supercomputers: 1. BOINC – 16.92 PFLOPS 2. SETI@Home 0.741 PFLOPS 3. GIMPS 0.313 PFLOPS 4. MilkyWay@Home 0.217 PFLOPS 11-03-2018Grid Computing Systems and Resource Management 6
- 7. VIRTUAL ORGNIZATION • The grid is a distributed system integrated from shared resources to form a virtual organization (VO). • The VO offers dynamic cooperation built over multiple physical organizations. • The virtual resources contributed by these real organizations are managed autonomously. 11-03-2018Grid Computing Systems and Resource Management 7
- 8. EXAMPLE OF VIRTUAL ORGNIZATION 11-03-2018Grid Computing Systems and Resource Management 8
- 9. FEATURES OF VIRTUAL ORGNIZATION 11-03-2018Grid Computing Systems and Resource Management 9
- 10. OPEN GRID SERVICE ARCHIECURE(OGSA) OGSA defines standards for • what Grid services are • what they should be capable of • what type of technologies they should be based on. • OGSA does not give a technical and detailed specification. They use WSDL 11-03-2018Grid Computing Systems and Resource Management 10
- 11. ARCHITECTURE OF OGSA It Comprised of 4 main layers • Grid Applications Layer • OGSA Architected Grid Services Layer ( core, program execution and data services) • Web Service Layer ( including OGSI) • Physical and Logical Resources Layer 11-03-2018Grid Computing Systems and Resource Management 11
- 12. OSGA FRAMEWORK 11-03-2018Grid Computing Systems and Resource Management 12
- 13. OPEN GRID SERVICES INFRASTRUCTURE (OGSI) • OGSA services are built around OGSI mechanism • OGSI specification defines grid services and builds upon web services. • OGSI is based on WSRF ( web services Resources Framework) • The Globus Toolkit is an implementation of OGSI 11-03-2018Grid Computing Systems and Resource Management 13
- 14. SERVICES IN THE WEB AND THE GRID OGSA, OGSI 11-03-2018Grid Computing Systems and Resource Management 14
- 15. 11-03-2018Grid Computing Systems and Resource Management 15 • OGSA: (The Definition) is the blueprints the architect creates to show how the building looks like. • OGSI: (The Specification) is the structural design that the engineer creates to support the architect's vision of the building. • GT: (The implementation) is the bricks, cement and beams used to build the building with the engineer's specifications.
- 16. DATA-INTENSIVE GRID SERVICE MODEL • Data intensive grid service models need to handle large volume of data. • So the grid systems designed must be able to discover, transfer, and manipulate these massive data sets • Desirable properties • Less time-consuming • Low storage costs • High-speed data movement 11-03-2018Grid Computing Systems and Resource Management 16
- 17. DATA-INTENSIVE GRID SERVICE MODEL… • Methods to handle data efficiently: • Data Replication (High availability) • Grid Data Access Models  Monadic model  Hierarchical model  Federation model  Hybrid model 11-03-2018Grid Computing Systems and Resource Management 17
- 18. DATA REPLICATION • Data caching : Data access operation in database is called caching • Data replication :  same data is scattered and stored in multiple grid locations  Users access data from multiple locations parallel based on the locality of reference ( Google search) • Benefits  Data availability is improved  One data storage becomes backup for another data storage • Two types of replication  Dynamic  Static 11-03-2018Grid Computing Systems and Resource Management 18
- 19. GRID DATA ACCESS MODELS Monadic model • Used when centralized data repository is required • All data stored in repository • Repository ( all data) is replicated within grid • To access data User submits request to central repository Permission given based on prior registration • Fault tolerance, performance, reliability very poor 11-03-2018Grid Computing Systems and Resource Management 19
- 20. Hierarchical model • Data centers designed as first, second…levels • First level data replicated to second level etc. • Data in each level accessed by its own grid users 11-03-2018Grid Computing Systems and Resource Management 20
- 21. Federation model • Multiple databases • Geographically distributed • Known as “mesh data access model” • Authorized specific location users can only access respective dB 11-03-2018Grid Computing Systems and Resource Management 21
- 22. Hybrid model • Combination of hierarchical and federation models • Needs high bandwidth network • Uses Grid FTP protocol 11-03-2018Grid Computing Systems and Resource Management 22
- 23. NATIONAL GRIDS AND INTERNATIONAL PROJECTS • Like supercomputers, national grids are mainly funded through government sources. • National grids are developed to promote:  research discovery  middleware products  utility computing 11-03-2018Grid Computing Systems and Resource Management 23
- 24. NATIONAL GRID PROJECT • GARUDA(Global Access to Resource Using Distributed Architecture) is India's Grid Computing initiative connecting 17 cities across the country. • The 70 participating institutes in this nationwide project include all the IITs and C-DAC centers and other major institutes in India. • From April 2008 the Foundation phase is in progress with an aim to include more users’ applications. 11-03-2018Grid Computing Systems and Resource Management 24
- 25. GARUDA ACCESS MATHODS Anyone can access garuda in two ways: • Command line Interface: • Useful for Advanced Users • Requirements: Valid IGCA Certificate, Garuda User Account. • Web Interface. • A Web Interface for Job Submission. • Browse Resources. • Can Select the desired OS • http://192.168.60.40/GridPortal1.3/ 11-03-2018Grid Computing Systems and Resource Management 25
- 26. GARUDA PARTNERS Current Participation • Total of 45 institutions • 36 research & academic institutions in 17 cities • ERNET-HQ in Delhi • 8 centers of C-DAC 11-03-2018Grid Computing Systems and Resource Management 26
- 27. GARUDA COLLABORATIONS • Space Application Centre, Ahmedabad • Disaster Management • Grid Middleware for Satellite Grids • Indian Institute of Science, Bangalore • Climate Modeling • Indian Institute of Technology, Mumbai • Computational Fluid Dynamics Package on PARAM Padma • University of Pune • Quantum Chemistry, Materials Modeling, Bioinformatics • INDMOL Molecular Package developed 11-03-2018Grid Computing Systems and Resource Management 27
- 28. NATIONAL GRID PROJECTS OF DIFFERENT COUNTRIES • GridPP (UK) • CNGrid (China) • D-Grid (Germany) • GARUDA (India) • VECC (Calcutta, India) • IsraGrid (Israel) • INFN Grid (Italy) • National Grid Service (UK) • Open Science Grid (USA) • TeraGrid (USA) 11-03-2018Grid Computing Systems and Resource Management 28
- 29. INTERNATIONAL GRID PROJECTS • Grid applications cannot be restricted to geographical boundaries. • These projects promote volunteer computing, utility computing, and specific software applications that utilizes grid infrastructure. • International grids involve both government and industrial funding. • The European Union has been a major player in grid computing. 11-03-2018Grid Computing Systems and Resource Management 29
- 30. NSF TERAGRID IN UNITED STATES • TeraGrid was an e-Science grid computing infrastructure combining resources at eleven partner sites. • The project started in 2001 and operated from 2004 through 2011. • The TeraGrid resources included more than a petaflop of computing capability and more than 30 petabytes of online and archival data storage. • TeraGrid construction was also made possible through corporate partnerships with Sun Microsystems, IBM, Intel Corporation, Qwest Communications, Juniper Networks, Myricom, Hewlett-Packard Company, and Oracle Corporation. 11-03-2018Grid Computing Systems and Resource Management 30
- 31. TERAGRID HARDWARE COMPONENTS • High-end compute hardware • Intel/Linux clusters • IBM POWER3 and POWER4 clusters • SUN visualization systems • Cray XT3 • IBM Blue Gene/L • Large-scale storage systems • hundreds of terabytes for secondary storage • Visualization hardware • Very high-speed network backbone (40Gb/s) • bandwidth for rich interaction and tight coupling 11-03-2018Grid Computing Systems and Resource Management 31
- 32. TERAGRID ARCHITECTURE • TeraGrid resources are integrated through a service-oriented architecture. • Computational resources run a set of software packages called "Coordinated TeraGrid Software and Services" (CTSS). • TeraGrid uses a 10 Gigabits per second dedicated fiber-optical backbone network, with hubs in Chicago, Denver, and Los Angeles • CTSS provides a familiar user environment on all TeraGrid systems, allowing scientists to more easily port code from one system to another. • CTSS includes:- • Globus Toolkit • Account management software • Set of compilers • Programming tools • Environment variables 11-03-2018Grid Computing Systems and Resource Management 32
- 33. TERAGRID RESOURCE PROVIDERS • Indiana University - Big Red - IBM BladeCenter JS21 Cluster[7] • Louisiana Optical Network Initiative (LONI)[8] • National Center for Atmospheric Research (NCAR) • National Center for Supercomputing Applications (NCSA) • Pittsburgh Supercomputing Center (PSC) operated by University of Pittsburgh and Carnegie Mellon University. • San Diego Supercomputer Center (SDSC) • Texas Advanced Computing Center (TACC) 11-03-2018Grid Computing Systems and Resource Management 33
- 34. DATA GRID IN EUROPEAN UNION • The European DataGrid Project (EDG) was funded by the European Union. • This grid aims to build the next-generation, high-throughput, production-quality grid infrastructure. • It supports I/O-intensive experiments in high-energy physics, earth observation, and bioinformatics. • The system moves and replicates data at high speeds from one geographical site to another 11-03-2018Grid Computing Systems and Resource Management 34
- 35. EU DATA GRID PROJECT OBJECTIVE • Use Grid technology to develop a sustainable computing model for effective share of computing resources and data for large scientific communities • Collaborate with and complement other European and US projects • Contribute to Open Standards and international bodies 11-03-2018Grid Computing Systems and Resource Management 35
- 36. EU DATA GRID PROJECT PARTNERS • CERN – International (Switzerland/France) • CNRS - France • ESA/ESRIN – International (Italy) • INFN - Italy • NIKHEF – The Netherlands • PPARC - UK 11-03-2018Grid Computing Systems and Resource Management 36
- 37. ARCHITECTURE OF THE EUROPEAN DATA GRID 11-03-2018Grid Computing Systems and Resource Management 37
- 38. RESOURCE MANAGEMENT AND JOB SCHEDULING • In a grid system, resources are usually autonomous • Organizations set their own Resource Management System (RMS) have there own resource management policies. • RMS can be in upper as well as lower level of organization. • In upper level it is considered as resource consumer and in lower level as resource provider. 11-03-2018Grid Computing Systems and Resource Management 38
- 39. COMPONENTS OF RMS 11-03-2018Grid Computing Systems and Resource Management 39
- 40. GRID RESOURCE MONITORING WITH CGSP • In ChinaGrid the monitoring system is essential to keep complex distributed system efficient. • CGSP is a grid middleware developed for the construction and evolution of ChinaGrid. • CGSP is designed to be an adaptable, stream-integrated grid monitoring system. • CGSP guarantees the integrity and uniformity of ChinaGrid platform by a global monitoring system. • Function modules in CGSP are: • Portal • Grid Developing Environment • Information Center • Security 11-03-2018Grid Computing Systems and Resource Management 40
- 41. RESOURCE BROKERING WITH GRIDBUS • The resource broker takes care of the user desire to automate the search for resources rather than keep track of the previously known ones. • The resource broker adds a layer of abstraction by making direct user and resource interaction unnecessary and allowing job submission accessible and user friendly. 11-03-2018Grid Computing Systems and Resource Management 41
- 42. 11-03-2018Grid Computing Systems and Resource Management 42
- 43. REFERENCES [1] Kai Hwang, Jack Dongarra Geoffrey Fox, Distributed and Cloud Computing: From Parallel Processing to the Internet of Things, Morgan Kaufmann Publishers Inc. San Francisco, CA, USA,1st Edition, eBook ISBN:9780128002049, Paperback ISBN: 9780123858801. [2] https://www.nsf.gov/ [3] http://www.garudaindia.in/ [4] http://www.eu-datagrid.org/ [5] http://www.computerworld.com/article/2552339/networking/open-grid- servicesarchitecture.html [6] Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in China”, IEEE, Dec. 2005 11-03-2018Grid Computing Systems and Resource Management 43
- 44. IEEE Paper Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in China”, IEEE, Dec. 2005 The focus of this paper is mainly on  current status of ChinaGrid project.  The impact of ChinaGrid project.  five different grid computing applications  Bioinformatics Grid  Computational Fluid Dynamics Grid  Image Processing Grid  Course Online Grid  Massive Data Processing Grid 11-03-2018Grid Computing Systems and Resource Management 44
- 45. 11-03-2018Grid Computing Systems and Resource Management 45