gLite Administration Workshop, Slides
1 like598 views
The document outlines an agenda for a gLite administration workshop. It will cover installing and configuring core gLite services including the site BDII, batch system, and CREAM Compute Element. Attendees will work in teams to set up a basic grid site infrastructure on provided machines following the presented order of services. Support will be provided to assist any teams experiencing issues.
gLite Administration Workshop, Slides
- 1. gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus gLite Administration Workshop Overview GridKa Summer School 2010 Grid middleware Plan for Today Stefan Freitag, Florian Feldhaus Services Robotics Research Institute TU Dortmund September 8, 2010
- 2. Contents gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus 1 Overview Overview Grid middleware 2 Grid middleware Plan for Today Services 3 Plan for Today 4 Brief introduction to gLite services
- 3. Overview gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus Monday Overview Grid Computing and Cloud Computing, an Overview (T. Cass) Grid middleware Plan for Today Tuesday Services gLite Introduction Course (M. Sober) Today gLite Administration Course (S. Freitag, F. Feldhaus)
- 4. Time table gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus 11:00 Brief introduction Overview 11:30 site BDII Grid 12:30 Lunch (northern canteen) middleware 13:45 Batch system Plan for Today 14:45 CREAM Compute Element Services 15:30 Coffee break 16:00 User Interface 17:00 Advanced exercises and site testing 18:30 End of the workshop
- 5. Why Grid middlewares? gLite Adminis- tration Workshop 2010 Resource sharing in early days of computing S. Freitag, F. Feldhaus 1 only few scientists had access to compute and storage Overview resources Grid middleware 2 resources only available at a very limited number of sites Plan for Today Services If you were not working at such a site → What a pity!
- 6. Why Grid middlewares? gLite Adminis- tration Maybe you have heard... Workshop 2010 If the mountain won’t come to Mohammed, Mohammed must S. Freitag, F. Feldhaus go to the mountain. Overview Consequences Grid middleware Plan for Today Scientists moved to the places where the required Services resources were available. Resource sharing allowed multiple scientists to access the resources at the same time
- 7. Why Grid middlewares? - Intervention gLite Adminis- tration Workshop 2010 You could ask: ”Why did those scientists move? If I was a one S. Freitag, F. Feldhaus of those scientists I would have simply used something like ssh?” Overview Grid Brief historical overview middleware Plan for Today rsh originated as part of the BSD Unix operating system Services in 1983 Version 1 of the ssh protocol was designed in 1995 Until protocols for remote shells were developed, there was no other choice
- 8. Why Grid middlewares? gLite Adminis- tration Over time resources became affordable for research Workshop 2010 facilities → more resources and more locations S. Freitag, F. Feldhaus Site B Overview Grid Site A middleware Plan for Today Site D Services Site C As some scientists had/ still have no access to local resources a way for sharing resources in the research community was required.
- 9. Why Grid middlewares? gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus Fosters checklist: Overview A Grid Grid middleware Plan for Today coordinates resources that are not subject to centralized Services control It integrates/ coordinates resources and users that exist within different control domains.
- 10. Why Grid middlewares? gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus Fosters checklist: Overview A Grid Grid middleware uses standard, open, general-purpose protocols and Plan for Today interfaces Services delivers nontrivial qualities of service. → allows its constituent resources to be used in a coordinated fashion to deliver various qualities of service.
- 11. Grid middleware implementations gLite Adminis- tration Workshop 2010 gLite S. Freitag, F. Feldhaus Overview Grid middleware Plan for Today NAREGI (National Research Grid Initiative), Japan Services
- 12. Grid middleware implementations gLite Adminis- tration Workshop 2010 ARC (Advanced Resource Connector) S. Freitag, F. Feldhaus Overview Grid middleware Globus Toolkit Plan for Today Services UNICORE (Uniform Interface to Compute Resources)
- 13. Grid middleware gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus Most services implemented by Grid middlewares can be assigned to one or more of the following categories: Overview Grid Service categories middleware Plan for Today Execution management, Services Data management, Information services and Security
- 14. Globus Toolkit 5 gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus Overview Grid middleware Plan for Today Services Figure: Globus Toolkit 5 Architecture
- 15. gLite gLite Adminis- tration Workshop 2010 S. Freitag, F. Execution Management Feldhaus Overview Workload Management System Grid CREAM Compute Element middleware Plan for Today Batch system Services Information services siteBDII TopLevelBDII
- 16. gLite gLite Adminis- tration Workshop 2010 S. Freitag, F. Data management Feldhaus Overview LFC (File Catalog) Grid dCache Storage Element middleware Plan for Today CASTOR (CERN Advanced Storage manager) Services DPM (Disk Pool Manager) Security GSI (Grid Security Infrastructure)
- 17. gLite Interaction gLite Adminis- tration is more complex than you can imagine. Workshop 2010 S. Freitag, F. Feldhaus Overview Grid middleware Plan for Today Services Figure: EGEEgLiteJobSubmissionSchema Thanks to Maarten Litmaath
- 18. Plan for today gLite Adminis- tration Workshop 2010 Focus: Installation & Configuration S. Freitag, F. Feldhaus of gLite services required for a site offering access to compute resources Overview Execution Management Grid middleware CREAM Compute Element Plan for Today Batchsytem (Torque server and one worker node) Services Information Services siteBDII Additional services User Interface VO Box Accounting service (APEL node) LFC (File catalog)
- 19. Worker Node gLite Adminis- tration Workshop 2010 S. Freitag, F. eager beaver of the Grid site Feldhaus all jobs submitted to the site are executed on worker nodes Overview the worker nodes are linked to a batch system server Grid middleware Plan for Today Installed software Services Batch system software (Torque mom) CLI/ libraries for e.g. file catalog and storage element operations (MPI)
- 20. Batch System Server gLite Adminis- tration Workshop 2010 S. Freitag, F. consists of different queues Feldhaus may vary in wallclocktime, max. resource allocation,. . . Overview job management Grid middleware assigning jobs to worker nodes, start/ stop jobs Plan for Today workernode management Services Installed software Torque Batchsystem server Maui Scheduler
- 21. Compute Element gLite Adminis- tration Workshop 2010 Contact point for WMS and end users to submit jobs to S. Freitag, F. the Grid site Feldhaus Authorizes remote users for job submission Overview Grid Acts on behalf of the users and submits jobs via the batch middleware system client. Plan for Today Services Installed software Tomcat (CREAM CE is using this) Accounting service (APEL) Information provider (GIP) Batchsystem client
- 22. siteBDII gLite Adminis- tration Workshop 2010 S. Freitag, F. Contact point for TopLevelBDIIs Feldhaus yep, there is not only one! Overview Queries local gLite services for information Grid middleware In both cases LDAP is used Plan for Today Services Installed software openLDAP GridMap: http://gridmap.cern.ch GStat 2.0: http://gstat-prod.cern.ch/gstat/geo/openlayers
- 23. VO Box gLite Adminis- tration Workshop 2010 S. Freitag, F. allows virtual organizations to run their own specific Feldhaus services and directly access their local software area at a Overview Grid site. Grid middleware (usually) software managers are able to connect to the Plan for Today VoBox via gsissh Services Installed software gsissh client (Port 1975 is default) gLite CLI/ libraries
- 24. APEL node gLite Adminis- tration Workshop 2010 Publishes accounting information to the Grid via S. Freitag, F. ActiveMQ Feldhaus Information is taken from a local database Overview Compute Elements fill the database by Grid middleware parsing the batch system accounting logs Plan for Today MySQL inserts Services Installed software MySQL Java APEL RSS: Link Accounting Portal: http://www3.egee.cesga.es/
- 25. Plan for today gLite Adminis- tration Workshop 2010 S. Freitag, F. Group to small teams of 4 - 6 people Feldhaus Each team will act as a SysAdmin team of a grid site. Overview Open http://gkswiki.fzk.de/index.php5 in your Grid middleware browser Plan for Today Select gLite administration workshop with hands-on Services Now it is your turn! Support If you experience problems while setting up the services, Florian and I will join your team and assist.
- 26. Hints gLite Adminis- tration Workshop Hint 1 2010 Make an initial sketch of your site setup containing an S. Freitag, F. Feldhaus assignment of grid services to hosts (and administrators) Overview Grid Hint 2 middleware Plan for Today We suggest to install the basic services in the following order Services siteBDII Batchsytem Compute Element To anyone interested How to establish a storage resource at your grid site is shown tomorrow in the dCache administrations workshop.
- 27. gLite Adminis- tration Workshop 2010 S. Freitag, F. Feldhaus Our experience Overview Do not expect that everything works ouf-of-the-box. Grid middleware Plan for Today For now Services Thanks for your attention! Questions !? Access to your machines: ssh -p24 [email protected]