Integration of Cloud and Grid Middleware at DGRZR
0 likes1,327 views
This document discusses integrating cloud middleware into the existing grid middleware stack at DGRZR. It describes how OpenNebula has been installed to manage virtual machines on DGRZR resources. Future plans include addressing open issues in user management, authorization, accounting and information systems when combining grid and cloud. The goal is to establish cloud middleware as a new pillar of the D-Grid software stack and expand cloud resources across allied universities.
![OpenNebula at DGRZR
Cloud and
Grid
Middleware at
DGRZR
S. Freitag
Created network definitions with MAC/IP mapping.
DGRZR
NAME = ”DGRZR W o r k er n o de s ”
D-Grid
Integration TYPE = FIXED
BRIDGE = e t h 0
LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 2 , MAC= 0 0 : 1 6 : 3 e : 6 f : d2 : 0 9 ]
LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 3 , MAC= 0 0 : 1 6 : 3 e : 5 b : 0 9 : c9 ]
LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 4 , MAC= 0 0 : 1 6 : 3 e : 1 4 : f f : b1 ]
LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 5 , MAC= 0 0 : 1 6 : 3 e : 2 7 : c6 : 0 4 ]
[...]](https://image.slidesharecdn.com/slides-100320115132-phpapp02/85/Integration-of-Cloud-and-Grid-Middleware-at-DGRZR-10-320.jpg)
![OpenNebula at DGRZR
Cloud and
Grid Created user accounts for D-Grid users
Middleware at
DGRZR ID USER PASSWORD ENABLE
S. Freitag [...]
2 ad0001 True
DGRZR
3 ad0002 True
D-Grid 4 ad0003 True
Integration
5 ad0004 True
6 ad0005 True
7 ad0006 True
8 ad0007 True
9 ad0008 True
10 ad0009 True
[...]
100 accounts per Virtual Organization (D-Grid recommendation:
200)
Supported VOs: at present 10, later 24](https://image.slidesharecdn.com/slides-100320115132-phpapp02/85/Integration-of-Cloud-and-Grid-Middleware-at-DGRZR-11-320.jpg)
![OpenNebula at DGRZR
Cloud and
Grid Workernode Template (not using context)
Middleware at
DGRZR
VCPU = 1 # usually 8
S. Freitag MEMORY = 512 # u s u a l l y ˜ 13 GByte
OS = [ b o o t l o a d e r = ”/ r o o t / b i n / domUloader . py ” ]
DGRZR
RAW = [ t y p e = ” xen ” ,
D-Grid
Integration d a t a = ” b o o t a r g s=”−−v e r b o s e −−e n t r y=xvda1 ”” ]
DISK = [
s o u r c e = ”<some path >/ w n s l 5 4 x 8 6 6 4 . img ” ,
t a r g e t = ” xvda ” , r e a d o n l y = ” no ” ]
DISK = [
t y p e = swap , s i z e = 1 0 2 4 , t a r g e t = ” xvdb ” ,
r e a d o n l y = ” no ” ]
DISK = [
type = ” b l o c k ” , c l o n e = ” yes ” , t a r g e t = ” xvdc ” ,
s o u r c e = ”/ dev / c c i s s / c0d0p4 ” , r e a d o n l y = ” no ” ]
NIC = [NETWORK=” d g r z r ” , IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 5 ]](https://image.slidesharecdn.com/slides-100320115132-phpapp02/85/Integration-of-Cloud-and-Grid-Middleware-at-DGRZR-12-320.jpg)
Integration of Cloud and Grid Middleware at DGRZR
- 1. Cloud and Grid Middleware at DGRZR S. Freitag Integration of Cloud and Grid Middleware at DGRZR DGRZR D-Grid International Symposium on Grid Computing 2010 Integration Stefan Freitag Robotics Research Institute Dortmund University of Technology March 12, 2010
- 2. Overview Cloud and Grid Middleware at DGRZR S. Freitag DGRZR D-Grid Integration 1 D-Grid Resource Center Ruhr 2 Clouds in the German Grid Initiative D-Grid
- 3. Introduction Cloud and Grid D-Grid Resource Center Ruhr Middleware at DGRZR S. Freitag DGRZR D-Grid Integration
- 4. Introduction Cloud and Grid Middleware at DGRZR S. Freitag D-Grid Resource Center Ruhr (DGRZR) DGRZR 256 Blades, Intel Xeon Dual CPU QuadCore, D-Grid Integration 16 GByte RAM Cluster runs SLES 10 SP3 with Xen 3.2 Kernel 100 TByte storage Since April 2008 in production as part of D-Grid infrastructure End of 2008: 25 TByte SFS (Lustre) storage extension
- 5. Services @ DGRZR Cloud and Grid Middleware at Site setup follows recommendations of D-Grid reference DGRZR installation1 (not 100%) S. Freitag Three compute middlewares DGRZR gLite 3.1 (lcg-CE) and 3.2 (CREAM-CE, BDII) D-Grid UNICORE 5 and 6 Integration Globus Toolkit 4.0.8 Two storage middlewares dCache 1.9.x OGSA-DAI 2.2 Additional LDAP for user management DNS, DHCP MySQL DB for OGSA-DAI All services run in Xen virtual machines 1 http://dgiref.d-grid.de/wiki/Introduction
- 6. D-Grid Services @ DGRZR Cloud and Grid Middleware at DGRZR Grid Compute Frontends Grid Storage Frontends S. Freitag UNICORE gLite WS OGSA dCache Grid VSite CE GRAM DAI SE Grid DGRZR Middle- Middle- ware dCache ware D-Grid Globus Toolkit Pool Integration File Local LRMS Torque & MAUI Databases Storage Systems Software Compute Cluster Online Storage Worker Nodes Fabric Fabric Virtualization Layer Figure: Pre-Cloud software stack
- 7. Extending DGRZR by Cloud Middleware Cloud and Grid Middleware at DGRZR Cloud Frontend Grid Compute Frontends Grid Storage Frontends S. Freitag UNICORE gLite WS OGSA dCache Grid VSite CE GRAM DAI SE Grid DGRZR Middle- Middle- ware dCache ware D-Grid Globus Toolkit Pool Integration File Local LRMS Torque & MAUI Databases Storage Systems Software Open Compute Cluster Online Storage Nebula Worker Nodes Fabric Fabric Virtualization Layer Figure: Current software stack including OpenNebula
- 8. OpenNebula at DGRZR Cloud and Grid Middleware at DGRZR S. Freitag Started with OpenNebula (ONE) 1.2 Currently running: slightly adapted ONE 1.4 SVN DGRZR D-Grid snapshot (January 2010) Integration Xen infrastructure and SSH transfer enabled Images/ templates for SL 4.8 and 5.4 (64bit) Grid workernodes In progress e.g. for gLite: lcg-CE, CREAM-CE, siteBDII Users interface with ONE via CLI one.grid.tu-dortmund.de supports OCCI via HTTP(S) (currently not in production use)
- 9. OpenNebula at DGRZR Cloud and Grid Middleware at DGRZR S. Freitag All blade servers are registered with OpenNebula DGRZR D-Grid one : ˜ # o n e h o s t l i s t Integration ID NAME RVM TCPU FCPU ACPU . . . STAT 0 udo−b l 1 1 0 1 0 800 700 700 on 1 udo−b l 1 1 0 2 2 800 0 0 on ... 246 udo−b l 6 3 0 7 0 800 98 98 on 247 udo−b l 6 3 0 8 0 800 99 99 on
- 10. OpenNebula at DGRZR Cloud and Grid Middleware at DGRZR S. Freitag Created network definitions with MAC/IP mapping. DGRZR NAME = ”DGRZR W o r k er n o de s ” D-Grid Integration TYPE = FIXED BRIDGE = e t h 0 LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 2 , MAC= 0 0 : 1 6 : 3 e : 6 f : d2 : 0 9 ] LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 3 , MAC= 0 0 : 1 6 : 3 e : 5 b : 0 9 : c9 ] LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 4 , MAC= 0 0 : 1 6 : 3 e : 1 4 : f f : b1 ] LEASES=[ IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 5 , MAC= 0 0 : 1 6 : 3 e : 2 7 : c6 : 0 4 ] [...]
- 11. OpenNebula at DGRZR Cloud and Grid Created user accounts for D-Grid users Middleware at DGRZR ID USER PASSWORD ENABLE S. Freitag [...] 2 ad0001 True DGRZR 3 ad0002 True D-Grid 4 ad0003 True Integration 5 ad0004 True 6 ad0005 True 7 ad0006 True 8 ad0007 True 9 ad0008 True 10 ad0009 True [...] 100 accounts per Virtual Organization (D-Grid recommendation: 200) Supported VOs: at present 10, later 24
- 12. OpenNebula at DGRZR Cloud and Grid Workernode Template (not using context) Middleware at DGRZR VCPU = 1 # usually 8 S. Freitag MEMORY = 512 # u s u a l l y ˜ 13 GByte OS = [ b o o t l o a d e r = ”/ r o o t / b i n / domUloader . py ” ] DGRZR RAW = [ t y p e = ” xen ” , D-Grid Integration d a t a = ” b o o t a r g s=”−−v e r b o s e −−e n t r y=xvda1 ”” ] DISK = [ s o u r c e = ”<some path >/ w n s l 5 4 x 8 6 6 4 . img ” , t a r g e t = ” xvda ” , r e a d o n l y = ” no ” ] DISK = [ t y p e = swap , s i z e = 1 0 2 4 , t a r g e t = ” xvdb ” , r e a d o n l y = ” no ” ] DISK = [ type = ” b l o c k ” , c l o n e = ” yes ” , t a r g e t = ” xvdc ” , s o u r c e = ”/ dev / c c i s s / c0d0p4 ” , r e a d o n l y = ” no ” ] NIC = [NETWORK=” d g r z r ” , IP = 1 2 9 . 2 1 7 . 2 4 1 . 2 1 5 ]
- 13. Grid and Cloud @ DGRZR Scenario 1: Private Cloud (already tested in small scale) Cloud and Grid Middleware at DGRZR Use ONE to deploy workernodes on-demand2 S. Freitag Assumption: VO software requirements are satisfied by VO DGRZR specific workernodes VMs D-Grid Integration Interaction with LRMS of Grid middleware required 1:1 mapping of workernode type to LRMS queue A daemon checks the status of each queue Empty queue: reduce number of workernode VM assigned to this queue Re-assign freed resources to another (overcrowded) queue 2 B. Konrad: Dynamic management of VMs on HPC resources of TU Dortmund (diploma thesis, 2009)
- 14. Grid and Cloud @ DGRZR Scenario 2: Public Cloud (starting a project in summer ’10) Cloud and Grid Middleware at DGRZR Major difference to scenario 1 S. Freitag Allow users to deploy services/ VM via a Cloud interface DGRZR D-Grid Split physical resources into a Cloud and a Grid partition. Integration Allow dynamic/ workload-dependend changes in partition size Cloud size=0: Grid resource Grid size=0: Cloud resource All other cases: hybrid resource Which VMs to suspend? → prioritization of VMs Simple Grid batch jobs, MPI batch jobs, services Normal, gold and platinum (paying?) customers Talk of Johannes Watzl this afternoon
- 15. Integration of Cloud Middleware in D-Grid Cloud and Grid Middleware at DGRZR (One) Goal of D-Grid S. Freitag Create sustainable & longterm Grid infrastructure in Germany DGRZR D-Grid Integration → D-Grid is focused on Grid usage. What about Clouds? Cloud interfaces offer a new and easier3 way to remote resources Integration of Cloud middleware into D-Grid Software stack seems pretty obvious (→ increase sustainability) Issues to be resolved for successful integration: user management, authorization, accounting/ billing, monitoring, and information system 3 that’s my personal view ;-)
- 16. User Management Cloud and Grid Middleware at D-Grid DGRZR Central virtual organization membership service VOM(R)S S. Freitag Resources connect to VOM(R)S to query user information DGRZR mapping to local user accounts D-Grid Integration User can have attributes & roles, belong to groups OpenNebula Users stored in a local SQLite3 database Open issues Connection between central VOM(R)S and ONE needed Support for groups, roles (First: Evaluation in D-Grid) Scalability
- 17. Authorization Cloud and Grid Middleware at DGRZR D-Grid S. Freitag Based on X.509 certificates DGRZR OpenNebula D-Grid Integration At present: username/ password mechanism With ONE 1.64 : Users are identified by abstract key/secret tokens. An underlying driver will then interface with the auth back-end (e.g. LDAP / X509 based / PAM / Policikit...) to authenticate the user. General Authorization policies can be implemented, for example quotas or allow a user to submit VMs in a given time frame, user groups.... 4 http://dev.opennebula.org/issues/203
- 18. Accounting Cloud and Grid D-Grid Middleware at DGRZR Jobs pass through a Grid frontend and reach the Grid S. Freitag LRMS DGRZR DGAS 5 and OGF-UR format are used D-Grid Integration Cloud ”jobs” do not reach LRMS, but start fabric level OpenNebula Accounting information can be gathered by joining tables (history table, vm attributes) in the SQLite3 database Open issues Evaluation if equivalent metrics can be collected Design & implement prototype tool 5 Distributed Grid Accounting System
- 19. Information system Cloud and Grid Middleware at DGRZR S. Freitag Each Grid middleware runs an information system D-MON 6 collects information from all these systems, DGRZR D-Grid aggregates and publishes it Integration D-MON uses an adapters/ plug-ins New adapter for Cloud Middlewares must be developed What information is needed? Virtualization software (e. g. Xen, VMware), the available virtual appliances/ templates, limits concerning the maximum amount of cores and memory per virtual appliance 6 http://www.d-grid.de/index.php?id=401
- 20. Future plans Cloud and Grid Middleware at For D-Grid DGRZR S. Freitag Close presented open issues and establish Cloud middleware as new pillar in the D-Grid software stack DGRZR D-Grid Project starts in summer 2010 Integration In Dortmund Integrate more resources into the Cloud Physics department (1000 Cores), in operation Mar 2010 Computer Sciences department (1000 Cores), around 2011 Extend the Cloud to the allianced universities Bochum, Essen/ Duisburg Thanks for your attention and for the great time being here!
- 21. Future plans Cloud and Grid Middleware at DGRZR S. Freitag DGRZR D-Grid Integration