High Performance Computing (HPC) and Engineering Simulations in the Cloud
1 like159 views
The document presents a comprehensive overview of UberCloud's offerings in high-performance computing (HPC) through cloud services, focusing on the benefits and challenges of using cloud computing versus on-premise solutions. It discusses the evolution of their software container technology, which enables efficient engineering simulations and provides a marketplace for application software, resources, and expertise. Additionally, it highlights cost comparisons, security considerations, and the importance of containerization for improved portability and resource management in HPC environments.
High Performance Computing (HPC) and Engineering Simulations in the Cloud
- 1. High Performance Computing in the Cloud UberCloud CustomerWorkshop Wolfgang Gentzsch, President, UberCloud
- 2. Agenda UberCloud Workstations, Servers, Clouds: Benefits & Challenges Total Cost of Ownership: On-premise vs On-cloud Live Demo: Engineering Simulations in the Cloud Application Software Containers Cooperation
- 3. UberCloud: What we do for HPC*) Taking engineering & scientific apps in software containers to ANY computer, from workstations, to servers, to any cloud *) High Performance Computing
- 4. UberCloud July 2012: Building user community with ‘cloud experiments’ (190 so far), feedback, measuring traction, cloud roadblocks March 2013: Starting HPC container development => Apps are easy packageable, portable, accessible, usable, and scalable Nov 2014: Opening online marketplace with currently 35 stores & established provider ecosystem (appl. software resources, software, expertise) April 2016: CAE software containers: ANSYS,CD-adapco,CFD Support, CFturbo, COMSOL, Dacolt, Flow Science, Friendship Systems, LS-Dyna, NICE DCV, Numeca,OpenFOAM, Red Cedar, Simulia, … On Clouds from Advania, AmazonAWS, MicrosoftAzure, Bull/Atos,CPU 24/7, OzenCloud R-Systems,… April 2016:Go to Market with partners Azure, HPE, Intel, ANSYS, many more January 2017: Pre-A $1.7 M with EarlybirdVenture Capital
- 5. UberCloud:Why Engineers design every product we see, own, and use Using sophisticated engineering simulation software But most engineers (95%) still just use desktop computers Buying your own servers is too expensive (highTotal Cost of Ownership) Hybrid approach is cost-effective solution: Small in-house cluster or workstation, plus bursting on demand into HPC Clouds UberCloud: vendor-independent one-stop shop for consumer/ provider community, online marketplace, container technology
- 6. Your major computing tools: , servers,and clouds Let’s analyze the benefits & challenges for 20 million engineers & scientists today workstations
- 7. But, workstations have limited capacity Computing: too slow Memory: too small 57 % of users are dissatisfied with their desktop computing capacity* * Source: US Council of Competitiveness: http://www.compete.org/
- 8. Benefits of servers More compute power and memory Higher quality design and products Reducing product failure early Shorten time to market => Strengthen your company’s competitiveness.
- 9. Challenges with in-house servers Our 193 UberCloud simulation experiments exposed the following pain points: Long procurement cycles for buying h/w and s/w HighTotal Cost ofOwnership (TCO) Complex technical software and infrastructure Bursty usage Rapid technology refresh cycles, servers aging fast Need for trained server and system experts.
- 10. TCO: Servers are expensive and complex $70,000 server => $0.5 - $1 mio cost over 3 years
- 12. Benefits of clouds More (infinite) computing No upfront Cap-Ex investment On demand, pay per use, at your fingertips Scaling resources dynamically, up and down Choice, with multiple providers Result: better, faster, cheaper.
- 13. But: Cloud Challenges It’s new ! Security / your intellectual property Traditional software licensing Data transfer Cloud expertise is needed Losing control over your jobs and data First access to clouds is often cumbersome 13
- 14. SoftwareContainers remove / reduce challenges
- 15. SoftwareContainers remove / reduce challenges http://www.theubercloud.com/workstations -servers-clouds-comparing-apples-apples/
- 16. Cost: server versus cloud Table: real server cost depending on average utilization. http://www.theubercloud.com/cost/ • HPC Server: 16 nodes, 3-yearTCO $1 million. Average utilization25% results in $0.60 per core per hour • HPC Cloud: $0.12 core per hour e.g. on AWS,Azure, or CPU 24/7 cloud resources
- 17. Real example: TCO for 100 node in-house cluster http://www.nextplatform.com/2015/06/03/the-hpc-cloud-versus-cluster-cost-conundrum/
- 18. Cluster cost – owning, hosting, renting 100-node cluster hardware = $41,721 per month 12-node cluster hardware = $5,006 per month $5,006 * 5(*) = $25,032TCO per month or $0.18 per core hour This $0.18 per core hour is for 100% utilized cluster Number of busy compute nodes 1 2 3 4 6 8 9 12 Resulting in cluster utilization of 8.3% 16.7% 25% 33.3% 50% 66.7% 75.0% 100% In-house cluster cost per core/h $2.16 $1.08 $0.72 $0.54 $0.36 $0.27 $0.24 $0.18 Cost per core/h cloud hosting $1.20 $0.60 $0.40 $0.30 $0.20 $0.15 $0.13 $0.10 Cost per core/h, cloud bursting $0.13 $0.13 $0.13 $0.12 $0.13 $0.13 $0.13 $0.13 (*)TCO estimates in the literature are a factor of 3 – 7 of the hardware cost => Factor 5
- 19. Including CAE software cost 1 hour in-house cluster, average utilization of x% with x = 80%, 60%, 40% on 16, 32, 64 cores = 1 hour in-house CAE software, average utilization of x% with x = 80%, 60%, 40% on 16, 32, 64 cores = 1 hour cloud resources on 16, 32, 64 cores = 1 hourCAE software on 16, 32, 64 cores =
- 20. Cost of ISV software licensing In-house: perpetual, annual, cost for upgrade, support & maintenance, etc. In-cloud: annual (hosting), monthly, weekly, daily, hourly, on-demand Examples: eCADFEM => Software on demand (pay per use) => Simulation as a Service (hw / sw on demand) via Customer Portal CADFEM Engineering Simulation Cloud (with Spirit/21) ANSYS Enterprise Cloud ANSYS Elastic Licensing
- 21. URL & pass: http://52.164.231.11:5901/vnc.html jDmcvczkZfiQ Live Demo *) OpenFOAM in der Cloud, im Container
- 22. UberCloud Marketplace Go to the UberCloud online Marketplace
- 23. Welcome email in your Inbox: Copy password and click on Login
- 24. New window in your Browser Paste password and clickConnect
- 25. UberCloud desktop with your application software folder
- 26. Every x hours: you receive email with screenshot and log files*) *) Allowing full control over your cloud activities
- 27. DemoVideos, 3 minutes each - https://www.youtube.com/watch?v=EbKDOxVnxbE - https://www.youtube.com/watch?v=cCSE0Lcj0wk - https://www.youtube.com/watch?v=yFNM9dsCV44.
- 28. UberCloud application containers based on Docker
- 29. The Foundation: Docker-based HPC Container Technology Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data center VMs, or the cloud Enterprise micro services + HPC Layers => HPC macro services
- 31. Virtual Machines vs Docker VM: Each virtualized application includes not only the application and the necessary binaries and libraries, but also an entire guest operating system. Docker Engine container comprises application and its dependencies. Isolated process in user space on host operating system, sharing kernel with other containers. Enjoys resource isolation and allocation benefits ofVMs but more portable and efficient. 31
- 32. UberCloud application containers Based on Docker, enhanced for engineering & scientific applications Software packages designed to deliver the tools that an engineer needs Ready to execute, in an instant. No need to install software, deal with complex OS commands, or configure. Software is pre-installed, configured, and tested, and running on bare metal, without loss of performance. ANSYS,CD-adapco, COMSOL, Numeca, OpenFOAM, Gromacs, Scilab, and more
- 33. UberCloud’s unique HPC container features Basic multi-container single-node ssh between containers Running withinVMs and on bare metal Up-to-date Linux (CentOS, Ubuntu,…) Screen sharing Cloud data storage connectors (to Dropbox, Box.com,…) Emailing features (reporting job status etc.) License server in every container MPI libraries NFS Administrative Automated password generation Self-service password change for increased privacy Performance checking Log monitoring Automated build/refresh process Diff utilities
- 34. Cont’d Advanced Rapid file transfer (from Globus) VPN Pay-per-use multi-container multi-node / inter- networking Infiniband NICE DCV remote visualization VCollab data extraction & visualization OwnCloud privacy & security for cloud storage GPU and Intel Phi enabled RDMA UberCloud storage containers (USC) Projects OpenHPC Pre-built images for ISV codes and complex scientific and engineering workflows Univa Grid Engine in-out integration: containers hosting UGE and can be managed by UGE Slurm container integration Automated QA testing suite OpenStack Qstack Living Heart Project Human Brain Project
- 35. Containers remove Portability & Standard related challenges. By integrating 3rd party tools we tackle the following:
- 36. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
- 37. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
- 38. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
- 39. Builder Launcher Controller ISV DataTools Stackable units with tools (ex: encryption), ISV application codes (ex: OpenFOAM). Just add your own codes and data. Run anywhere with UberCloud Run Time. Scale up or down the compute power as needed. Collect granular usage data, logs. Monitor, alert, report. Any Workstation Any Cluster Any Cloud Run Time Run Time Run Time Containers: Build once, run anywhere
- 40. HTTPS/VPN Access* OS Firewall OS PKI Login Disk Encryption* Dedicated Servers in High Security Data Centers * HTTPS/VPN access and Disk Encryptionare optional UberCloud Security Layers We deploy on single tenant servers (not shared between customers).These servers reside in professionally managed, highly secured data centers. Data at rest can be encrypted. Disk and file level encryption are both available (optional, may degrade performance) Admin access to our servers are protected by PKI Public Key Encryption (vs passwords, which can be guessed) Our servers are protected with firewalls. Only necessary ports are turned on to reduce attacks Connection to our servers are protected by strong encryption techniques such as HTTPS andVPN
- 41. Security Information security, privacy: protecting the users intellectual property guarding raw data (physics, geometry, results) Recommendation: document your security requirements select a suitable provider Encryption,VPN, dedicated cloud server,TRUST, etc. Comment: Security in the cloud is at the same standard level as any other IT infrastructure. It’s more a mental issue.
- 42. DataTransfer Internet too slow for heavy data transfer: Often GBytes of results Especially last mile problem Recommendation: Don’t ship every byte, just important ones (VCollab) Use remote visualization (NICE DCV) if necessary, fedex the data over-night Comment: Clouds can’t solve your last mile problem
- 43. CloudCost Cost of Cloud is often hidden: Server versus Cloud cost is often unclear pay-per-use billing can result in unpredictable cost project can easily run out of budget Recommendation: Cost estimation upfront (UberCloud paper www…./cost *) Automated, policy driven monitoring of usage & billing Comment: Replacement and bursting are 2 different questions *) https://www.TheUberCloud.com/cost/
- 44. Software licensing Incompatible software licensing models: ISVs often still stick to their annual, perpetual licensing Recommendation: Check app software stack on UberCloud Marketplace Talk to your ISV ISVs should develop on-demand short-term s/w licensing Comment: Some ISVs have cloud licensing (CD-adapco,COMSOL, ANSYS,…) Many ISVs are developing cloud licensing Engineers need pay-per-project (week, month)
- 45. Solving software licensing issue, with containers 200+ experiments led to an excellent network of ISV relationships with 120+ ISVs Solution: UberCloud including a license server into every container License Server License Server License Server Any License Server ISV Software ContainerCustomer Software Provider
- 46. UberCloud multi-container environment Intel MPI libraries pre-setup for multi-node HPC Intel compilers achieve architecture specific HPC Networking between containers Setting up shared file system access (NFS) Secure communication (ssh) between containers Benefits from single-node transferred to multi-node containers: No setup of the environment Portability and packaging Easy access and use Low overhead +
- 47. UberCloud multi-container environment Docker Runtime Docker Runtime Docker Runtime . . . . NFS Host Host Host Container Container Container MPI Verticalscaling Horizontal scaling Dramatically simplifying
- 48. UberCloud multi-container environment Docker Runtime Docker Runtime Docker Runtime . . . . NFS Host Host Host Container Container Container MPI Verticalscaling Horizontal scaling Packaging, Porting, Access, Use, Scaling, with Low Overhead
- 49. When in multi-node cluster mode: Only the ‘master’ is accessible via GUI. GUI stays the same, with tools included in the ‘master’ container. Additional servers are deployed, UC container finds & communicates with other nodes automatically. Cluster nodes accessible from GUI node, containers automatically establish password-less SSH group, with nodes talking to each other. Machine file is automatically generated: MPI jobs are distributed across cluster nodes, and workloads are being distributed. No need for queuing; all servers are dedicated to user.Workload requiring queue software is handled as a special requirement.
- 50. OpenFOAMTest: Bare Metal vs Docker OpenFOAM with 1.4 million cells on bare metal and in UberCloud Docker container
- 51. Containers as basic technology for the UberCloud SaaS Factory UberCloud SaaS Factory Software SaaS Commercial ISV Open Source In-house PublicCloud Private Cloud On-Premise $$$ We turn high-TCO, complex software into user-friendly pay-per-use services: Development Services (Docker) -------------------- Deployment & Management Services ------------------ Marketplace Providers Consumers $ UberCloud SaaS Factory
- 53. Cooperation with UberCloud Proof of Concept Project: with your own application in the Cloud Application – HPC-Container for one of your applications Training and Support: 24 hours on 32 cores, with up to 8 hours professional service TCO: Total Cost ofOwnership analysis for your on-prem servers vs cloud vs hybrid cloud solution Vision: Private Cloud, Hybrid Cloud, PublicCloud, enabled by UberCloud containers