Cloud Foundry - Second Generation Code (CCNG). Technical Overview
36 likes10,670 views
The document provides an overview of the Cloud Foundry technical platform. It describes how Cloud Foundry simplifies application deployment by allowing developers to push applications to the cloud with simple commands. It then summarizes the key components of Cloud Foundry, including the router, cloud controller, health manager, DEAs, buildpacks, messaging, service brokers, and BOSH. BOSH allows Cloud Foundry to be deployed and managed on an IaaS through the use of stemcells, agents, and a cloud provider interface.
Cloud Foundry - Second Generation Code (CCNG). Technical Overview
- 1. Cloud Foundry Technical Overview Pivotal November 2013 © Copyright 2013 Pivotal. All rights reserved. 1
- 2. Simplify Application Deployment, from: this… * An actual application provisioning/update flow in a large enterprise. Image is blurred for privacy reasons © Copyright 2013 Pivotal. All rights reserved. 2
- 3. To: Pushing apps to the cloud with a few easy verbs Operator cf-iaas.yml provision <my cloud> add_capacity <my cloud> Cloud Deployment: 2-4 hours © Copyright 2013 Pivotal. All rights reserved. Developer target <my cloud> push <my app> create <my services> bind <my services> scale <my app> +100 App Deployment: 30-90 seconds 3
- 4. Overview: Deploying App to Cloud Foundry Runtime Blobstore push app DB Service credentials + app MD ② Create and bind services ③ Stage application ④ Deploy application ⑤ Manage application health Router ① Upload app bits and metadata Developer Cloud Controller DEA DEA DEA + = DEA Service Broker Node(s) Cloud Foundry Runtime (PaaS) …which we will depict in a moment © Copyright 2013 Pivotal. All rights reserved. 4
- 5. Creating and Binding a Service DB CLI create service (HTTP) Router bind service (HTTP) © Copyright 2013 Pivotal. All rights reserved. Cloud Controller Developer Service credentials create service (HTTP) bind service (HTTP) Service Broker reserve resources Data Service obtain connection data Cloud Foundry Runtime (PaaS) 5
- 6. Stage an Application Router Blobstore Developer DB Cloud Controller DEA System Buildpacks Detect Yes No Compile + Upload = Cloud Foundry Runtime (PaaS) © Copyright 2013 Pivotal. All rights reserved. 6
- 7. Deploying an Application Developer Cloud Controller Blobstore Messaging (NATS) Router Access App © Copyright 2013 Pivotal. All rights reserved. DEA DEA DEA Cloud Foundry Runtime (PaaS) 7
- 8. Monitoring and Replacing an Application Developer Desired State Actual State Cloud Controller Blobstore Health Monitor Router Messaging (NATS) © Copyright 2013 Pivotal. All rights reserved. DEA DEA DEA Cloud Foundry Runtime (PaaS) 8
- 9. Cloud Foundry Architecture The Cloud Foundry platform is abstracted as a set of large-scale distributed services. It uses Cloud Foundry Bosh to operate the underlying infrastructure from IaaS providers (e.g., VMware, Amazon AWS, OpenStack). Internet Dynamic Router Cloud Controller UAA/Login Servers Health Manager DEA Pool Service Broker Node(s) Apps PaaS User Provided Service Instances Build Packs Logging Messaging (NATS) Cloud Foundry BOSH Underlying Infrastructure © Copyright 2013 Pivotal. All rights reserved. 9
- 10. Cloud Foundry – The Inner Shell This is the inner shell. Components are dynamically discoverable and loosely coupled, exposing health through HTTP endpoints so agents can collect state information (app state & system state) and act on it. Dynamic Router Cloud Controller UAA/Login Servers Health Manager DEA Pool Service Broker Node(s) Apps User Provided Service Instances Build Packs Logging Messaging (NATS) Cloud Foundry BOSH © Copyright 2013 Pivotal. All rights reserved. 10
- 11. Router Responsible For: How It Works: The router shapes and routes all external system traffic (HTTP/API) and application traffic from the internet/intranet. It maintains a dynamic routing table for each load-balanced app instance with IP addresses and ports. • • • • Load balancing Maintaining an active routing table Access logs Supports web-sockets Roadmap: App-specific Metrics Throughput © Copyright 2013 Pivotal. All rights reserved. Latency HTTP Response Codes Bandwidth SSL Termination 11
- 12. Cloud Controller Responsible For: How It Works: The Cloud Controller maintains command and control systems, including interface with clients (CLI, Web UI, Spring STS), account and provisioning control. It also provides RESTful interface to domain objects (apps, services, organizations, spaces, s ervice instances, user roles, and more). • • • • • • • Expected App state, state transitions, and desired convergence Permissions/Auth Orgs/Spaces/Users Services management App placement Auditing/Journaling and billing events Blob storage Roadmap: Availability Zone Aware Placement Richer Auditing with Queries and Filters © Copyright 2013 Pivotal. All rights reserved. Oauth Scope and Role Mapping OpenStack Swift Blob Configuration 12
- 13. UAA and Login Servers Responsible For: How It Works: “User Authorization and Authentication” provides identity, security and authorization services. It manages third party Oauth 2.0 access credentials and can provide application access and identity-as-a-service for apps running on Cloud Foundry. Composed of: UAA Server, Command Line Interface, Library. • • • • Token Server ID Server (User management) OAuth Scopes (Groups) and SCIM Login Server • • • UAA Database SAML support (for SSO integration) and Active Directory support with the VMWare SSO Appliance Access auditing Roadmap: LDAP Login Server Horizontally Scalable Login Server App User Management Services © Copyright 2013 Pivotal. All rights reserved. 13
- 14. Health Manager Responsible For: How It Works: Health Manager monitors application uptime by listening to the NATS message bus for mismatched application states (expected vs. actual). The Cloud Controller publishes expected state and the DEAs publish actual state. State mismatches are reported to the Cloud Controller. • • • Maintains the actual state of apps Compares to expected state Sends suggestions to make actual match expected (cannot make state changes itself – only CC can do that!) Roadmap: Short-lived Apps Horizontally Scalable Health Manager Configurable Restart Policies © Copyright 2013 Pivotal. All rights reserved. 14
- 15. DEA Responsible For: How It Works: “Droplet Execution Agents” are secure and fully isolated containers. DEAs are responsible for an Apps lifecycle: building, starting and stopping Apps as instructed. They periodically broadcast messages about their state via the NATS message bus. Roadmap: • • Managing Linux containers (Warden) Monitoring resource pools • • • • • • • Process File system Network Memory Managing app lifecycle App log and file streaming DEA heartbeats (NATS to CC, HM) Placement Pools for Advanced Resource Allocation and Isolation Evaluation of Windows .NET DEAs from Iron Foundry Aggregated Logs Including All App Instances and App-related System Logs App Log Draining with Syslog © Copyright 2013 Pivotal. All rights reserved. 15
- 16. Buildpacks Responsible For: How It Works: Buildpacks are Ruby scripts that detect application runtimes/frameworks/plugins, compile the source code into executable binaries, and release the app to an assigned DEA. Runtime components can be cached for faster execution of subsequent app pushes. • Staging* • • • • /bin/detect /bin/compile /bin/release Configure droplet • • • Runtime (Ruby/Java/Node/Python) Container (Tomcat/Websphere/Jetty) Application (.WAR, .rb, .js, .py) (*) Cloud Foundry Buildpacks are compatible with Heroku Roadmap: vFabric Import Tool Enhanced Caching Buildpack Management Including Updates and Versioning © Copyright 2013 Pivotal. All rights reserved. 16
- 17. Messaging (NATS) Responsible For: How It Works: NATS is a fast internal messaging bus to manage system wide communication via a publish-and-subscribe mechanism. • • • • Non-Persistent messaging Pub/Sub Queues (app events) Directed messages (INBOX) Roadmap: Robust Message Bus Interface Horizontal Scaling RabbitMQ Investigation © Copyright 2013 Pivotal. All rights reserved. 17
- 18. Service Broker Responsible For: How It Works: Service Brokers provide an interface for native and external 3rd party services. Service processes run on Service Nodes or with external as-a-service providers (e.g., email, database, messaging, etc.). • • • • • Advertising service catalog Makes create/delete/bind/unbind calls to service nodes Requests inventory of existing instances and bindings from cloud controller for caching, orphan management SaaS marketplace gateway Implemented as HTTP enpoint, written in any language. Roadmap: Communication with REST © Copyright 2013 Pivotal. All rights reserved. Multi-Node Support 18
- 19. Service Broker Example: run.pivotal.io + AppDirect (run.pivotal.io) © Copyright 2013 Pivotal. All rights reserved. 19
- 20. User Provided Service Instances Responsible For: How It Works: UPSI (formerly “Service Connectors”) store meta-data in the Service Broker to enable Cloud Foundry to connect to local services that are NOT managed by Cloud Foundry (e.g., OracleDB, DB2, SQLServer, etc.) • Metadata management Roadmap: Service Type Templates (OracleDB, DB2, SQLServer, MQSeries) Investigate Sharing Service Instances Across Spaces © Copyright 2013 Pivotal. All rights reserved. 20
- 21. User Provided Service Instances (on-prem example) Synchronous AppDirect INTERNET Synchronous Service Connector IBM DB2 Service Broker Service Connector Synchronous ORACLE DB Service Broker Gateway Service Broker Alt Broker CF MySQL MYSQL DB Mongo Lab Send Grid © Copyright 2013 Pivotal. All rights reserved. LB ClearDB 21
- 22. To: Pushing apps to the cloud with a few easy verbs Operator cf-iaas.yml provision <my cloud> add_capacity <my cloud> © Copyright 2013 Pivotal. All rights reserved. Developer target <my cloud> push <my app> create <my services> bind <my services> scale <my app> +100 22
- 23. Deploying the CF Runtime with Cloud Foundry BOSH Developer DB Deploy my CF BOSH Director Worker VMs Blobs Message Bus Health Manager Target VM Deployment • • • • • Packages Jobs Blobs Source Manifest © Copyright 2013 Pivotal. All rights reserved. Messaging Health Monitor Cloud Controller VM Target Cloud Foundry BOSH (Operating the PaaS) Target VM IaaS 23
- 24. BOSH (Outer Shell) Logical View Deploys and manages large scale distributed systems. BOSH provides the means to go from deployment (i.e., Chef/Puppet) to VM creation and management (i.e., cloud CPI). It includes interfaces for vSphere, vCloud, AWS and OpenStack. Additional CPI can be written for alternative IaaS providers. Key Elements: • CLI • Director • Blobstore • Workers • • • • © Copyright 2013 Pivotal. All rights reserved. Message Bus Health Monitor IaaS CPI Agents Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell 24
- 25. BOSH: Command Line Interface The Command Line Interface is how users interact with BOSH using a terminal session to do a deployment, create and upload releases, and upload „stemcells‟ (i.e. a VM template with an embedded Agent). Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 25
- 26. BOSH: Director The core orchestrating component in BOSH which controls creation of VMs, deployment, and other life cycle events of software and services. Command and control is handed over to the the Director-Agent interaction after the CPI has created resources. Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 26
- 27. BOSH: Cloud Provider Interface (CPI) The core BOSH engine is abstracted from any particular IaaS. IaaS interfaces are implemented as plugins to BOSH. Currently, BOSH supports both VMware vSphere and Amazon Web Services. These CPIs allow for automated VM and storage disk provisioning, and network management. Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 27
- 28. BOSH: Cloud Provider Interface Stemcell create_stemcell(image, cloud_properties) delete_stemcell(stemcell_id) VM create_vm(agent_id, stemcell_id, resource_pool, networks, disk_locality, env) delete_vm(vm_id) reboot_vm(vm_id) configure_networks(vm_id, networks) IaaS Neutral ...ETC Disk create_disk(size, vm_locality) delete_disk(disk_id) attach_disk(vm_id, disk_id) detach_disk(vm_id, disk_id) © Copyright 2013 Pivotal. All rights reserved. 28
- 29. BOSH: Blobstore Used to store the content of Releases, Jobs and Packages in their source form as well as the compiled image. When you deploy a Release, BOSH will orchestrate the compilation of packages and store the result in Blobstore. When BOSH deploys a Job to a VM, the Agent will pull the specified Job and associated Packages from the Blobstore. Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 29
- 30. BOSH: Agents Every VM contains an Agent. Through the Director-Agent interaction, VMs are given Jobs, or roles, within Cloud Foundry. If the VM's job is to run MySQL, for example, the Director will send instructions to the Agent about which packages must be installed and what the configurations for those packages are. Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 30
- 31. BOSH: Stemcells A Stemcell is a VM template with an embedded Agent. Stemcells are uploaded using the CLI and used by the Director when creating VMs through the CPI. When the Director creates a VM through the CPI, it will pass along configurations for networking and storage, as well as the location and credentials for the Message Bus (NATS) and the Blobstore. Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 31
- 32. BOSH: Health Monitor Receives health status and life cycle events from Agents and can send alerts through notification plugins (such as email) to operations staff. Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 32
- 33. BOSH: NATS Blob Store DB CLI Director Worker BOSH components use NATS, a lightweight pub sub messaging system, for command and control. NATS Health Monitor CPI Inner shell Agents Outer shell © Copyright 2013 Pivotal. All rights reserved. 33
- 34. BOSH: Putting it all together When you deploy Cloud Foundry the following sequence of steps occur: 1. 2. 3. 4. 5. Target a BOSH director using CLI Upload a Stemcell Get a Release from a repo Create a deployment manifest BOSH Deploy Cloud Foundry: • Prepare deployment • Compile packages • Create and bind VMs • Pull in job configurations • Create needed job instances – this is where things get pushed live © Copyright 2013 Pivotal. All rights reserved. Blob Store DB CLI Director Worker NATS Health Monitor CPI Inner shell Agents Outer shell 34
- 35. BUILT FOR THE SPEED OF BUSINESS
Editor's Notes
- #4: Cloud Foundry PaaSAn application runs in a DEA, which is a droplet execution agent. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A servicegateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients (e.g., vmc, STS) access Cloud Controller via RESTful services.
- #5: Cloud Foundry PaaSAn application runs in a DEA, which is a droplet execution agent. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A servicegateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients (e.g., vmc, STS) access Cloud Controller via RESTful services.
- #6: Cloud Foundry PaaSAn application runs in a DEA, which is a droplet execution agent. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A servicegateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients (e.g., vmc, STS) access Cloud Controller via RESTful services.
- #7: Cloud Foundry PaaSAn application runs in a DEA, which is a droplet execution agent. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A servicegateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients (e.g., vmc, STS) access Cloud Controller via RESTful services.
- #8: Cloud Foundry PaaSAn application runs in a DEA, which is a droplet execution agent. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A servicegateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients (e.g., vmc, STS) access Cloud Controller via RESTful services.
- #10: Cloud Foundry PaaSAn application runs in a DEA, which is a droplet execution agent. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A servicegateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients (e.g., vmc, STS) access Cloud Controller via RESTful services.
- #11: Cloud Foundry PaaSAn application runs in a DEA, which is a droplet execution agent. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A servicegateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients (e.g., vmc, STS) access Cloud Controller via RESTful services.
- #20: Add: HP, IBM, RAX, Dell, NTT, Rakuten, SAP, ATT, Verizon