Easy deployment & management of cloud apps
1 like•809 views
The document discusses tools for deploying and managing cloud applications including Terraform, Packer, and Jsonnet. Terraform allows declarative configuration of infrastructure resources, Packer builds machine images, and Jsonnet is a configuration language designed to generate JSON or YAML files from reusable templates. The document demonstrates how to use these tools together to deploy a sample application with load balancing and auto-scaling on Google Cloud Platform. It also proposes ways to further abstract configurations and synchronize application and infrastructure details for improved usability.
![Build images, content defined by a JSON configuration file:
{
"builders": [{
"type": "googlecompute",
"source_image": "debian-7-wheezy-v20140718",
.. credentials ..
}],
"provisioners": [
{
"type": "shell",
"inline": [
"sudo apt-get update",
"sudo apt-get install -y redis-server"
]
},
...
]
}
Packer By Mitchell Hashimoto (Hashicorp founder)
http://www.packer.io](https://image.slidesharecdn.com/easydeploymentmanagementofcloudapps-150701211144-lva1-app6891/85/Easy-deployment-management-of-cloud-apps-6-320.jpg)
!["I don't know how to stop it, there was never any
intent to write a programming language [...] I have
absolutely no idea how to write a programming
language, I just kept adding the next logical step
on the way."
Rasmus Lerdorf (Author of PHP)](https://image.slidesharecdn.com/easydeploymentmanagementofcloudapps-150701211144-lva1-app6891/85/Easy-deployment-management-of-cloud-apps-8-320.jpg)
![Abstracting top-level services
# App.jsonnet
local LbPair = import "LbPair.jsonnet";
{
foo: LbPair {
BaseInstance+: {
disk: {image: "foo-image"},
tags: ["foo"],
"startup-script": ...,
}
},
bar: LbPair {
BaseInstance+: {
disk: {image: "bar-image"},
tags: ["bar"],
"startup-script": ...,
}
},
}
# LbPair.jsonnet
{
BaseInstance:: {
...
},
infrastructure: {
google_compute_address: {
"${-}": { ... }
}
google_compute_forwarding_rule: {
"${-}": { ... }
}
google_compute_health_check: {
"${-}": { ... },
}
google_compute_instance: {
"${-}-1": $.BaseInstance,
"${-}-2": $.BaseInstance,
}
}
}](https://image.slidesharecdn.com/easydeploymentmanagementofcloudapps-150701211144-lva1-app6891/85/Easy-deployment-management-of-cloud-apps-13-320.jpg)
![# Example terraform config fragment
google_compute_instance: {
myinst: {
disks: [
{
image: "myimage"
}
]
}
},
google_compute_disk: {
mydisk: {
image: "myimage"
}
}
Packer Integration
"myimage" must already
exist
# Example terraform config fragment
google_compute_instance: {
myinst: {
disks: [
{
image: {
source: "debian-wheezy",
cmds: [ ... ]
}
}
]
}
},
google_compute_disk: {
mydisk: {
image: { /*similarly */ }
}
}
● Image name chosen automatically by hashing config
e.g. "micromanage-XXXXXXXXXXXX"
● Image built if it does not already exist
● Unused images garbage-collected if older than n days
Also allow
● sh lines
● file copies
● dir copies](https://image.slidesharecdn.com/easydeploymentmanagementofcloudapps-150701211144-lva1-app6891/85/Easy-deployment-management-of-cloud-apps-15-320.jpg)
![Packer Integration
# Example terraform config fragment
google_compute_instance: {
myinst: {
disks: [
{
"image": {
source: "debian-wheezy",
cmds: [ ... ]
}
}
]
cmds: [ ... ] # Same syntax, compiles to startup-script.
}
}
Images are
frozen instances!
cmd
cmd
cmd
---------- freeze image here
cmd
cmd
Tension: More in image => faster instance boot time
More in instance => less likely to have to rebuild image
Best practice: Downloads / builds in the image cmds
Small config files in the instance cmds](https://image.slidesharecdn.com/easydeploymentmanagementofcloudapps-150701211144-lva1-app6891/85/Easy-deployment-management-of-cloud-apps-16-320.jpg)
Easy deployment & management of cloud apps
- 1. Easy Deployment & Management of Cloud Applications Dave Cunningham Software Engineer, Google Cloud Vail Computer Elements Workshop 2015-06-01
- 2. Overview 1. Intro to Cloud resource model 2. Intro to Terraform 3. Intro to Packer 4. Intro to Jsonnet 5. Application management (Fractal demo) }
- 3. Resource: something that can be created / managed / referenced Zone: failure domain Examples... Instance: a virtual machine Persistent Disk: can be assigned to instances (more than 1 if read only) Image: data used to initialize a disk (e.g. Debian OS) Load balancer: forwards packets/requests to a target pool Target pool: A dynamic group of instances Health check: A rule for auto-disqualifying broken instances in a pool Address: can be assigned to an instance (1:1 NAT) or load balancer ... Cloud Terminology
- 4. Example: VM Instance API Resources are represented as JSON objects e.g. Instance JSON describes: ● cores / RAM / scheduling policies / zone ● attached disks, addresses ● description / metadata / tags ● startup script Available operations: instances.insert(json) instances.delete(name) instances.get(name) instances.attach/detachDisk(name, json) instances.add/deleteAccessConfig(name, json) // change address ... All "providers" basically the same ● GCP (Google) ● AWS (Amazon) ● Azure (Microsoft) ● Digital Ocean ● Rackspace ● Openstack / cloudstack (on-prem) ● ...
- 5. myservice.tf apply Resources Forwarding Rule Forwarding Rule Address Disk Route Firewall Health Check Target Pool Network Address Instance Target Pool Address Instance Instance InstanceInstanceInstance Address The first time: 1. Builds plan ○ Ordered by Dependency ○ Parallelized 2. Executes plan 3. Writes local state file Subsequent changes: 1. Examine & refresh state 2. Diff, build plan ○ Ordered by Dependency ○ Parallelized ○ Minimally disruptive 3. Executes plan 4. Updates local state file Terraform By Hashicorp http://www.terraform.io/ Provider & credentials Provider & credentials
- 6. Build images, content defined by a JSON configuration file: { "builders": [{ "type": "googlecompute", "source_image": "debian-7-wheezy-v20140718", .. credentials .. }], "provisioners": [ { "type": "shell", "inline": [ "sudo apt-get update", "sudo apt-get install -y redis-server" ] }, ... ] } Packer By Mitchell Hashimoto (Hashicorp founder) http://www.packer.io
- 7. Jsonnet https://google.github.io/jsonnet/doc/ Addresses the config language problem: Write application Simple config file + Comments + vars + String arith + conditionals+ repetition + int arith Turing completeness!!1 + templates + closures + user def. functions Typical config language
- 8. "I don't know how to stop it, there was never any intent to write a programming language [...] I have absolutely no idea how to write a programming language, I just kept adding the next logical step on the way." Rasmus Lerdorf (Author of PHP)
- 9. Hazards of ad-hoc language design: Jsonnet https://google.github.io/jsonnet/doc/ Complex / surprising behaviorNo specification: difficult to develop tools Feature creep (overlapping features) Ugly implementation Hard to improve / replace implementation with same semantics Hard to port implementation (e.g. to Javascript)
- 10. Jsonnet https://google.github.io/jsonnet/doc/ // Trivial Example { person1: { name: "Alice", welcome: "Hello " + self.name + "!", }, person2: self.person1 { name: "Bob" }, } { "person1": { "name": "Alice", "welcome": "Hello Alice!" }, "person2": { "name": "Bob", "welcome": "Hello Bob!" } } ➡ A configuration language designed like a programming language ● Simple: Just 9 features, (3 are from JSON) ○ Literals, arrays, objects, variables, conditionals, arithmetic, closures, mixins, errors ● Powerful: Multi-paradigm (supports OO and functional) ● Hermetic: Repeatable evaluation, code/data interchangeable ● Familiar: All syntax and semantics compatible with JSON / Python ● Concise: Code / data interleaving, prototype inheritance ● Formal: Complete operational semantics
- 11. Jsonnet configuration Jsonnet configuration Jsonnet configuration Micromanage JSON Blueprint (just data) Packer Terraform Side-effects (deployment time) No side-effects (elaboration time) Local elaboration "Micromanage" (900 lines of Python glue) Providers (AWS, Google, etc) Can be verified with JSON Schema Packer + Terraform provide: ● Hybrid cloud support ● Diff-update management model "Glue" changes Terraform config in 2 subtle but impactful ways: ● Top-level services ● Packer integration
- 12. Top-level services # Example Terraform config... { provider: { /* credentials */ }, resource: { google_compute_address: { foo: { ... }, bar: { ... }, }, google_compute_forwarding_rule: { foo: { ... }, bar: { ... }, }, google_compute_health_check: { foo: { ... }, bar: { ... }, }, google_compute_instance: { foo-1: { ... }, foo-2: { ... }, bar-1: { ... }, bar-2: { ... }, } } } # Example blueprint { environment: { default: { /* credentials */ } } foo: { infrastructure: { google_compute_address: { "${-}": { ... } } google_compute_forwarding_rule: { "${-}": { ... } } google_compute_health_check: { "${-}": { ... }, } google_compute_instance: { "${-}-1": { ... }, "${-}-2": { ... }, } } }, bar: { ... } } Compile to
- 13. Abstracting top-level services # App.jsonnet local LbPair = import "LbPair.jsonnet"; { foo: LbPair { BaseInstance+: { disk: {image: "foo-image"}, tags: ["foo"], "startup-script": ..., } }, bar: LbPair { BaseInstance+: { disk: {image: "bar-image"}, tags: ["bar"], "startup-script": ..., } }, } # LbPair.jsonnet { BaseInstance:: { ... }, infrastructure: { google_compute_address: { "${-}": { ... } } google_compute_forwarding_rule: { "${-}": { ... } } google_compute_health_check: { "${-}": { ... }, } google_compute_instance: { "${-}-1": $.BaseInstance, "${-}-2": $.BaseInstance, } } }
- 14. Variants # Prod.jsonnet import "App.jsonnet" { environment: { default: { /* credentials */ } }, bar+: { BaseInstance+: { machine_type: "n1-standard-4", } }, } # Dev.jsonnet import "App.jsonnet" { environment: { default: { /* credentials */ } }, foo+: { BaseInstance+: { machine_type: "f1-micro", } }, } More interesting abstractions in the demo...
- 15. # Example terraform config fragment google_compute_instance: { myinst: { disks: [ { image: "myimage" } ] } }, google_compute_disk: { mydisk: { image: "myimage" } } Packer Integration "myimage" must already exist # Example terraform config fragment google_compute_instance: { myinst: { disks: [ { image: { source: "debian-wheezy", cmds: [ ... ] } } ] } }, google_compute_disk: { mydisk: { image: { /*similarly */ } } } ● Image name chosen automatically by hashing config e.g. "micromanage-XXXXXXXXXXXX" ● Image built if it does not already exist ● Unused images garbage-collected if older than n days Also allow ● sh lines ● file copies ● dir copies
- 16. Packer Integration # Example terraform config fragment google_compute_instance: { myinst: { disks: [ { "image": { source: "debian-wheezy", cmds: [ ... ] } } ] cmds: [ ... ] # Same syntax, compiles to startup-script. } } Images are frozen instances! cmd cmd cmd ---------- freeze image here cmd cmd Tension: More in image => faster instance boot time More in instance => less likely to have to rebuild image Best practice: Downloads / builds in the image cmds Small config files in the instance cmds
- 17. Application Server Application Server Application Server Application Server Application ServerTile Generation Service Cassandra Cassandra Cassandra HTTP HTTP Cassandra protocol Live version: www.fractaldemo.com Fractal Application Architecture
- 18. ● Deploy & Manage complex web applications ● Declarative config at the lowest level ● Leverage OSS community: support all platforms ● Encourage micro-service architecture ● Build multiple layers of abstraction to hide complexity ● Allow complete control of low-level details if needed Conclusions
- 19. Appendix
- 20. Use existing general purpose scripting language? Write application Simple config file Python / Ruby / Lua / etc. Jsonnet https://google.github.io/jsonnet/doc/ Not hermetic: Can yield different config in different environment Designed for specifying behavior, not data Code cannot be substituted with data (side effects) Heavyweight implementations
- 21. ● Currently implementing instance "cmds" with startup-script ○ GCE has 32k limit for all files, AWS 16k ○ Have to base64 binary files ● Solution: ○ Push material into a GCS / S3 bucket ○ Filename is a hash of content ○ curl -s it to the instance from startup-script Future Work
- 22. ● Abstraction - say less ○ Build template libraries, factor out repetitive code ■ Both tilegen and appserv use Nginx + uWSGI + Flask ○ Override bits of default Nginx / uWSGI / Cassandra configs as needed ○ Higher level templates allow listing of apt packages, repos, keys, etc ○ Define variants with deep control and no repetition ● Synchronize details ○ Backend endpoints / credentials feature in ■ frontend / backend application config (packer configs) ■ infrastructure (metadata, firewalls, health checks, load balancer...) Conclusions