Automated Application Management with SaltStack
0 likes875 views
Arnold Bechtoldt gave a presentation on using SaltStack for automated application management and orchestration. He began by defining orchestration and what is needed for true orchestration. SaltStack meets these needs through its state modules, remote execution, pillars, returners, reactors and other components. He demonstrated how to use SaltStack to automate the deployment of a multi-tier application, installing software and configuring servers at each stage and responding to events to reconfigure load balancers.
![〉 ... uses remote execution to distribute system commands
〉 ... should be more than just executing commands (most of the tools end here)
〉 ... should be able to respect internal & external dependencies/relations
〉 ... should be deterministic and fast
〉 ... must [RFC2119] be easy to understand/learn (YMMV)
Orchestration...
6](https://image.slidesharecdn.com/automated-application-management-bechtoldt-meetup-06-2016-160616074957/85/Automated-Application-Management-with-SaltStack-6-320.jpg)
![http://www.ansible.com/blog/orchestration-you-keep-using-that-word
For instance, if you use a single-purpose deployment
tool [...] to trigger some legacy configuration
management, that might be called “orchestrating X
with Y”. In reality, it’s usually just replacing a non-
scaling [...] server solution X with a stand-alone
implementation of Y.
8](https://image.slidesharecdn.com/automated-application-management-bechtoldt-meetup-06-2016-160616074957/85/Automated-Application-Management-with-SaltStack-8-320.jpg)
![salt/job/20151104191820394966/new {
"_stamp": "2015-11-04T18:18:20.512126",
"arg": [
”orchestration.bootstrap",
{
"__kwarg__": true,
"test": false
}
],
"fun": "state.sls",
"jid": "20151104191820394966",
"minions": [
”mw42"
],
"tgt": ”mw42",
"tgt_type": "glob",
"user": ”root”
}
Event Structure
20
custom/minion/haste_server_started {
"_stamp": "2015-11-04T18:33:54.650568",
"cmd": "_minion_event",
"data": {
”custom": {
"onchanges": [],
"foo": "bar",
"num": 42,
},
"sfun": "wait"
},
"id": "mw2",
"pretag": null,
"tag": "custom/minion/haste_server_started"
}](https://image.slidesharecdn.com/automated-application-management-bechtoldt-meetup-06-2016-160616074957/85/Automated-Application-Management-with-SaltStack-20-320.jpg)
Automated Application Management with SaltStack
- 1. Automated Application Management with SaltStack Arnold Bechtoldt Cologne, 08.06.2016
- 2. Arnold Bechtoldt Systems Engineer / Consultant @ inovex 〉 Software-Defined Datacenters 〉 Continuous Integration/Delivery 〉 DevOps 2
- 3. Assets & Links 3 inovex.de arnoldbechtoldt.comgithub.com/bechtoldt/talk-salt-orchestration
- 5. Highway To Heaven 5 Asset- Management, Accounting Bare-Metal Deployment Configuration Management/ Containerization Orchestration Auto-Scaling + Elasticity Cloud
- 6. 〉 ... uses remote execution to distribute system commands 〉 ... should be more than just executing commands (most of the tools end here) 〉 ... should be able to respect internal & external dependencies/relations 〉 ... should be deterministic and fast 〉 ... must [RFC2119] be easy to understand/learn (YMMV) Orchestration... 6
- 7. 7 MCollective + Shell Script A not-so-cool Example
- 8. http://www.ansible.com/blog/orchestration-you-keep-using-that-word For instance, if you use a single-purpose deployment tool [...] to trigger some legacy configuration management, that might be called “orchestrating X with Y”. In reality, it’s usually just replacing a non- scaling [...] server solution X with a stand-alone implementation of Y. 8
- 10. SaltStack implements the same techiniques that other tools do: 〉 send commands to servers 〉 send commands to servers that have different operating systems 〉 send commands only to a subset of servers 〉 run command A, then B, then C Often this isn’t enough for us! SaltStack Orchestration 10
- 11. What we really want to have: 〉 scale accross thousands of servers 〉 easy configuration (less software programming) 〉 an interface to implement internal/external relationships (pre/post tasks) 〉 fully automated workflows/procedures (no manual interaction) SaltStack Orchestration 11
- 12. What we really want to have: 〉 scale accross thousands of servers SOA – Message Bus Architecture 〉 easy configuration (less software programming) YAML – JSON – Python DSL 〉 an interface to implement internal/external relationships (pre/post tasks) State – Remote Execution – Pillar – Returner – Beacon – Cloud (~ 600 modules) 〉 fully automated workflows/procedures (no manual interaction) Event System – Reactor – Runner – APIs SaltStack Orchestration 12
- 13. Salt Peer-Publish: Realtime Execution 13 minionminion minion minion minion master minion minion minion minion minion
- 14. 〉 Peer-2-Peer like 〉 Privilege management on master-side 〉 Synchronous & blocking Use Cases: 〉 Gathering realtime metrics of other systems (minions) 〉 Automatically-triggered remote execution Salt Peer-Publish: Realtime Execution 14
- 15. Salt Mine: Dynamic Runtime Data Cache 15 minionminion minion minion minion mine (master) minion minion minion minion minion minion
- 16. 〉 Caching arbitrary data/metrics 〉 Replaces slow/expensive peer public calls Use Cases: 〉 Gathering backend addresses for LB config 〉 Gathering SSH public keys (host/user) of other other hosts 〉 Asynchronous collecting of metrics for monitoring Salt Mine: Dynamic Runtime Data Cache 16
- 17. Salt Reactor: Event-Driven System Automation 17 minion minion minion minionminion minion minion minion minion minionminion reactor (master) salt/key salt/job/new node/stonith/new update/application salt/auth salt/job/return scale/out loadbalancer/config/update salt/minion/start salt/presence/present monitoring/hosts/new firewall/config/update salt/minion/stop salt/presence/change monitoring/hosts/remove coffee/new
- 18. Event-Driven System Automation 18 minion minion minion minion minion minionmaster MySQL Foreman etcd MySQL ElasticsearchSMTP Syslog Icinga Appliances HTTP Docker Slack PostgreSQLminion AWS
- 19. 〉 Automated cfg mgmt and remote execution across several hosts 〉 „Real Orchestration“ 〉 Asynchronous 〉 Synchronous/blocking alternative: Orchestration Runner Use Cases: 〉 Automated loadbalancer configuration updates 〉 Automated host bootstrapping and software provisioning Salt Reactor: Event-Driven System Automation 19
- 20. salt/job/20151104191820394966/new { "_stamp": "2015-11-04T18:18:20.512126", "arg": [ ”orchestration.bootstrap", { "__kwarg__": true, "test": false } ], "fun": "state.sls", "jid": "20151104191820394966", "minions": [ ”mw42" ], "tgt": ”mw42", "tgt_type": "glob", "user": ”root” } Event Structure 20 custom/minion/haste_server_started { "_stamp": "2015-11-04T18:33:54.650568", "cmd": "_minion_event", "data": { ”custom": { "onchanges": [], "foo": "bar", "num": 42, }, "sfun": "wait" }, "id": "mw2", "pretag": null, "tag": "custom/minion/haste_server_started" }
- 21. State A resource should look like {…} Pillar Database/CMDB (Files, RDBMS, NoSQL) Execution Execute command X on a server. Used by other Salt components. Returner Log store of job results (DB, Elasticsearch, Syslog, Monitoring) Beacon Special event triggers (inotify, load, procs, network/user activity) Reactor Reacts to events by triggering subsequent procedures (execution modules) Cloud IaaS provisioning (EC2, OpenStack, Digital Ocean, Linode, GCE, VMware) … SaltStack Terminology 21
- 22. Hands on: Demo
- 24. Demo Concept 24 1. Tells MW hosts to install MW 4. Tells FE hosts to install FE 6. Tells MW hosts to deploy App 10. Tells FE to reconfigure FE 2. Install Node.JS (MW) 3. Report back to Master 7. Deploy App 8. Send Notification Mail 9. Report back to Master 5. Install Haproxy (FE) 6. Report back to Master 11. Add/Remove HAProxy backends 12. Report back to Master Salt Master/Reactor MW hosts FE hosts
- 25. Demo Concept 25 1. Tells MW hosts to install MW 4. Tells FE hosts to install FE 6. Tells MW hosts to deploy App 10. Tells FE to reconfigure FE 2. Install Node.JS (MW) 3. Report back to Master 7. Deploy App 8. Send Notification Mail 9. Report back to Master 5. Install Haproxy (FE) 6. Report back to Master 11. Add/Remove HAProxy backends 12. Report back to Master Salt Master/Reactor MW hosts FE hosts
- 26. Assets & Links 26 sh.arbe.io/cloud-provision youtu.be/9MzeK4u4pkM (demo)github.com/bechtoldt/talk-salt-orchestration
- 27. Q&A
- 28. Arnold Bechtoldt inovex GmbH [email protected] CC BY-NC-ND inovex.de +ArnoldBechtoldtGER github.com/bechtoldt arbe.io youtube.com/inovexGmbH