MongoDB Server Provisioning - From 2 Months to 2 Minutes
Download as PPTX, PDF1 like207 views
The document outlines the successful automation of MongoDB server provisioning at Anthem, significantly reducing deployment time from two months to just two minutes. It details the processes implemented, including the use of Ops Manager's server pool feature and Python automation scripts, highlighting advantages such as improved efficiency and customer satisfaction. The lessons learned and future ideas for further automation and integration with orchestration tools are also discussed.
MongoDB Server Provisioning - From 2 Months to 2 Minutes
- 1. MongoDB server provisioning: From 2 months… to 2 minutes Chetan Gavankar, DB Executive Advisor@ Anthem September 2018
- 2. 22 Topics to Cover • MongoDB in Anthem • Demand for quick deployment • Road Map to achieve our goal • How we did it • Lesson learned • Future Ideas • Q/A
- 3. MongoDB in Anthem • Anthem as an organization – 2nd largest HealthCare company – Lots of development using SQL and NoSQL • Why MongoDB a) Flexible schema design b) High availability • From few serves to 200 servers in short time 3
- 4. 4 Demand for quick deployment Current Process (1-2 months) Mongodb Server App team request server Work in Take Process to acquire HW Request to create all IDs Verify HW configurati on OS setup and verificatio n Firewall /Port setup Centrify setup Install MongoDB Setup Ops manager DBA checklist
- 5. 5 Demand for quick deployment Proposed Process (2 minutes) Mongodb Server App team request server MongoDB server delivered from Ops manager server pool Work in Take Process acquired new servers Servers are added back to server pool
- 6. 6 What’s the advantage of Automation • Faster delivery of servers to customers i.e. happy customers • Consistent setup when delivering these servers • DBA efficiency Time and effort saved to build these servers can be utilized to work closely with application team on design review , performance testing and more interesting things for DBA to work on .
- 7. 7 How We did it • Automation of Server Build process • Ops Manager’s Server Pool feature • Automation Software using Ops Manager API and Python
- 8. 8 Automation of Server Build process • Standard offering – We developed SPO (server image) which allows to build servers with specific T-shirt sizes i.e. predefined CPU/MEMORY/STORAGE (mount points)/OS • Scripted OS configuration – The script performs OS setup as recommended in MongoDB Production notes https://docs.mongodb.com/manual/administration/production-notes • No more individual firewall rules! – We created a Firewall group for the given VLAN so required ports are open to allow communications between DB servers and Ops Managers and within DB server itself • Minimizing human intervention – By the time servers are delivered to DBA team only manual process is to install ops manager automation agent and edit the server pool configuration file (this also we will automate soon)
- 9. 9 Ops Manager Server Pool Feature
- 10. 10 Ops Manager Server Pool Feature Custom user defined properties (stored in the pool properties file on each host)
- 11. 11 Automation Software • Ops Manager REST API – Gives incredible flexibility in building scripts/software to manage MongoDB deployments • Python – Easy to learn programming language – OOO development – Package to handle REST API calls
- 12. 12 Single Command to Deploy A single command will create new Group in Ops manager , bind servers to new group depending on the deployment topology and characteristics selected , install MongoDB version selected , setup monitoring , setup alerts , setup backup and put DB servers under Ops manager Automation …and bring you coffee and fresh milk python maintenance/creategroupanddeploy.py --opsmgrUri http://***.**.com:80 --opsmgrapiuser <email user id> --opsmgrapikey *** --organizationId 5af1482f99c8472f18541146 --groupName "FAMRTEST01" --environments DEV --deploymentTopology 'single-node' --tShirtSize small --chipset intel --location dc1 --loglevel debug --mongoDbVersion "3.6.5“ --deploymentName "farmDV01" --disableAlerts --disableBackup
- 13. 13 Originally just a small single script Automation SW Architecture Diagram
- 14. 14 Ops manager REST API - Example
- 15. 15 Calling Ops Manager API from Python - Example import requests Connector = OpsMgrConnector(args.opsMgrUri, args.opsMgrApiUser, args.opsMgrApiKey) return creategroup(args) _________________________________________________________________________________ def creategroup(args): return addGroup(args.opsMgrUri, args.groupName, environmentType, organizationId) def addGroup(opsMgrUri, groupName, groupType, organizationId): addGroupResultJson = opsMgrConnector.addGroup(newGroupName, organizationId) def addGroup(self, newGroupName, organizationId): payload = { "name" : newGroupName, "orgId" : organizationId } return self.post("{}/groups".format(self.apiURL), payload) def post(self, url, payload): result = requests.post(url, json=payload, auth=self.auth).json() return result
- 16. 16 MongoDB Server Deployed in Ops manager
- 17. 17 Lesson Learned • DNS Caching – Nontrivial issues when DNS queries are taking long time • Lots of testing – Different test cases – Different HW such as Intel , Power , Private Cloud, etc. • Non-technical – Project planning and roadmap is a must – Everyone involved should understand project goals and individual responsibilities
- 18. 18 Future Ideas • More automation – Alert templates • Explore integration with other automation/orchestration tools – Chef, Puppet, Ansible – Kubernetes
- 19. 19 Yes it is possible , we did it and All Clicked Faster delivery of servers to customers i.e. happy customers Consistent setup when delivering these servers Time saved to build these servers can be utilized to work closely with application team on design review , performance testing and more interesting things for DBA to work on . Automation is flexible enough to be deployed on other platforms