How we Upgraded Public Cloud From Juno to Queens with Minimal Downtime? | Nguyễn Duy Bình
0 likes•90 views
BizFly Cloud upgraded their OpenStack platform from the older Juno release to the newer Queens release with minimal downtime. They developed a plan to address challenges like converting the LDAP authentication to MySQL, maintaining network and metadata configurations, and ensuring data integrity across over 1,200 VMs during the upgrade. Through automation scripts and a 5 minute downtime window per VM, the upgrade was completed over several months with users able to continue using the cloud services throughout. After the upgrade, BizFly Cloud saw improvements like a new dashboard, support for additional services, and more powerful hardware infrastructure.
How we Upgraded Public Cloud From Juno to Queens with Minimal Downtime? | Nguyễn Duy Bình
- 2. Content overview BizFly Cloud - before upgrade Problems and Upgrade Challenges Big Upgrade from Juno to Queens with minimal downtime BizFly Cloud - after upgrade
- 3. VCCORP Joint stock company Electronic Media - ChannelVN E-commerce - Zamba VOD, Video Streaming, Game - Soha Advertisement - Admicro IaaS, PaaS and SaaS – BizFly Cloud
- 4. THE DEVELOPMENT PRODUCTS BY VCCORP
- 5. Build Private Cloud since 2012 Release Public Cloud since 6/2014 Release CDN since 2015 and have distributed system at almost big Datacenter in VietNam • Serving projects at VCCorp • Size ~ 5000 cloud server ~ 4000 Physical CPU Core ~ 800 TB RAM ~ 5000 TB DATA CLOUD AT VCCORP
- 6. BizFly Cloud - Before Upgrade Openstack release: Juno Keystone with LDAP Simple dashboard, less feature Provide only 2 services: Cloud server and CDN
- 7. BizFly Cloud - Before Upgrade
- 8. BizFly Cloud - Before Upgrade
- 9. Problems DNS, proxy Cache, CDN, DDoS Filter Juno is a very old Openstack release at that time New keystone version can’t CRUD with LDAP Hard to develop new services based on Openstack such as: Loadbalancer, VPN, DBaaS, Auto Scale….. Lack many new features of core services of new Openstack version Hard to develop new dashboard for users with new features
- 10. Upgrade Challenges DNS, proxy Cache, CDN, DDoS Filter Queens - Juno = 6 Openstack versions Convert from LDAP to Mysql Minimal downtime for ~1200 VM on Public Cloud Data integrity, network, IP, metadata of VM Dashboad, billing... after upgrade
- 11. Big Upgrade from Juno to Queens with minimal downtime 3/2018 4/2018 6/2018 Tháng 7/2018 Upgrade and finish convert VM from Juno to Queens Define Problems, Challenges and Start Research Build up full Plan for Upgrade Build Queens Cloud Develop New Dashboard for Cloud Convert LDAP to MySQL Acceptance Test Full System Automation upgrade process
- 12. Convert from LDAP to MySQL User Login to Dashboard and change password Update to LDAP Keystone Create Project/User and write to Mysql LDAP MySQL
- 13. Network, IP, Metadata of VM and data integrity Get all information of an instance: ID, volume ID, Snapshot of volume, name, network, keypair, metadata Juno Openstack Cloud Python Script Create the same VM, volume, snapshot with all the information get from Juno Cloud Queens Openstack Cloud Ceph Cluster Stop Queens Cloud VM and update database to mapping VM with volume on Ceph backend VM running on Ceph Cluster
- 14. Minimal downtime with ~1200 VM Results: downtime ~5 minutes per VM Juno Openstack Cloud Queens Openstack Cloud Stop VM on Juno Cloud and Start corresponding VM on Queens Cloud
- 15. BizFly Cloud - After upgrade Openstack release: Queens Keystone with MySQL New Dashboard with many new features Develop new Openstack Cloud based services New hardware with more powerful: ● Compute node with new and high performance CPU ● Upgrade network from 20GB to 40GB, dedicate network for VM and storage ● More high availablity in all system
- 16. BizFly Cloud - After upgrade More and more Openstack Cloud Based services released
- 17. BizFly Cloud - After upgrade
- 18. BizFly Cloud - After upgrade
- 19. BizFly Cloud - After upgrade
- 20. THANK YOU