Backroll: Production Grade KVM Backup Solution Integrated in CloudStack
0 likes1,165 views
Backroll is an open-source KVM backup solution integrated within Apache CloudStack, aimed at providing a user-friendly backup strategy with features like incremental backups and automated tasks. The solution is developed by Dimsi, a French cloud provider focused on small to medium enterprises, and offers integration, a modern UI, and a RESTful API for easy management. Future developments and community-driven enhancements are planned, fostering a collaborative project with ongoing support for users.
Backroll: Production Grade KVM Backup Solution Integrated in CloudStack
- 1. BackROLL: An open-source KVM Backup solution fully integrated inside Apache Cloudstack From November 14 to 16, 2022
- 2. Table of contents § Bringing an easy-to-use KVM backup solution to the community What's our vision of a production grade backup solution ? § Key features All the features you need to build a backup strategy § Integration within your workflow Our design of the BackRoll API was built around integration with a wide range of workflows § Integration with the CloudStack ecosystem (plugin) Integrate our tool with CloudStack to make backup processes transparent to users and administrators § Roadmap What's next ? Planned new features and our vision of the future of BackRoll § Q & A Ask us anything you want about us or BackRoll’s project J § Who are we About DIMSI and your speakers § What is BackROLL ? All the features you need to build a backup strategy
- 3. Your fellow speakers (Greg, Quentin, Pierre) Grégoire LAMODIERE Co-Founder & solution architect Backroll evangelist Quentin ROCCIA Senior SRE and Devops Engineer Backroll guru Pierre CHARTON Senior C# / Java developer, Pierre provides Backroll integration within Apache Cloudstack, main contributor of Backroll plugin Senior backend developer Backroll integrator Automation addict, Quentin is in charge of maintaining Cloudstack infrastructure up and running. He also helps our developers to speed-up code integration and deployment Senior solution architect, Grégoire helps our customers to define Cloud-based, hybrid and OnPremise strategies
- 4. Who are we ? We are a small independent French cloud provider that keeps its focus on small and medium sized companies We also provide for companies using Microsoft Dynamics CRM, and bring them our expertise on project management and development to meet their business needs We think that even a small team such as ours can bring great knowledge and success to our customers Our human-size teams put people at the heart of our digital services
- 5. ABOUT DIMSI In 2022, DIMSI became a mission-driven company Foundation 2006 Area of expertise Company Size 4 sites : § Paris § Lorient § Sousse § Hong-Kong Offerings Microsoft Power Platform Cloud & Services Modern Apps Lorient Paris Sousse Hong-Kong
- 6. What’s Backroll ? What prompted us to create BackROLL • No similar open-source community project • No turnkey solution available • No KVM backup software yet integrated into Apache CloudStack • You may have noticed that we are KVM/CloudStack enthusiasts :) What we wanted • No agent to install on virtual machines nor on hosts • Quiescent backup support • Backup deduplication • Incremental backups • Fully integrated into CloudStack Already proven safeguards • Currently in production for all internal and customer Apache Cloudstack instances at DIMSI • Has already been used for client requests for data restoration • Used to build our disaster recovery plan • No performance impact during backups Open Source with Apache2 license • The same licensing model as Apache Cloudstack • Everyone can check source code and bring their ideas to improve the software Aiming at becoming a community driven project !
- 8. The various components of BackROLL Modern web UI • Front-end The web UI, offers the same functionality as the API • Back-end Contains API and core logic, it receives all API calls that will trigger tasks (Celery) • Workers Used to run the Celery tasks (VM list, backup list, host list, backup job, and so on...), can be stand-alone or clustered to maximize task parallelization • Flower Flower is used to monitor tasks in real-time and get logs via web interface / web API • Beat Beat is the scheduler, allowing us to create scheduled backup policies and run redundant and automated tasks (like backup cleaning every night) • Redis Redis is used as a database to store keys specific for every task, allowing tracking and keeping records.
- 9. Bring an easy-to-use KVM backup solution to the community 1. Modern web UI Designed to be easy to use Easily manage your backups § Easily track the progress and logs of your backup tasks § Manage virtual machine backups, pools or backup policies via a graphical interface § Schedule backup and restore tasks § Built in VueJS 3 The web interface is optional, but still convenient for non-tech users 2. RESTful API That follows best practices Can be used in your existing workflows § Stateless § Uniform interface § Layered system § Can be used with scripts or tools like Ansible The learning curve is minimal if you are already used to working with other REST APIs 3. Containerized application Reduces maintenance requirements § Deploy and update using a docker-compose template § One click deployment § Customize deployment template to meet your needs Reduce maintenance hassle as everything needed is embedded in the docker image
- 10. What does an API call look like? Modern web UI
- 11. What does an API call look like? Modern web UI
- 12. How does a backup work? Modern web UI
- 13. What you can already do A sample of what BackROLL is capable of Instant start of backup or restore task of a VM / Pool Everything is handled in the background Nothing to do on the VM side, even the needed restart is handled by BackROLL Backup / restore task scheduling If you prefer to schedule your tasks to run at night, it is possible! Self-management of lock for each VM, no way to go wrong Whenever a VM is linked to a task, BackROLL locks out any further operations on the virtual machine Retention management and automatic cleaning Only keep data you need Operation history and webhook notification To keep track of what has been done
- 14. Caveats and limitations For a better understanding • It is not currently possible to backup a shutdown VM that is managed by cloudstack • Restore and redeploy (to another VMs) task is currently in work-in-progress state • Only supports Slack for webhook notifications • Only NFS storage has been used to store the backups • You cannot select a specific disk you'd like to backup / restore, current task state is all-or-nothing • You cannot backup a VM that is currently using a snapshot disk file • Integration of BackROLL within CloudStack is in work-in-progress state
- 15. Integration within your workflow ANALYSE Revue de l’architecture Prérequis Document de conception d’infrastructure complété incluant la volumétrie de données, le nombre d’utilisateurs, etc. Livrable Rapport de l'examen d'architecture accompagné des préconisations DIMSI délivré par un Solution Architecte D365 CONCEPTION Revue de la conception Prérequis Document de spécifications fonctionnels et Document de Design Technique complétés Plan d‘ intégration & Interfaces complété Livrable Document d’examen de la conception : Conclusions et recommandations. PERSONNALISATION Revue de la personnalisation Prérequis Document de conception technique détaillées. Accès à tous les codes sources de l'application. Définition d'un périmètre pour l'examen de codes. (L'étude portera sur un périmètre restreint représentatif d'un processus métier clé) Livrable Document d'examen de la personnalisation : Conclusion et recommandations DÉPLOIEMENT Revue de performances Prérequis Disponibilité du document de de déploiement. Plan de production complétés Environnement iso production et opérationnel (non accessibles aux utilisateurs) Définitions des scénarios de tests. Document d'exploitation. Livrable Document d’examen des performances et de l'exploitation, conclusions et recommandations.
- 16. Integration with the CloudStack ecosystem (plugin) Cloudstack Back and restore framework Cloudstack Backroll Plugin How we’ve built Backroll plugin for Cloudstack (http://docs.cloudstack.apache.org/en/latest/adminguide/backup_and_recovery. html) We are developping our plugin using the Cloudstack Back and Restore framework How to setup the Backroll plugin • Set 'backroll' to the "backup.framework.provider.plugin" global configuration variable • Set these 3 variables: • "backup.plugin.backroll.config.url" • "backup.plugin.backroll.config.appname" • "backup.plugin.backroll.config.password" Code sample, authent, etc To do: diagram to describe components • Allows Cloudstack users to use 3rd party backup solutions • Abstracts the API commands required for common backup and recovery operations Concepts • 'SLA' Backups • Adhoc and user scheduled backups Key features • Authentication • List of existing backup policies • Start a backup for registred virtual machines • Restore virtual machines from backups • Delete backups
- 17. Live demo Modern web UI
- 18. Roadmap Modern web UI What's next ?
- 19. www.dimsi.fr | [email protected] | (+33) 01 75 43 89 66 4 Avenue Laurent Cély Hall D, 92600 Asnières-sur-Seine Thank you !