Idera live 2021: Managing Databases in the Cloud - the First Step, a Successful Initial Migration by Bert Scalzo
0 likes143 views
The document provides a comprehensive guide on managing database migrations to the cloud, emphasizing the importance of careful planning and consideration of various factors such as database size, user roles, and cloud architecture. It includes a checklist of questions to assess before migration and highlights the necessity of understanding cloud services and licensing options. Additionally, it discusses benchmarking tools and the significance of documenting data models for a successful transition.
Idera live 2021: Managing Databases in the Cloud - the First Step, a Successful Initial Migration by Bert Scalzo
- 1. © 2021 IDERA, Inc. All rights reserved. © 2021 IDERA, Inc. All rights reserved. Managing Databases in the Cloud The First Step, a successful initial migration 1
- 2. © 2021 IDERA, Inc. All rights reserved. Books written by the presenter 2 The two most recent The first & most known My website has over 100 papers www.bertscalzo.com
- 3. © 2021 IDERA, Inc. All rights reserved. Books presenter as contributor or tech editor SQL in a Nutshell, 4th Edition Kevin Kline, Regine O. Obe, and Leo S. Hsu
- 4. © 2021 IDERA, Inc. All rights reserved. Look before you leap, still waters run deep Moving to the cloud is not an easy or automatically successful task, you can easily jump into harms way if you’re not careful! If you take your time and plan, then zero reasons not to succeed – really …
- 5. © 2021 IDERA, Inc. All rights reserved. Database migration checklist 1. What is the size of your database? 2. How many schemas and tables do you have? 3. How many really big tables do you have (200 gigabytes or 200 million rows in size)? 4. What do the transaction boundaries look like? 5. Do you have engine-specific data types that won’t be migrated by your migration tool? 6. Do you have LOBs in your tables, and how large are they? 7. Do all your tables with LOBs have primary keys? 8. How hot (busy) is your source database? 9. What kind of users, roles, and permissions do you have on the source database? 10. When was the last time you vacuumed, or compacted, your database? 11. How can your database be accessed (firewalls, tunnels, VPNs)? 12. Do you know what VPC you want to use in AWS? 13. Do you know what VPC security group you can use? 14. Do you have enough bandwidth to move all your data? 15. Can you afford downtime? How much? 16. Do you need the source database to stay alive after the migration? For how long? 17. Do you know why you preferred one target database engine over another? 18. What are your high availability (HA) requirements? 19. Does all the data need to move? 20. Does it need to move to the same place? 21. Do you understand the benefits offered by Amazon RDS? 22. Do you understand any Amazon RDS limitations which might affect you? 23. What happens to your application after the migration? 24. What is your contingency plan if things go wrong? https://aws.amazon.com/blogs/database/database-migration-what-do-you-need-to-know-before-you-start/ Very complicated and full of new terms, concepts, and answers: this presentation will help to prepare you … See – really not simple nor easy …
- 6. © 2021 IDERA, Inc. All rights reserved. What’s Moving to the Cloud? 6 •List databases (MySQL, PostgreSQL, SQL Server, …) •List database versions (e.g. SQL 2014, 2017, 2019, etc.) •Any special needs (e.g. Oracle) – ASM – Flex ASM – RAC – Flex Clusters – Exadata – Replication § GoldenGate § SharePlex These requirements may limit your possible cloud vendor options Oracle supported? Extra requirements? Cloud alternatives?
- 7. © 2021 IDERA, Inc. All rights reserved. Choice of numerous cloud vendors 7
- 8. © 2021 IDERA, Inc. All rights reserved. For DB’s, Really Big Three + Oracle 8 In this presentation we try to remain cloud neutral and use generic terminology
- 9. © 2021 IDERA, Inc. All rights reserved. Same Thing, Different Terminology 9 AMI 1 2 3
- 10. © 2021 IDERA, Inc. All rights reserved. Traditional On-Premise 10 SERVER Operat ing Syst em App App App Bare Metal Virtualized DBMS DBMS
- 11. © 2021 IDERA, Inc. All rights reserved. Cloud Offers Extra Option 11 SERVER Operat ing Syst em App App App Bare Metal Virtualized (Infrastructure as a Service) Database as a Service 1 2 3 DBMS DBMS
- 12. © 2021 IDERA, Inc. All rights reserved. Know the licensing options & ramifications (e.g. Oracle) 12 • Amazon – RDS On-Demand (metered) or BYOL – Amazon EC2 AMI + BYOL • Google – Partner-managed Oracle instances from Marketplace (metered) – Google Base Metal Solutions + BYOL • Microsoft – Azure VM + BYOL – Bridge to Oracle Cloud Infrastructure (OCI) License Included vs. BYOL • Oracle – Cloud Infrastructure (OCI) License Included vs. BYOL § Bare Metal vs. Virtual Machine DB Systems § Exadata Cloud Service § Autonomous Databases Note: BYOL licensing does not get to use core factor calculation based upon virtual CPU’s on non-Oracle clouds! You went through a similar licensing reconsideration when you virtualized
- 13. © 2021 IDERA, Inc. All rights reserved. Understand Cloud Architecture 13 •Regions and Availability Zones •On-Demand vs. Spot vs. Reserved Instances vs. Dedicated Hosts •Image type: General Purpose vs. Compute Optimized vs. Memory Optimized vs. Storage Optimized vs. Accelerated Computing •For image type, various options: vCPU, Mem, Storage, Dedicated Bandwidth, and Network Performance •Learn the cloud vendor’s cost calculator to see what all the above will cost per month (you may be surprised)
- 14. © 2021 IDERA, Inc. All rights reserved. 14
- 15. © 2021 IDERA, Inc. All rights reserved. 15
- 16. © 2021 IDERA, Inc. All rights reserved. 16
- 17. © 2021 IDERA, Inc. All rights reserved. 17
- 18. © 2021 IDERA, Inc. All rights reserved. So How Do You Pick Cloud Settings 18 • First, attend some cloud vendor free training (they are very good) to learn the basics and vocabulary for the chosen cloud solution • Then pretty much the same as we did for on-premise when ordering servers from a new hardware vendor new to us – compare vs what we know … • Ascertain the kind of workload your database application approximates (e.g. OLTP, Data Warehouse, Decision Support, Data Analytics, etc.) • Option #1: Run TPC-C, TPC-E, TPC-H, or TPC-DS standard benchmarks to gauge performance on-premise vs. various cloud characteristic setups (good to get a basic idea of what the new environment options can do) • Option #2: run workload capture & replay from actual system to gauge performance on-premise vs. various cloud characteristic settings (good if you really must have a perfect old vs new performance comparison)
- 19. © 2021 IDERA, Inc. All rights reserved. Benchmarking Tools 19 • HammerDB (formerly HammerORA) – Open source / freeware – Oracle, SQL Server, IBM DB2, MySQL, MariaDB, and PostgreSQL • Swingbench – Freeware – Oracle only • SLOB (Silly Little Oracle Benchmark) – Freeware – Oracle only – Really just a low level IO benchmark – Presentation author offers a customized version of this tool https://github.com/Apress/database-benchmarking-and-stress-testing
- 20. © 2021 IDERA, Inc. All rights reserved. Look at all the results (not just TPS) 20 Notice that stopping at 500 users did not reveal a clear winner!!!
- 21. © 2021 IDERA, Inc. All rights reserved. 21 Common Database Benchmark Pattern – True Point of Saturation 0 1000 2000 3000 4000 5000 6000 7000 100 200 300 400 500 600 700 800 900 1000 TPS Avg Resp Time Looking for inflection point where TPS is still increasing or just decreasing and close to max where average response time is below customer defined SLA Notice the line characteristics between roughly 750 and 850 concurrent users – for current configuration and optimization benchmark results interesting Common mistake to simply attempt maximize TPS – remember TPS is not IOPS
- 22. © 2021 IDERA, Inc. All rights reserved. DB Benchmarking Needs Tools & Expert 22 Database Benchmarking Tool: stress database & make it sweat, that’s all they do (no built-in smarts) Database Monitor Expert – DBA must know database, OS, hardware, software, tuning, & have read the benchmark specs Expert decides what treadmill inclination, what speed, what duration, what to monitor, how to diagnose, how to treat, etc.
- 23. © 2021 IDERA, Inc. All rights reserved. Successful DB Benchmarking - SQL Diagnostic Manager 23 Expert – must know database, O/S, hardware, software, tuning & have read the benchmark specs HammerDB Stress test the database ? ? ? And now cloud too!
- 24. © 2021 IDERA, Inc. All rights reserved. Successful DB Benchmarking – Precise DB Monitor 24 Expert – must know database, O/S, hardware, software, tuning & have read the benchmark specs HammerDB Stress test the database ? ? ? And now cloud too!
- 25. © 2021 IDERA, Inc. All rights reserved. Review & Document Everything 25 • Database options, configuration parameters, DBA accounts, database IP addresses, space usage per database, etc. • Data Model, Data Model, Data Model – you must understand and be able to readily visualize all your critical schema objects • Any special needs or requirements – e.g. root or SUDO access, ability to deploy OS agents, 3rd party software requirements • Job scheduler and batch schedules • Applications, both 3rd party and home grown • Infrastructure management tools RDS not allow this RDS may limit this
- 26. © 2021 IDERA, Inc. All rights reserved. ER Studio – document logical & physical data models 26
- 27. © 2021 IDERA, Inc. All rights reserved. Why do I stress data models so much? 27 • Data models are the contextual roadmap of your data assets that you’ll be moving to the cloud • When questions come up (as they guaranteed will), you’ll have a visual roadmap to work with • You won’t be changing the logical model based upon moving to the cloud – i.e. business remains the same • But the physical model may need changes due to the performance found within the new cloud environment • You must know the logical model in order to figure out any optional physical designs you may want to try since often the logical model defines what your physical options may be …
- 28. © 2021 IDERA, Inc. All rights reserved. Recent similar article worth reading… 28 Plan, Set Goals, and Prioritize Map Your Data Estate https://www.dbta.com/Columns/Next-Gen-Data-Management/Prepare-Now-for-Cloud-Database-Migration-148242.aspx
- 29. © 2021 IDERA, Inc. All rights reserved. Closing thoughts … 29 •This was only the first step, must successfully migrate and then tweak / optimize based upon cloud choices •Be patient, steep initial learning curve with lots to learn and be prepared for setbacks •Make sure to size cloud images appropriately and know the costs of what you choose •Run benchmark or data capture replay on old system and new cloud based system, then compare all of the performance results / metrics
- 30. © 2021 IDERA, Inc. All rights reserved. 30 THANKS! ANY QUESTIONS? You can contact me at: [email protected]