Accelerating a Path to Digital with a Cloud Data Strategy
4 likes816 views
1) The document discusses accelerating a path to digital transformation with a cloud data strategy. It covers topics like the seismic shifts in organizations and application architectures, and the need to rethink underlying data layers. 2) The presentation discusses building an enterprise data fabric at Royal Bank of Scotland using MongoDB to provide data storage, query, and distribution as a service. This simplified development, reduced costs, and improved velocity. 3) MongoDB was presented as the foundation for cloud data strategies, providing the freedom to run applications anywhere while leveraging the benefits of multiple clouds.
![WHAT’S SO HARD ABOUT DATA MIGRATION?
Then A Miracle
Occurs!
[Insert Product Placement Here]](https://image.slidesharecdn.com/cloud-data-strategy-1efbe886-412a-4973-8262-ce219695d12d-690957168-180316193529/85/Accelerating-a-Path-to-Digital-with-a-Cloud-Data-Strategy-49-320.jpg)
Accelerating a Path to Digital with a Cloud Data Strategy
- 1. ACCELERATING A PATH TO DIGITAL WITH A CLOUD DATA STRATEGY Wednesday 14th March 2018
- 2. WELCOME & INTRODUCTION Daniel Barrett, Vice President UK&I [email protected]
- 3. 3 SPEAKERS Accelerating a Path to Digital with a Cloud Data Strategy DANIEL BARRETT MAT KEEP SION O’HARE VICE PRESIDENT, UK&I MONGODB HEAD OF ONLINE SERVICES TELEFÓNICA/02 DIRECTOR OF PRODUCT MONGODB SENIOR SOLUTIONS ARCHITECT MONGODB MOSTAFA ZAKARIA
- 4. EXECUTING ON A CLOUD DATA STRATEGY Mat Keep, Director, Product Team [email protected] @matkeep
- 5. Mainframe Client-Server Web Control & Efficiency AGILITY & INNOVATION Cloud & Mobile Distributed Systems 1980s Late 90s – 2000s 2015>1960s-70s The Cloud Age Generational Shift
- 6. Seismic Shifts — Organizational
- 7. Seismic Shifts — Application Architecture
- 8. Why do I need to rethink my underlying data layer…..? Architectural affinity Geo-locality Rapid, iterative changes
- 9. API Access Layer Operational Data Customers Products Accounts ML Models Shared Physical Infrastructure App1 App2 App3 1.Development agility 2.Corporate governance 3.Data re-use Cloud Data Strategy Standardized, on-demand database service Cloud Portable Any Cloud, Any Where
- 10. Best way to work with data Intelligently put data where you need it Freedom to run anywhere Intelligent Operational Data PlatformIntelligent Operational Data PlatformIntelligent Operational Data PlatformIntelligent Operational Data PlatformIntelligent Operational Data Platform Why MongoDB?
- 11. MongoDB – Freedom To Run Anywhere Database that runs the same everywhere Coverage in any geography Leverage the benefits of a multi-cloud strategy Avoid lock-in Mainframe Database as a service ServerLaptop Self-managed in the cloud
- 12. Private Cloud
- 13. 10-step Methodology For Private DBaaS Step 1: Common Workload Reqs Step 4: Enabling Multitenancy Step 5: Security Enforcement Step 6: Performance & Uptime SLAs Step 7: Provisioning & Mgmt Step 8: Implementation Step 3: Virtualization Strategy Step 2: Hardware & OS Selection Step 9: Cost Accounting & Chargeback Step 10: Design Reviews Discovery Design Deploy
- 14. From Traditional To DBaaS • Slow to build and launch new applications • Multiple copies of data • Complex data reconciliation controls • High licensing costs • Sprawling server estate 12,000+ RDBMS Instances 3,500 Systems, 40,000 Cores 1,200+ Coherence Instances
- 15. Data Fabric Multi-tenant PaaS Exposing APIs for data streaming and storage Cloud native, self-service Modern, industry standard, open source technologies Intra-day releases Multi-data center “Data Fabric provides data storage, query and distribution as a service, enabling application developers to concentrate on business functionality.” Data Fabric Clients Java, .NET, REST API Layer CRUD & Streaming App Server Layer Java + Linux Database MongoDB Messaging Kakfa Security Authentication&Audit Data Fabric
- 16. Results • £m license cost avoidance (Coherence) • Plans to decommission hundreds of servers • Coherence • Oracle/SQL Server databases Cost Reduction • 2 foundational applications refactored off Coherence • Supporting data needs of a dozen applications Simplification • Velocity: Develop new applications in days • No need for database administration • self-service data service • Promotes collaboration and data sharing Velocity Slide taken from https://www.mongodb.com/presentations/mongodb-days-uk-building-an-enterprise-data-fabric-at-royal-bank-of-scotland-with-mongodb
- 17. Database on Public Cloud
- 18. 18 You have 2 choices: Self-Managed or DBaaS Cloud Migration or Cloud First Self-Managed Aka “Lift and Shift” Database as a service Fork in the road
- 19. 19 You have 2 choices: Self-Managed or DBaaS Cloud Migration or Cloud First Self-Managed Aka “Lift and Shift” Database as a service Fork in the road 1. Provision instances and storage 2. Configure HA 3. Configure security 4. Configure backup/restore 5. Monitoring & alerting 6. Ongoing upgrades & maintenance
- 20. 20 You have 2 choices: Self-Managed or DBaaS Cloud Migration or Cloud First Self-Managed Aka “Lift and Shift” Database as a service Fork in the road 1. Provision instances and storage 2. Configure HA 3. Configure security 4. Configure backup/restore 5. Monitoring & alerting 6. Ongoing upgrades & maintenance Choose instance, hit deploy, wait a couple of minutes
- 21. MongoDB Atlas Managed DatabaseAs A Service For MongoDB • Run for You • Resilient & Secure • Multi-Cloud
- 22. 22 MongoDB Atlas unlocks agility & reduces cost Self-service, elastic, and automated Global and highly available Secure by default Comprehensive monitoring Managed disaster recovery Cloud agnostic
- 23. MongoDB Atlas Powering Microservices Architecture Problem Why MongoDB ResultsProblem Solution Results Over 35 different apps accessed by 10,000+ unique customers on AWS Each experiment produces millions of “rows” of data, which led to suboptimal performance with incumbent databases RDS & Aurora slow, added code complexity DynamoDB limited query functionality & expensive MongoDB Atlas managed database service Flexible document model allows storage of multi-structured data Expressive query language and secondary indexes allow ad-hoc analysis of experiment data Scalability to handle growing data volumes Thermo Fisher customers now obtain real-time insights from mass spectrometry experiments from any mobile device or browser; not possible before Improved developer productivity with 40x less code, improved performance by 6x Easy migration process & zero downtime. Testing to production in under 2 months
- 25. Conclusion 1 Cloud is more than just a new platform: microservices, agile, DevOps enable new ways of delivering apps faster 2 Cloud data strategy is an essential building block 3 MongoDB is the foundation for cloud data
- 27. SAFE HARBOUR STATEMENT This presentation contains “forward-looking statements” within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E of the Securities Exchange Act of 1934, as amended. Such forward-looking statements are subject to a number of risks, uncertainties, assumptions and other factors that could cause actual results and the timing of certain events to differ materially from future results expressed or implied by the forward-looking statements. Factors that could cause or contribute to such differences include, but are not limited to, those identified our filings with the Securities and Exchange Commission. You should not rely upon forward-looking statements as predictions of future events. Furthermore, such forward-looking statements speak only as of the date of this presentation. In particular, the development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB’s sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements.
- 28. MongoDB: Already Used Across Every Industry and Use Case... eCommerce Travel Graph & Recommendation System Analytics Internet of Things Product Catalog Artificial Intelligence Digital Transformation Drug Sequencing Gaming Single View Mobile Database as a Service ...without multi-document ACID transactions
- 29. DATA MODELS AND TRANSACTIONS Relational Database Related data split across multiple records and tables. Multi-record transactions essential Different databases take different approaches Document Database Related data contained in a single, rich document. Transaction scoped to the document
- 30. MULTI-DOCUMENT TRANSACTION EXAMPLES Finance Account debit/credit Retail Insert orders, decrement inventory Telco Insert call detail, update subscriber’s plan
- 31. MONGODB MULTI-DOCUMENT TRANSACTIONS Just like relational transactions ● Multi-statement, familiar relational syntax ● Easy to add to any application ● Multiple documents in 1 or many collections ACID guarantees ● Snapshot isolation, all or nothing execution ● No performance impact for non-transactional operations Schedule ● MongoDB 4.0, Summer ‘18: replica set ● MongoDB 4.2: extended to sharded clusters
- 32. OUR JOURNEY TO ACID TRANSACTIONS Major engineering investment over 3+ years touching every part of the server and drivers ● Storage layer ● Replication consensus protocol ● Sharding architecture ● Consistency and durability guarantees ● Global logical clock ● Cluster metadata management ● Exposed to drivers through API enhancements We’re 85% done…...
- 33. SUMMARY: MONGODB UNIQUELY DELIVERS…... Scale-out, data locality, and resilience of distributed systems ACID transactional guarantees of relational databases Developer productivity of document databases Freedom to Run Anywhere
- 34. JOURNEY INTO THE CLOUD CLOUD EXPLORERS Sion O’Hare, Head of Online Services Telefónica
- 35. Journey into the Cloud Cloud explorers_ Sion O’Hare Head of Online, Integrate & Cloud Services Telefónica Telefonica 14.03.2018
- 36. 36 Telefónica is one of the largest telecommunications companies in the world by market capitalization and number of customers providing a comprehensive offering and quality of connectivity that is delivered over world class fixed, mobile and broadband networks. We run operations in 17 countries, split into two geographic regions: Europe and Latin America. We have 350 million customers out of which more than 276 million are mobile, over 9 million fiber/cable customers and more than 8 million pay TV customers Telefonica Our Story_
- 37. 37 We are building a powerful data-house, evolving each one of our assets and combining around 4 Platforms. Video from 2018 MWC @ LINK – with one clear focus: to put them to work at the service of our customers. Telefonica Our business model_ - Harmonise client information across our platforms and transform into knowledge - Focused on digital services our differentiated offer takes advantage from our physical assets & systems. - Enabled real-time, automated end to end support & commercial systems, digitize out customer orientation - Our physical estate from Network base stations to high street stores, generates real time customer relevant data Markets_ Spain UK Germany Hispam North Hispam South Brasil
- 38. 38 Value Agility Cost Security Legacy Security Network Infrastructure Access Tooling IaaS Saas PaaS DaaS VPC On Premise % of estate / Total Data Costs Security Performance Compliance GRAPH 1 Operation Measure Adoption Framework Cloud First in the UK 2017 Cloud explorers_ Placement Foundation Set Direction Mobilise Prove Review & Repeat Channels Virtualisation - £1.3m Infrastructure - HW Hosting/Sup. halved BDaas PaaS/IaaS+ IaaS On Premise/Hybrid/VPC On Prem
- 39. 39 Cloud first strategy Atlas DBaaS Our journey_ Community Open Source adoption Tactically implemented by local teams with no strategic intent. On Premise V2.2.1 Out of support On Premise 2 DBs / V2.6.9 Vendor Support / version level Public Cloud 14 DBs / V3.2.8 and V3.6.0 DBaaS Atlas POC Virtualisation programme requiring BDs to be upgraded to min version x.x
- 40. 40 Atlas DBaaS Telefonica UK_ 2 Environments (Production & Non Production) 2 Replica sets & 6 DB instances O2 ASK Application Environment SDLC-Cycle Replica Set Aura Non Production Reference 1 Replica set with 3 members Production Production 1 Replica set with 3 members Total 2 Replica sets Component Version Cloud Provider AWS Region Ireland (eu-west-1) Instance Type M10 (2GB RAM, 10GB storage) MongoDB Version 3.4
- 41. 41 Atlas DBaaS Telefonica UK_ 2 Environments (Production & Non Production) 6 Replica sets & 18 DB instances Access & Registration Application Environment SDLC-Cycle Replica Set Messaging Non Production Reference 1 Replica set with 3 members Performance 1 Replica set with 3 members Production Production 1 Replica set with 3 members Total 3 Replica sets Application Environment SDLC-Cycle Replica Set Identity Non Production Reference 1 Replica set with 3 members Performance 1 Replica set with 3 members Production Production 1 Replica set with 3 members Total 3 Replica sets Component Version Cloud Provider AWS Region Ireland (eu-west-1) Instance Type M20 (4GB RAM, 20GB storage) M60 (64GB RAM, 320GB storage) MongoDB Version 3.2 / 3.4
- 42. 42 Atlas DBaaS Telefonica UK_ Priority O2 Component Version Cloud Provider AWS Region Ireland (eu-west-1) Instance Type M10 (2GB RAM, 10GB storage) M20 (4GB RAM, 20GB storage) MongoDB Version 3.6.1 Application Environment SDLC-Cycle Replica Set Version Non production Dev 3 Replica sets with 3 Instances v3.6.1 Non production Test 3 Replica sets with 3 Instances v3.6.1 Non production Perf 3 Replica sets with 3 Instances v3.6.1 Non production Stage 3 Replica sets with 3 Instances v3.6.1 Production Prod 3 Replica sets with 3 Instances v3.6.1 Total 15 Replica sets & 45 Instances v3.6.1 12 services 5 Environments (Production & Non Production) 15 Replica sets & 45 DB instances
- 44. LANDING IN THE CLOUD MIGRATING FROM RELATIONAL, ON PREMISES TO MONGODB IN THE CLOUD Mostafa Zakaria, Senior Solutions Architect [email protected]
- 45. LANDING IN THE CLOUD Migrating from Relational, On Premises, to MongoDB in the Cloud.
- 46. AGENDA • What’s so Hard about Data Migration? • Crossing the Idiomatic Divide. • How to Get MongoDB Off the Ground. • Why MongoDB Atlas?
- 47. WHAT’S SO HARD ABOUT DATA MIGRATION?
- 48. WHAT’S SO HARD ABOUT DATA MIGRATION? 1. Apply for ESTA 2. Identify Travel Dates 3. Search for Available Flights 4. Book and Confirm Flights 5. Search for Accommodation 6. Reserve Accommodation 7. Ensure Cat is Adequately Fed/Sedated 8. Pack Bags 9. Check-In Online 10.Identify best way of going to Airport. 11.Go to Airport 12.Locate Bag Drop 13.Tag and Drop of Bags 14.Go through Security 15.Identify and Locate Gate 16.Wait for Boarding 17.Board and Go to Seat 18.Identify and Locate Gate 19.Wait for Boarding 20.Identify Seat 21.Read Book/Watch Movies 22.Survive Landing 23.Go through Immigration 24.Collect Baggage 25.Locate Mode of Transportation 26.Go to Accommodation 27.Locate and Go to Rooms 28.Unpack Baggage 29.Decide What to do…
- 49. WHAT’S SO HARD ABOUT DATA MIGRATION? Then A Miracle Occurs! [Insert Product Placement Here]
- 50. WHAT’S SO HARD ABOUT DATA MIGRATION? • Data Migration doesn’t just Happen. • Data Migration is not just Copying the Data. • Data Migration requires Understanding the Data. • Data Migration is a Data Discovery & Data Quality Project. • Data Migration process must be Repeatable.
- 51. CROSSING THE IDIOMATIC DIVIDE Hello !
- 52. CROSSING THE IDIOMATIC DIVIDE Tabular Document
- 53. CROSSING THE IDIOMATIC DIVIDE Design Decision Embedding Referencing 1. Top Down vs. Bottom Up 2. Use Case / Access Patterns 3. Entity Relationships 4. Cardinality & Direction 5. MongoDB Document Size
- 54. HOW TO GET OFF THE GROUND
- 55. HOW TO GET OFF THE GROUND Migrate existing deployments running anywhere into MongoDB Atlas with minimal impact to your application. Live migration works by: ● Performing a sync between your source database and a target database hosted in MongoDB Atlas ● Syncing live data between your source database and the target database by tailing the oplog ● Notifying you when its time to cut over to the MongoDB Atlas cluster Live Migration
- 56. HOW TO GET OFF THE GROUND 3rd Party Tools
- 57. WHY MONGODB ATLAS? Cloud Migration or Cloud First Self-Managed Aka “Lift and Shift” Database as a service Fork in the road 1. Provision instances and storage 2. Configure HA 3. Configure security 4. Configure backup/restore 5. Monitoring & alerting 6. Ongoing upgrades & maintenance Choose instance, hit deploy, wait a couple of minutes “You have 2 Choices…”
- 58. Morphia MEAN Stack Support for Popular Languages and Frameworks DRIVERS & ECOSYSTEM
- 59. MONGODB CONNECTOR FOR BI • Run analytical queries on live MongoDB data • Can run against your primary or secondaries • Runs on any server, or co-locate with your database process for optimal performance • BI Connector is also available in Atlas
- 60. All MongoDB Atlas nodes are single-tenant and deployed into their own VPC for security isolation. In-flight security: ● TLS/SSL for in-flight data encryption ● Authentication and authorization access controls with SCRAM-SHA1 ● IP whitelists ● VPC Peering At-rest security: ● Encrypted storage volumes ● AES-256 (CBC mode) hardware encryption with Seagate Self-Encrypting Drives Under the hood SECURE IN THE CLOUD
- 61. MongoDB 4.0: Multi-Document ACID Transactions
- 62. Under the hood Out-of-the-Box Operational Tooling ● Fine Grained Monitoring and Alerts ● Real-Time Performance Panel ● Data Explorer ● Query/Performance Optimization ● Continuous Backup/Point-in-Time Restore ● Query-able Backup ● Enterprise APM Integration ● Programmatic API Management
- 63. Highly Available by Default Under the hood ● A minimum of three data nodes per replica set/shard are automatically deployed across availability zones for high availability ● If your primary node does go down for any reason, the self healing recovery process in MongoDB Atlas will typically occur in under 2 seconds ● BI Connector also available on Atlas
- 64. The Latest MongoDB Features Under the hood ● MongoDB Atlas comes out-of-the-box with MongoDB 3.2 and MongoDB 3.4 available. When maintenance releases become available, MongoDB Atlas will automatically upgrade your cluster, using a rolling process to maintain cluster availability at all times ● When new versions of MongoDB are released, MongoDB Atlas will also allow you to make seamless upgrades without risking downtime
- 65. FREEDOM TO RUN ANYWHERE
- 66. ELASTIC SCALABILITY & ON-DEMAND PROVISIONING Under the hood ● WiredTiger storage engine with compression and fine- grained concurrency control to meet the most demanding SLAs ● MongoDB Atlas supports automatic-sharding, giving you the ability to scale up or out with no impact to your app ● MongoDB Atlas allows you to pick the shard key that best suits your application needs ● All 10m+ MongoDB Atlas clusters are single-tenant and deployed on servers allocated specifically to the cluster
- 67. REPLICATE DATA NEAR USERS
- 68. Integrated services and functions for complex, multi-stage workflows Native SDKs for Android, JS, and iOS apps Direct Database Access MongoDB Stitch