Database Cloud Services Office Hours : Oracle sharding hyperscale globally distributed database
- 1. Hyperscale, Globally Distributed Database Copyright © 2019 Oracle and/or its affiliates. Shailesh Dwivedi Vice President Product Management, Oracle Mark Dilman Sr. Director, Product Development, Oracle
- 2. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. Statements in this presentation relating to Oracle’s future plans, expectations, beliefs, intentions and prospects are “forward-looking statements” and are subject to material risks and uncertainties. A detailed discussion of these factors and other risks that affect our business is contained in Oracle’s Securities and Exchange Commission (SEC) filings, including our most recent reports on Form 10-K and Form 10-Q under the heading “Risk Factors.” These filings are available on the SEC’s website or on Oracle’s website at http://www.oracle.com/investor. All information in this presentation is current as of September 2019 and Oracle undertakes no duty to update any statement in light of new information or future events. Safe Harbor Copyright © 2019 Oracle and/or its affiliates.
- 3. Confidential – Oracle Internal/Restricted/Highly Restricted 3 Sharding Overview Sharding Oracle Database Cloud Service New Features 1 2 3 Agenda
- 4. What Is Database Sharding? 4 Dominant approach for scaling Internet applications • Horizontal partitioning of data across independent databases (shards) – Each shard holds a subset of the data – Can be single-node or RAC or PDB – Replicated for high availability • Shared-nothing architecture: – Shards don’t share any hardware (CPU, memory, disk), or software (clusterware) A single logical DB sharded into N physical Databases Server1 Database Table1 Shard1 Server2 Table1 Shard2 Server3 Table1 Shard3
- 5. 5 Oracle Database Sharding – Benefits Linear Scalability Add shards online to increase database size and throughput. Online split and rebalance. Extreme Availability Shared-nothing architecture. Fault of one shard has no impact on others. Geographic Distribution User defined data placement for performance, availability, DR or to meet regulatory requirements. … … … …
- 6. Oracle Confidential – Internal/Restricted/Highly Restricted Sharding – Flexible Deployment Models 6 On-Premises CloudHybrid Shards support all DB deployment models like single instance database, PDB, Exadata, RAC
- 7. Key Customer Use Cases Customer Use Case Other Products Evaluated by Customers before choosing Oracle Sharding Internet Scale Realtime OLTP Cassandra, MongoDB, MemSQL,MariaDB, Couchbase, Aerospike, ScyllaDB Global Databases/Data Sovereignty Google Spanner, Azure Cosmos DB, AWS Aurora, CockroachDB Log Store Apache Lucene, Elastic Search, Solr Metric/Time Series store, IoT, Infrastructure Monitoring, APM AWS Redshift/EMR, Druid, Cassandra, Graphite, InfluxDB Machine Learning Apache Spark, HDFS, NoSQL and SQL Sharded DBs Big Data Analytics Apache Spark, MemSQL Many of the products evaluated by customers are sharded systems and many lack enterprise grade features like support for complex joins, strict data consistency, security, cross region replication, performance optimizer, backup and recovery, triggers, stored procedures, regular security patches, manageability at scale
- 8. Oracle Database Sharding • SQL and all the programmatic interfaces (PL-SQL, OCI, JDBC, etc.) that you expect • Better consistency than NoSQL databases (strictly consistent within shard) • Easier application maintenance – schema in database instead of application • Enterprise features: Advanced Security, RMAN, ASM, Data Guard, GoldenGate, Advanced Compression, Partitioning, etc. • All the Oracle innovations: high-performance storage engine, SMP scalability, RAC, Exadata, in-memory columnar, online redefinition, JSON document store, etc. • Leverage in-house and world-wide Oracle DBA skillset • Enterprise-standard support • Plus extreme scalability & availability of NoSQL databases 8 Best of mature RDBMS capabilities and NoSQL databases
- 9. 9 © 2020 Oracle • Oracle is the only commercially available database which supports native sharding for structured (relational) and unstructured (JSON/Text) data • Unlike NoSQL databases, the Oracle database provides blazing performance for both transactions and analytics • All Oracle Database features are fully available with sharding • Sharding is easy to configure and requires minimal application changes Achieve Linear Scalability, Extreme Availability and Geo- distribution Hyperscale with Oracle Database sharding “Customers like Oracle’s capability to support many workloads including OLTP, IoT, microservices, multimodel, data science, AI/ML, spatial, graph, and analytics.” The Forrester Wave™ Translytical Data Platforms, Q4 2019 Download
- 10. 10 © 2020 Oracle The Oracle Database now includes several different sharding capabilities, enhancing both automated data distribution and hybrid cloud (shards can reside on- premises and in the cloud). AWS’s surveyed reference customers scored its overall product capabilities one standard deviation (STD) below the mean. Their responses identified missing features such as multiregion writes and autosharding. Ranked #1 in all 4 Critical Capabilities Use Cases Gartner Magic Quadrant: Oracle Ranks First for all OpDBMS Use Cases Once the norm, “self administered” sharding and distribution strategies are now seen as complex and challenging. Distributed variable data use cases — particularly geographically distributed ones — are on the rise, especially in the cloud. This is very much the case among enterprises that view automated data distribution as a requirement (and that favor vendors that make it a product feature) rather than a complex skill to be acquired. Gartner CC reportGartner MQ report
- 11. Sharded Tables Shard 1 Shard 2 Shard 3 … … … 11 Schema Creation – Sharded and Duplicated Tables Customers Orders Line Items Products Database Tables Duplicated Table
- 12. Concept: Chunk • Group of related partitions of a sharded table family • Ex: Chunk#1 contains Customers_P1, Orders_P1, LineItems_P1 • All data pertinent to a sharding key resides in a given chunk • No need to go to multiple shards • Unit of data movement for resharding 12 Customers_P1 (1-1000000) Orders_P1 Lineitems_P1 Chunk #1 Sharded Tables
- 13. Customers_P1(1-1M) Orders_P1 Lineitems_P1 Chunk #1 Sharded Tables Customers_P6(5000001-6M) Orders_P6 Lineitems_P6 Chunk #120 Sharded Tables Products (Duplicated Table) ... Shard 1 Set of chunks with data from sharded tables + duplicated tables Contents of a Shard
- 14. Par Optionally – complement replication with Oracle RAC for server HA HA / DR Configurations 14 Active Data Guard with Fast-Start Failover GoldenGate ‘chunk-level’ active-active replication with automatic conflict detection/resolution (OGG 12.3)
- 15. App Tier Routing Tier Data Tier 1 5 Connection Pool Sharding key … … • Client Routing (JDBC, OCI, UCP, ODP.NET) – Direct routing from Connection pools – Proxy routing for Multi-shard queries • Shard Catalog – Stores SDB metadata – Acts as a coordinator for multi-shard queries – Contains app gold schema & duplicated tables • Shard Director – A global service manager for direct routing of connection requests to shards – Publishes run-time SDB topology map, load balancing advisory, FAN events via ONS Cross Shard Query Coordinators Client Request Flow
- 16. Oracle Sharding Methods System Managed Sharding • by Consistent Hash Range of hash values assigned to each chunk User-defined Sharding • by Range • Range of sharding key values assigned to each chunk • by List • Each chunk associated with a list of sharding key values Composite Sharding • by Range - Consistent Hash or by List - Consistent Hash Two-level sharding, uses two keys 16
- 17. Composite Sharding 1 Geographic Distribution and Linear Scalability Customers Americas Customers AsiaCustomers Europe Sharded Database
- 18. • Provides even better fault isolation and scalability • Reduces the number of DB connections • Improves midtier cache locality • For geographically distributed shards eliminates chatty midtier-to-database connections across datacenters • Allows A/B testing with subset of users accessing new application functionality • REST API to route requests to the midtier associated with a sharding key • Called by customers’ routing layer 18 Topology Server Web Tier Data Tier App Tier Router REST “Swim Lanes” across tiers Midtier Sharding
- 19. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Life Cycle Management
- 20. 20 Online Addition and Rebalancing of Shards • A chunk is a unit of resharding • Chunk move is initiated automatically or manually (by DBA) • Uses RMAN Incremental Backup & Transportable Tablespace Add shard
- 21. 21 Auto Rebalance • A chunk is a unit of resharding • Chunk move is initiated automatically or manually (by DBA) • Uses RMAN Incremental Backup & Transportable Tablespace Online Addition and Rebalancing of Shards
- 22. • Monitoring using Oracle Cloud Infrastructure console or Enterprise Manager Cloud Control • Patching/Upgrade using existing Oracle Database tooling Monitoring and Management
- 23. 2 3 Sharding Advisor • Tool to advise on schema migration from non- sharded databases to Sharding • Key goals are to maximize parallelism (spread query execution across all shards), minimize cross shard operations and minimize duplicate data • Analyze existing database schema, user workloads and makes recommendations like which tables to Shard, which column to use as Sharding Key, Sharding Method to use, which tables to duplicate Shard Advisor Sample Output
- 24. 2 4 Sharding Oracle Database Cloud Service • Automation for Sharding Oracle Database Cloud Service • Automation for Sharding with Kubernetis and Docker containers in Oracle Cloud (OKE) • Customer inspired Oracle Cloud reference architecture patterns for deployment best practices for scalability, availability and security
- 25. Oracle Database Sharding on Oracle Cloud (OCI) Marketplace Bharath Ramaprasad Principal Software Engineer - Oracle Email : [email protected]
- 26. • Oracle Sharded Database can now easily be deployed on Oracle Cloud Infrastructure (OCI), with just a few clicks. • Easy management and scaling of Sharded database with just a single click. • Easy navigation and access to all Sharded database resources once deployed. Oracle Database Sharding on OCI Marketplace
- 27. Sharded database example OCI topology
- 28. • Supports system-managed sharding on Oracle Enterprise Extreme performance database edition. • Automatic and uniform distribution of Shards, Catalog and Shard Directors across ADs in a region and Fault Domains within each AD. • Supports dataguard based data replication for shards and catalog and thus provides high availability within a region. • Provides the ability to easily scale the number of Shards and provides horizontal scalability. • Provides the ability to easily scale the number of Shard Directors for high availability and load balancing of requests. Features of Oracle Database Sharding v1.0 on OCI Marketplace
- 29. OCI Marketplace based sample Oracle Sharded Database Deployment Architecture
- 30. Live Demo of Oracle Database Sharding on OCI Marketplace • Live Demo Demo Link • Oracle Sharded Database is live and publicly available on OCI Marketplace starting today : Try Oracle Database Sharding on OCI Marketplace for Free
- 31. Oracle Sharding on OKE Paramdeep Saini Sanjay Singh RACPack MAA, Cloud Innovation and Solution Engineering Team
- 32. Oracle Sharding on Docker/Container Clone the Sharding repo from GitHuB https://github.com/oracle/db- sharding/blob/master/oke-based-sharding- deployment/oracle-gds-docker- image/README.md Docker Images Build image locally or pull image from OCR Provision Containers GSM Container Catalog Container Shard Containers Connect your applications Confidential – Oracle Internal/Restricted/Highly Restricted 32
- 33. Oracle Sharding on Kubernetes Clone the Sharding repo from GitHuB https://github.com/oracle/db- sharding/tree/master/oke-based-sharding- deployment Create OKE Cluster Run Ansible Playbook Setup Shards, Catalog and GSM Run a Helm/Chart Add and delete shard based on your requirement Demo Confidential – Oracle Internal/Restricted/Highly Restricted 33
- 34. Federated Sharding Multi-Shard DMLs Cross-Shard Query Resilience Identification of Sharding Key Sharding Advisor, Terraform Scripts Multltiple PDB shards per CDB Multiple Table Families Multiple Proxy Coordinators Global Sequences Parameter setting across shards User-Defined Sharding PDB Sharding RAC Sharding Mid-tier Sharding Update-able Duplicated Tables Automated Deployment with Replication System-managed & Composite Sharding Centralized Sharded Schema Management Direct & Proxy Routing Online Scale-out w/Auto-resharding Chunk Move and Split 34 Oracle 12.2 Oracle Sharding Evolution Oracle 18c Oracle 20c Oracle 19c
- 35. • Sharding of multiple PDBs to allow consolidation and fault isolation • A CDB can now support multiple PDB shards which helps with consolidation • Different PDBs from same sharded databases are on different CDBs to provide for fault isolation • Scalable multi-shard query coordinator for reporting and analytical workloads • Shard catalog’s Active Data Guard standbys can act as multi-shard query coordinators • Improve resource utilization by allowing sharding different tables by different keys in the same database • Allows an sharded databases to support multiple table families, each of which can be sharded with a different sharding key Oracle Sharding | 19c Features
- 36. SH-C3SH-B3SH-A3SH-C2SH-B2SH-A2 CDB-2 Consolidation of Sharded Databases SH-C1SH-B1SH-A1 CDB-1 DB-A DB-B DB-C CDB-3
- 37. Multiple cross shard parallel query coordinators can be configured with Active Data Guard standbys • Improves scalability of cross shard parallel query execution for reporting and analytical workloads • Coordinators can be placed in different availability domains and geo regions Coordinator Primary ADG Standby ADG Standby ADG Standby Scalability for Cross Shard Parallel Query
- 38. Multiple Table Families • Allows you to make a trade-off: • Reduces amount of duplicated data • May increase time for executing joins • Supported for system-managed sharding (by consistent hash) • All table families share the same set of chunks Customers Orders LineItems Parts 1st Table Family 2nd Table Family Suppliers States Duplicated Table
- 39. Oracle Sharding | 20c Features • Federated Sharding • Allows queries across existing similar databases in multiple geo-regions • Multi Shard DML and Query Enhancements • Updates in parallel across all Shards • Cross Shard Query execution continuation in case of a Shard failover • Automatic Identification of Sharding Key • Simplifies application design and maintenance • Deployment Automation Enhancement • Schema design advise for moving from non-Sharded to Sharded deployment with Sharding Advisor • Deployment Automation with Terraform scripts • Reactive Streams Ingestion Library • Data is split and loaded directly to shards in parallel using direct path write • Native Support for Databases in Persistent Memory
- 40. Federated Sharding 40 Unify multiple existing databases Federated Sharded Database• Global customer at times have multiple instances of same applications deployed for multiple departments in multiple regions • Federated sharding allows mapping of databases of such applications in to a single federated database • Queries can seamlessly be executed against single federated database using cross shard query coordinator • Obviates data replication for reporting and analytics purposes • Tolerance for schema and version variations Cross Shard Query Coordinator
- 41. Cross Shard Coordinator 4 1 Multi Shard DML and Query Resilience • Multi Shard DML Support • Coordinator fans out DML in parallel to all the Shards • ACID transaction properties are preserved • Multi Shard Query Resilience • In case of a Shard failure, Data Guard automatically fails over the failed Shard to the standby Shard • Query coordinator resumes query execution on the failed-over Shard Parallel DML Fanout to Shards Database Shards Update Employee Set salary = salary*1.2 /* 20% raise to all Employees */
- 42. Cross Shard Query Coordinators App Tier Data Tier 4 2 JDBC Connection Pool … … • A new JDBC driver that enables Java connectivity to sharded databases without the need for an application to furnish a sharding key. • Automatically identify which bind variable for a given SQL statement is the sharding key • Automatic routing of queries • Single shard queries to the appropriate shards • Cross shard queries to the coordinator Automatic Identification of Sharding Key
- 43. Reactive Streams Ingestion Library 1) Streaming capability: receive data from a large group of clients without blocking 2) Group records according to RAC and Shard affinity using native UCP. 3) Optimize CPU allocation while decoupling record processing from IO. 4) Fastest insert method for the Oracle Database through Direct Path Insert, bypassing SQL and writing directly in the Database files.
- 44. 4 4 • Native Persistent Memory stores database data and redo in local Persistent Memory (PMEM) – For workloads that need lower latency than flash memory • SQL runs directly on data stored in mapped Persistent Memory file system – Eliminates IO code path, no need for large buffer cache – New database algorithms prevent partial or inconsistent stores to Persistent Memory • Requires Data Guard for protection from server failures or persistent memory failures • Scale-out using Native Database Sharding Persistent Memory DB Server Persistent Memory DB Server Data Guard Support for Databases in Persistent Memory
- 45. Oracle Sharding | Resources https://www.oracle.com/goto/oraclesharding https://docs.oracle.com/en/solutions/db-sharding- single-region Product Documentation https://docs.oracle.com/en/database/oracle/oracle- database/19/shard/index.html 45 https://blogs.oracle.com/database/ https://github.com/oracle/db-sharding
- 46. Oracle Confidential – Internal 46 Q & A