FDW-based Sharding Update and Future
4 likes2,395 views
The document discusses FDW-based sharding in PostgreSQL. It provides an overview of what database sharding and FDW-based sharding are. It then demonstrates FDW-based sharding in PostgreSQL 9.6, covering how data is inserted and queried across foreign child tables. The document outlines several challenges for FDW-based sharding like distributed transactions and asynchronous execution. It also reviews key techniques being developed for PostgreSQL 10 like push down optimizations and declarative partitioning.
FDW-based Sharding Update and Future
- 1. Copyright©2017 NTT corp. All Rights Reserved. FDW-based Sharding Update and Future NTT Open Source Software Center Masahiko Sawada PGConf Russia 2017 (16th March)
- 2. 2Copyright©2017 NTT corp. All Rights Reserved. Who am I? Masahiko Sawada Twitter : @sawada_masahiko GitHub: MasahikoSawada PostgreSQL Contributor Freeze Map(PG9.6) Multiple Synchronous Replication(PG9.6) Quorum-based Synchronous Replication(PG10) PostgreSQL Technical Support pg_repack committer
- 3. 3Copyright©2017 NTT corp. All Rights Reserved. 1. What is database sharding 2. What is FDW-based sharding 3. Demonstration 4. Use cases 5. Challenges and key techniques 6. Conclusion Agenda
- 4. 4Copyright©2017 NTT corp. All Rights Reserved. • Scale-up • Vertical scaling • Simple • Price • Not safe against hardware failure Scale-up and Scale-out • Scale-out • Horizontal scaling • Easier to run fault- tolerantly • More complex
- 5. 5Copyright©2017 NTT corp. All Rights Reserved. • A scale-out technique • The tables are divided and distributed into multiple servers • Row based • Column based • A database shard can be placed on separate hardware What is database sharding
- 6. 6Copyright©2017 NTT corp. All Rights Reserved. • Pros • Write scale out (Horizontal scaling) • Reduce I/O on each shard, by splitting data across shard • Access only required shard • Cons • Node management • Cross-shard transaction could be cause of slow query • Downtime might be required when changing the sharding layout Pros and Cons
- 7. 7Copyright©2017 NTT corp. All Rights Reserved. • Reliability • Backups of individual database shards • Replication of database shards • Automated failover • Distributed queries • Avoidance of cross-shard joins • Auto-increment key, like sequence • Distributed transactions Challenges
- 8. 8Copyright©2017 NTT corp. All Rights Reserved. • Postgres-XC by NTT, EDB • Postgres-XL by 2ndQuadrant • Postgres Cluster by Postgres Professional • Greenplum by Pivotal • pg_shard by CitusData • Other than PostgreSQL, • VoltDB • MySQL Cluster • Spanner • etc Well-known Products
- 9. 9Copyright©2017 NTT corp. All Rights Reserved. • FDW-based sharding is a database sharding techniques using mainly FDW (Foreign-Data-Wrapper) and Table Partitioning • Our goal is providing a sharding solution as a Built-in feature. What is FDW-based sharding PostgreSQL PostgreSQL PostgreSQL PostgreSQL Client Client Client Client postgres_fdw
- 10. 10Copyright©2017 NTT corp. All Rights Reserved. Basic Architecture (PG9.6) PostgreSQL Heap Table (Parent Table) Foreign Table (child) Foreign Table (child) PostgreSQLHeap Table (child) PostgreSQLHeap Table (child) Client postgres_fdw Coordinator Node Data NodeData Node Table Partitioning
- 11. 11Copyright©2017 NTT corp. All Rights Reserved. Basic Architecture (PG9.6) PostgreSQL Heap Table (Parent Table) Foreign Table (child) Foreign Table (child) PostgreSQLHeap Table (child) PostgreSQLHeap Table (child) Client postgres_fdw Coordinator Node & Data Node Data NodeData Node Table Partitioning Heap Table (Parent Table)
- 12. 12Copyright©2017 NTT corp. All Rights Reserved. Data Node Multiple coordinator nodes (future) PostgreSQL PostgreSQLPostgreSQL Client Client Client Client PostgreSQL PostgreSQL Heap Table Foreign Table PostgreSQL Coordinator Node
- 13. 13Copyright©2017 NTT corp. All Rights Reserved. PostgreSQL server behaves both (future) PostgreSQL Client Client Client Client PostgreSQL PostgreSQL Heap Table Foreign Table Coordinator & Data Node Coordinator & Data Node Coordinator & Data Node
- 14. 14Copyright©2017 NTT corp. All Rights Reserved. Insert data ID=50 PostgreSQL Heap Table (Parent Table) Foreign Table (child) Foreign Table (child) PostgreSQLHeap Table (child) PostgreSQLHeap Table (child) Client postgres_fdw Coordinator Node Data NodeData Node Table Partitioning INSERT INTO parent_table VALUES(50); ID : 0 ~ 100 ID : 101 ~ 200
- 15. 15Copyright©2017 NTT corp. All Rights Reserved. Select data ID=150 PostgreSQL Heap Table (Parent Table) Foreign Table (child) Foreign Table (child) PostgreSQLHeap Table (child) PostgreSQLHeap Table (child) Client postgres_fdw Coordinator Node Data NodeData Node Table Partitioning ID : 0 ~ 100 ID : 101 ~ 200 SELECT … FROM … WHERE id = 150;
- 16. 16Copyright©2017 NTT corp. All Rights Reserved. Sort Push Down -- 9.5 Sort Output: p.col Sort Key: p.col -> Append -> Seq Scan on public.p Output: p.col -> Foreign Scan on public.s1 Output: s1.col Remote SQL: SELECT col FROM public.s1 -> Foreign Scan on public.s2 Output: s2.col Remote SQL: SELECT col FROM public.s2 -- 9.6 Merge Append Sort Key: p.col -> Sort Output: p.col Sort Key: p.col -> Seq Scan on public.p Output: p.col -> Foreign Scan on public.s1 Output: s1.col Remote SQL: SELECT col FROM public.s1 ORDER BY col ASC NULLS LAST -> Foreign Scan on public.s2 Output: s2.col Remote SQL: SELECT col FROM public.s2 ORDER BY col ASC NULLS LAST =# EXPLAIN (verbose on, costs off) SELECT * FROM p ORDER BY col;
- 17. 17Copyright©2017 NTT corp. All Rights Reserved. • Using PostgreSQL 9.6.2 • Insert to foreign child table • Partition pruning Demonstration PostgreSQL Parent Table Child Table Child Table PostgreSQLID 1 ~ 100 PostgreSQLID 101 ~ 200 postgres_fdw Foreign Server 1 Foreign Server 2 SQL Coordinator
- 18. 18Copyright©2017 NTT corp. All Rights Reserved. • Transparent to the user • No need to modify application code • No special DDLs for table management • same as local table partitioning • Can use multiple partitioning method; list, range (and hash) • Horizontal partitioning • Can support not only PostgreSQL shard node but also other source that corresponding FDW exists • Coordinator node can be a shard node as well • All features are Implemented as a generic feature • FDW features are useful on their own merit FDW-based Sharding
- 19. 19Copyright©2017 NTT corp. All Rights Reserved. • PostgreSQL 9.6 can cover use cases where, • Frequent reads • The system requires write scale-out • Write single shard node in a transaction • If you don’t need transaction, you can do it with multiple server Use cases
- 20. 20Copyright©2017 NTT corp. All Rights Reserved. • More push down* • Distributed query optimization* • Asynchronous execution* • Partitioning* • Transaction support* • Node registration • High availability • etc. Challenges and Key Techniques of FDW-based sharding
- 21. 21Copyright©2017 NTT corp. All Rights Reserved. • Push-down makes distributed query execution more efficient • What push down we can and can’t • Conditionals • data types, operators, function (including extension-provided) • Join, Sort, Aggregate(PG10+) • Grouping sets, window function aren’t yet • Patches for PostgreSQL 10 • “Push down more full joins in postgres_fdw” by Etsuro Fujita • “Push down more UPDATEs/DELETEs in postgres_fdw” by Etsuro Fujita • “postgres_fdw: support parameterized foreign joins” by Etsuro Fujita More push down
- 22. 22Copyright©2017 NTT corp. All Rights Reserved. postgres_fdw and distributed queries Operation PostgreSQL 9.5 PostgreSQL 9.6 PostgreSQL 10 SELECT Foreign table pruning Foreign table pruning Foreign table pruning Conditionals Push down Push down Push down Aggregations Local Local Push down Sorts Local Push down Push down Joins Local Push down (Left, Right, Full) Push down* (Left, Right, Full) UPDATE, DELETE Tuple based using CURSOR Directly execution Directly execution* (with joins) INSERT INSERT to remote server using Prepare/Execute INSERT to remote server using Prepare/Execute INSERT to remote server using Prepare/Execute
- 23. 23Copyright©2017 NTT corp. All Rights Reserved. • Need declarative partitioning • Committed basic infrastructure and syntax to PostgreSQL 10! • Still missing building blocks • Tuple routing feature • doesn’t support insert foreign partitioned table so far • Executor improvement • Global unique index Partitioning
- 24. 24Copyright©2017 NTT corp. All Rights Reserved. • Executor improvement • Data fetching request to different site can be sent asynchronously • Improves foreign table scanning performance • Patch • Under discussion • “Asynchronous execution for postgres_fdw” by Kyotaro Horiguchi Asynchronous Execution
- 25. 25Copyright©2017 NTT corp. All Rights Reserved. • Provide cluster-wide transaction (ACID) • Atomic commit • Under reviewing • Transaction involving multiple foreign servers commits using two- phase-commit protocol • Patch • “Transactions involving multiple postgres foreign servers” by Masahiko Sawada, Ashutosh Bapat Distributed Transaction Management
- 26. 26Copyright©2017 NTT corp. All Rights Reserved. Processing Sequence of 2PC on FDW Coordinator Foreign server 1 Client Foreign server 2 Two-phase commit is used transparently.
- 27. 27Copyright©2017 NTT corp. All Rights Reserved. • Business report (complex query analyzing large data) • By aggregation pushdown, optimizer improvement • Update partition key across nodes atomically • By atomic commit distributed transaction Use cases with PostgreSQL 10
- 28. 28Copyright©2017 NTT corp. All Rights Reserved. Conclusion
- 29. 29Copyright©2017 NTT corp. All Rights Reserved. • FDW-based sharding brings us a native PostgreSQL scale- out solution • A lot of work in-progress building blocks • Do we really need it? • To expand the applicability to more critical system • Each sharding feature improves PostgreSQL generically • More detail of FDW-based sharding, • https://wiki.postgresql.org/wiki/Built-in_Sharding Conclusion - Keep challenging -
- 30. 30Copyright©2017 NTT corp. All Rights Reserved. • The Future of Postgres Sharding • https://momjian.us/main/writings/pgsql/sharding.pdf • Shard (database architecture) • https://en.wikipedia.org/wiki/Shard_(database_architecture) • Planning Parallel and Distributed Queries • https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVs dGRvbWFpbnxyb2JlcnRtaGFhc3xneDo1ZmFhYzBhNjNhNzVhMDM0 References
- 31. 31Copyright©2017 NTT corp. All Rights Reserved. Thank you Спасибо Masahiko Sawada [email protected]
- 32. 32Copyright©2017 NTT corp. All Rights Reserved. • NTT has been developing feature related to FDW-based sharding since PostgreSQL 9.3, with the knowledge obtained through the development Postgres-XC. FDW features 9.3 • *Introduce postgres_fdw • *Write via FDW • *Foreign table inheritance • *Join push down • *Sort push down • *Direct perform UPDATE and DELETE • Extension-provided operator push down • *Partitioning • Aggregate push down • (*Async execution) • (*2PC on FDW) • Trigger on Foreign table 9.4 9.5 9.6 10