Preserve conflict-relevant data during logical replication.
authorAmit Kapila <[email protected]>
Wed, 23 Jul 2025 02:56:00 +0000 (02:56 +0000)
committerAmit Kapila <[email protected]>
Wed, 23 Jul 2025 02:56:00 +0000 (02:56 +0000)
commit228c3708685542d34e6f02c74240656327a5c622
tree3bd24d14d5c926117692e4a50c44ade8956aed13
parent039f7ee0fe9a25cf26d915d4b68091e080c6c6ec
Preserve conflict-relevant data during logical replication.

Logical replication requires reliable conflict detection to maintain data
consistency across nodes. To achieve this, we must prevent premature
removal of tuples deleted by other origins and their associated commit_ts
data by VACUUM, which could otherwise lead to incorrect conflict reporting
and resolution.

This patch introduces a mechanism to retain deleted tuples on the
subscriber during the application of concurrent transactions from remote
nodes. Retaining these tuples allows us to correctly ignore concurrent
updates to the same tuple. Without this, an UPDATE might be misinterpreted
as an INSERT during resolutions due to the absence of the original tuple.

Additionally, we ensure that origin metadata is not prematurely removed by
vacuum freeze, which is essential for detecting update_origin_differs and
delete_origin_differs conflicts.

To support this, a new replication slot named pg_conflict_detection is
created and maintained by the launcher on the subscriber. Each apply
worker tracks its own non-removable transaction ID, which the launcher
aggregates to determine the appropriate xmin for the slot, thereby
retaining necessary tuples.

Conflict information retention (deleted tuples and commit_ts) can be
enabled per subscription via the retain_conflict_info option. This is
disabled by default to avoid unnecessary overhead for configurations that
do not require conflict resolution or logging.

During upgrades, if any subscription on the old cluster has
retain_conflict_info enabled, a conflict detection slot will be created to
protect relevant tuples from deletion when the new cluster starts.

This is a foundational work to correctly detect update_deleted conflict
which will be done in a follow-up patch.

Author: Zhijie Hou <[email protected]>
Reviewed-by: shveta malik <[email protected]>
Reviewed-by: Masahiko Sawada <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>
Reviewed-by: Nisha Moond <[email protected]>
Reviewed-by: Amit Kapila <[email protected]>
Discussion: https://postgr.es/m/OS0PR01MB5716BE80DAEB0EE2A6A5D1F5949D2@OS0PR01MB5716.jpnprd01.prod.outlook.com
45 files changed:
doc/src/sgml/catalogs.sgml
doc/src/sgml/config.sgml
doc/src/sgml/func.sgml
doc/src/sgml/logical-replication.sgml
doc/src/sgml/protocol.sgml
doc/src/sgml/ref/alter_subscription.sgml
doc/src/sgml/ref/create_subscription.sgml
src/backend/access/transam/twophase.c
src/backend/access/transam/xact.c
src/backend/access/transam/xlog.c
src/backend/access/transam/xlogrecovery.c
src/backend/catalog/pg_subscription.c
src/backend/catalog/system_views.sql
src/backend/commands/subscriptioncmds.c
src/backend/replication/logical/applyparallelworker.c
src/backend/replication/logical/launcher.c
src/backend/replication/logical/reorderbuffer.c
src/backend/replication/logical/tablesync.c
src/backend/replication/logical/worker.c
src/backend/replication/slot.c
src/backend/replication/walsender.c
src/backend/storage/ipc/procarray.c
src/backend/utils/adt/pg_upgrade_support.c
src/bin/pg_dump/pg_dump.c
src/bin/pg_dump/pg_dump.h
src/bin/pg_upgrade/check.c
src/bin/pg_upgrade/info.c
src/bin/pg_upgrade/pg_upgrade.c
src/bin/pg_upgrade/pg_upgrade.h
src/bin/pg_upgrade/t/004_subscription.pl
src/bin/psql/describe.c
src/bin/psql/tab-complete.in.c
src/include/catalog/catversion.h
src/include/catalog/pg_proc.dat
src/include/catalog/pg_subscription.h
src/include/commands/subscriptioncmds.h
src/include/replication/logicallauncher.h
src/include/replication/slot.h
src/include/replication/worker_internal.h
src/include/storage/proc.h
src/include/storage/procarray.h
src/test/regress/expected/subscription.out
src/test/regress/sql/subscription.sql
src/test/subscription/t/035_conflicts.pl
src/tools/pgindent/typedefs.list