PostgreSQL 9.4 add logical replication command and parameters

本文涉及的产品
云原生数据库 PolarDB MySQL 版,通用型 2核4GB 50GB
云原生数据库 PolarDB PostgreSQL 版,标准版 2核4GB 50GB
简介:
逻辑复制已经讲了蛮久, 以前拿未提交的补丁测试过一下.
现在终于看到9.4提交的一些信息了, 从提交的信息来看, 基础基本上打好了. 看样子逻辑复制很快就要实现了.

Add new wal_level, logical, sufficient for logical decoding.

When wal_level=logical, we'll log columns from the old tuple as
configured by the REPLICA IDENTITY facility added in commit
07cacba983ef79be4a84fcd0e0ca3b5fcb85dd65.  This makes it possible
a properly-configured logical replication solution to correctly
follow table updates even if they change the chosen key columns,
or, with REPLICA IDENTITY FULL, even if the table has no key at
all.  Note that updates which do not modify the replica identity
column won't log anything extra, making the choice of a good key
(i.e. one that will rarely be changed) important to performance
when wal_level=logical is configured.

Each insert, update, or delete to a catalog table will also log
the CMIN and/or CMAX values of stamped by the current transaction.
This is necessary because logical decoding will require access to
historical snapshots of the catalog in order to decode some data
types, and the CMIN/CMAX values that we may need in order to judge
row visibility may have been overwritten by the time we need them.

Andres Freund, reviewed in various versions by myself, Heikki
Linnakangas, KONDO Mitsumasa, and many others.




其中包含一些变更  : 
alter table 语法中和逻辑复制相关的部分, REPLICA IDENTITY, 设置如何定位到变更或删除的数据行. 可以设置为主键, FULL(表示所有字段), 或者非空的唯一索引. 例如设置为主键的话, 那么XLOG中记录OLD值就只记录主键的OLD值. FULL的话是记录所有的字段的OLD值. 值影响logical级别的wal.
http://www.postgresql.org/docs/devel/static/sql-altertable.html
    REPLICA IDENTITY {DEFAULT | USING INDEX index_name | FULL | NOTHING}

REPLICA IDENTITY
This form changes the information which is written to the write-ahead log to identify rows which are updated or deleted. This option has no effect except when logical replication is in use. DEFAULT records the old values of the columns of the primary key, if any. USING INDEX records the old values of the columns covered by the named index, which must be unique, not partial, not deferrable, and include only columns marked NOT NULL. FULL records the old values of all columns in the row. NOTHING records no information about the old row. In all cases, no old values are logged unless at least one of the columns that would be logged differs between the old and new versions of the row.

参数和逻辑复制相关的部分, wal_level新增了logical级别, 是目前的最高级别, 包含逻辑复制需要的额外xlog信息.
http://www.postgresql.org/docs/devel/static/runtime-config-wal.html#RUNTIME-CONFIG-WAL-SETTINGS
 
    wal_level (enum) 
    

wal_level determines how much information is written to the WAL. The default value is minimal, which writes only the information needed to recover from a crash or immediate shutdown. archive adds logging required for WAL archiving; hot_standby further adds information required to run read-only queries on a standby server; and, finally logical adds information necessary to support logical decoding. Each level includes the information logged at all lower levels. This parameter can only be set at server start.

In minimal level, WAL-logging of some bulk operations can be safely skipped, which can make those operations much faster (see Section 14.4.7). Operations in which this optimization can be applied include:

CREATE TABLE AS
CREATE INDEX
CLUSTER
COPY into tables that were created or truncated in the same transaction
But minimal WAL does not contain enough information to reconstruct the data from a base backup and the WAL logs, so archive or higher must be used to enable WAL archiving (archive_mode) and streaming replication.

In hot_standby level, the same information is logged as with archive, plus information needed to reconstruct the status of running transactions from the WAL. To enable read-only queries on a standby server, wal_level must be set to hot_standby or higher on the primary, and hot_standby must be enabled in the standby. It is thought that there is little measurable difference in performance between using hot_standby and archive levels, so feedback is welcome if any production impacts are noticeable.

In logical level, the same information is logged as with hot_standby, plus information needed to allow extracting logical changesets from the WAL. Using a level of logical will increase the WAL volume, particularly if many tables are configured for REPLICA IDENTITY FULL and many UPDATE and DELETE statements are executed.


系统表和逻辑复制相关的部分
pg_class.relreplident
relreplident char   Columns used to form "replica identity" for rows: d = default (primary key, if any), n = nothing, f = all columns i = index with indisreplident set, or default

测试部分 : 

[参考]
3.  http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=07cacba983ef79be4a84fcd0e0ca3b5fcb85dd65
相关实践学习
使用PolarDB和ECS搭建门户网站
本场景主要介绍基于PolarDB和ECS实现搭建门户网站。
阿里云数据库产品家族及特性
阿里云智能数据库产品团队一直致力于不断健全产品体系,提升产品性能,打磨产品功能,从而帮助客户实现更加极致的弹性能力、具备更强的扩展能力、并利用云设施进一步降低企业成本。以云原生+分布式为核心技术抓手,打造以自研的在线事务型(OLTP)数据库Polar DB和在线分析型(OLAP)数据库Analytic DB为代表的新一代企业级云原生数据库产品体系, 结合NoSQL数据库、数据库生态工具、云原生智能化数据库管控平台,为阿里巴巴经济体以及各个行业的企业客户和开发者提供从公共云到混合云再到私有云的完整解决方案,提供基于云基础设施进行数据从处理、到存储、再到计算与分析的一体化解决方案。本节课带你了解阿里云数据库产品家族及特性。
目录
相关文章
|
4月前
|
SQL 监控 关系型数据库
实时计算 Flink版操作报错合集之在设置监控PostgreSQL数据库时,将wal_level设置为logical,出现一些表更新和删除操作报错,怎么办
在使用实时计算Flink版过程中,可能会遇到各种错误,了解这些错误的原因及解决方法对于高效排错至关重要。针对具体问题,查看Flink的日志是关键,它们通常会提供更详细的错误信息和堆栈跟踪,有助于定位问题。此外,Flink社区文档和官方论坛也是寻求帮助的好去处。以下是一些常见的操作报错及其可能的原因与解决策略。
|
存储 关系型数据库 数据库
How to Optimize PostgreSQL Logical Replication
How to Optimize PostgreSQL Logical Replication
122 0
How to Optimize PostgreSQL Logical Replication
|
SQL 关系型数据库 Java
PostgreSQL增量订阅方案:利用Logical Decoding订阅增量
PostgreSQL增量订阅方案:利用Logical Decoding订阅增量
1117 0
PostgreSQL增量订阅方案:利用Logical Decoding订阅增量
|
关系型数据库 数据库 SQL
PostgreSQL Logical Replication
限制及特性 1、只支持普通表生效,不支持序列、视图、物化视图、外部表、分区表和大对象 2、只支持普通表的DML(INSERT、UPDATE、DELETE)操作,不支持truncate、DDL操作 3、需要同步的表必须设置REPLICA IDENTITY 不能为noting(默认值是default).
8778 0
|
关系型数据库 C语言 PostgreSQL
PostgreSQL 开启with-llvm(JIT)后,新增插件异常(clang: Command not found)处理
标签 PostgreSQL , llvm , clang , jit 背景 PostgreSQL 11版本开始引入了对JIT的支持,在OLAP类型的SQL有比较大的性能提升。 如果你使用的是YUM安装的PG,clang可能没有加入,在后期编译其他插件时可能遇到类似的报错: 比如pg_hint_plan插件 git clone https://github.
2972 0
|
Web App开发 Oracle 关系型数据库
|
关系型数据库 流计算 PostgreSQL
|
流计算 关系型数据库 网络协议