PostgreSQL PITR实验

本文涉及的产品
云原生数据库 PolarDB MySQL 版,Serverless 5000PCU 100GB
简介:

看PostgreSQL中与PITR相关的设定:

复制代码
                                        
wal_level=archive                                        
…                                        
# - Archiving -                        
archive_mode = on                                     # allows archiving to be done 
                                                      # (change requires restart) 
archive_command = 'cp %p "/usr/local/pgsql/arch/%f"'  # command to use to archive a logfile segment                                        
#archive_timeout = 0                                  # force a logfile segment switch after this  
                                                      # number of seconds; 0 disables 
复制代码

开始,在运行之前,看xlog状况:

复制代码
[postgres@pg201 pgsql]$ ls -lrt ./data/pg_xlog                            
total 32812                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001                            
drwx------ 2 postgres postgres     4096 Aug  6 14:16 archive_status                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000002                            
[postgres@pg201 pgsql]$                             
                            
[postgres@pg201 pgsql]$ ls -lrt ./arch                            
total 16404                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001                            
[postgres@pg201 pgsql]$                             
复制代码

为文件拷贝进行准备:

复制代码
[postgres@pg201 pgsql]$ ./bin/psql                
psql (9.1.2)                
Type "help" for help.                
                
postgres=# select pg_start_backup('gao');                
 pg_start_backup                 
-----------------                
 0/3000020                
(1 row)                
                
postgres=#                 
复制代码

此时,xlog和archive log状况如下: 

复制代码
[postgres@pg201 pgsql]$                             
[postgres@pg201 pgsql]$ ls -lrt ./data/pg_xlog                            
total 49216                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000004                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002                            
drwx------ 2 postgres postgres     4096 Aug  6 14:17 archive_status                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000003                            
                            
[postgres@pg201 pgsql]$ ls -lrt ./arch                            
total 32808                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001                            
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002                            
[postgres@pg201 pgsql]$                             
复制代码

完成文件拷贝:

[postgres@pg201 pgsql]$ tar -cvf basedata.tar ./data   

结束基础备份:

复制代码
postgres=# select pg_stop_backup();                            
NOTICE:  pg_stop_backup complete, all required WAL segments have been archived                            
 pg_stop_backup                             
----------------                            
 0/30000A0                            
(1 row)                            
                            
postgres=#                             
复制代码

再次查看log:

复制代码
[postgres@pg201 pgsql]$ ls -lrt ./data/pg_xlog                                
total 49220                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000004 
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002 
drwx------ 2 postgres postgres     4096 Aug  6 14:20 archive_status                                
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup 
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000003                                
[postgres@pg201 pgsql]$                                 
                                
[postgres@pg201 pgsql]$ ls -lrt ./arch                                
total 49216                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup   
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000003                                
[postgres@pg201 pgsql]$                                 
复制代码

可以看到产生了一个backup文件,看内容:

复制代码
[postgres@pg201 pgsql]$ cat ./data/pg_xlog/000000010000000000000003.00000020.backup             
START WAL LOCATION: 0/3000020 (file 000000010000000000000003)            
STOP WAL LOCATION: 0/30000A0 (file 000000010000000000000003)            
CHECKPOINT LOCATION: 0/3000020            
BACKUP METHOD: pg_start_backup            
START TIME: 2013-08-06 14:17:53 CST            
LABEL: gao            
STOP TIME: 2013-08-06 14:20:53 CST            
[postgres@pg201 pgsql]$             
复制代码

强制日志切换,模拟大量数据产生导致日志switch:

复制代码
postgres=# select pg_switch_xlog();            
 pg_switch_xlog             
----------------            
 0/4000000            
(1 row)            
            
postgres=#             
复制代码

log发生了变化:

复制代码
[postgres@pg201 pgsql]$ ls -lrt ./data/pg_xlog                                
total 49220                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000006  
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup 
-rw------- 1 postgres postgres 16777216 Aug  6 14:25 000000010000000000000004  
drwx------ 2 postgres postgres     4096 Aug  6 14:27 archive_status                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:27 000000010000000000000005                                
[postgres@pg201 pgsql]$                                 
[postgres@pg201 pgsql]$ ls -lrt ./arch                                
total 65620                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002 
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup 
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000003 
-rw------- 1 postgres postgres 16777216 Aug  6 14:25 000000010000000000000004                                
[postgres@pg201 pgsql]$                                 
复制代码

再作几次log switch,最后是这样:

复制代码
[postgres@pg201 pgsql]$ ls -lrt ./data/pg_xlog                                
total 49220                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000006 
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup  
-rw------- 1 postgres postgres 16777216 Aug  6 14:25 000000010000000000000004  
drwx------ 2 postgres postgres     4096 Aug  6 14:27 archive_status                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:27 000000010000000000000005                                
[postgres@pg201 pgsql]$                                 
[postgres@pg201 pgsql]$ ls -lrt ./arch                                
total 65620                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001 
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002 
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000003
-rw------- 1 postgres postgres 16777216 Aug  6 14:25 000000010000000000000004   
[postgres@pg201 pgsql]$                                 
复制代码

经过一次正常关机后,日志再次变化:

[postgres@pg201 pgsql]$ ./bin/pg_ctl -D ./data stop                    

日志变为:

复制代码
[postgres@pg201 pgsql]$ ls -lrt ./data/pg_xlog                                
total 49220                                
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup
-rw------- 1 postgres postgres 16777216 Aug  6 14:25 000000010000000000000007 
drwx------ 2 postgres postgres     4096 Aug  6 14:39 archive_status                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:39 000000010000000000000006  
-rw------- 1 postgres postgres 16777216 Aug  6 14:39 000000010000000000000005                                
[postgres@pg201 pgsql]$                                 
[postgres@pg201 pgsql]$ ls -lrt ./arch                                
total 82024                                
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001 
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002 
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup 
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000003 
-rw------- 1 postgres postgres 16777216 Aug  6 14:25 000000010000000000000004 
-rw------- 1 postgres postgres 16777216 Aug  6 14:39 000000010000000000000005                                
[postgres@pg201 pgsql]$                                 
复制代码

重新启动PostgreSQL,添加数据,然后杀掉进程,模拟崩溃:

复制代码
[postgres@pg201 pgsql]$ ./bin/psql
psql (9.1.2)
Type "help" for help.

postgres=# create table test(id integer);
CREATE TABLE
postgres=# insert into test values(1);
INSERT 0 1
postgres=# \q
[postgres@pg201 pgsql]$ 
复制代码
复制代码
[root@pg201 ~]# ps -ef | grep post
root      3120  3049  0 14:08 pts/1    00:00:00 su - postgres
postgres  3121  3120  0 14:08 pts/1    00:00:00 -bash
root      3216  3191  0 14:15 pts/2    00:00:00 su - postgres
postgres  3217  3216  0 14:15 pts/2    00:00:00 -bash
root      3299  3279  0 14:17 pts/3    00:00:00 su - postgres
postgres  3300  3299  0 14:17 pts/3    00:00:00 -bash
postgres  3516     1  0 15:00 pts/2    00:00:00 /usr/local/pgsql/bin/postgres -D ./data
postgres  3518  3516  0 15:00 ?        00:00:00 postgres: writer process               
postgres  3519  3516  0 15:00 ?        00:00:00 postgres: wal writer process           
postgres  3520  3516  0 15:00 ?        00:00:00 postgres: autovacuum launcher process   
postgres  3521  3516  0 15:00 ?        00:00:00 postgres: archiver process             
postgres  3522  3516  0 15:00 ?        00:00:00 postgres: stats collector process      
root      3559  3535  0 15:01 pts/4    00:00:00 grep post
[root@pg201 ~]# kill -s SIGQUIT 3516
[root@pg201 ~]# 
复制代码

下面开始回复动作、找出基础备份:

[postgres@pg201 pgsql]$ tar -xvf basedata.tar ./data

其pg_xlog目录下,还是旧的online redo log:

复制代码
[postgres@pg201 pg_xlog]$ ls -lrt
total 49216
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000004
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002
drwx------ 2 postgres postgres     4096 Aug  6 14:17 archive_status
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000003
[postgres@pg201 pg_xlog]$ 
复制代码

把它清空后,不补足online redolog,仅仅用 archive log的效果:

在解压好的data目录下,建立 recovery.conf文件:

[postgres@pg201 data]$ cat recovery.conf
restore_command = 'cp /usr/local/pgsql/arch/%f %p'
[postgres@pg201 data]$ 

启动PostgreSQL:

复制代码
[postgres@pg201 pgsql]$ ./bin/pg_ctl -D ./data status
pg_ctl: no server running
[postgres@pg201 pgsql]$ ./bin/pg_ctl -D ./data start
pg_ctl: another server might be running; trying to start server anyway
server starting
[postgres@pg201 pgsql]$ LOG:  database system was interrupted; last known up at 2013-08-06 14:17:53 CST
LOG:  creating missing WAL directory "pg_xlog/archive_status"
LOG:  starting archive recovery
LOG:  restored log file "000000010000000000000003" from archive
LOG:  redo starts at 0/3000078
LOG:  consistent recovery state reached at 0/4000000
LOG:  restored log file "000000010000000000000004" from archive
LOG:  restored log file "000000010000000000000005" from archive
cp: cannot stat `/usr/local/pgsql/arch/000000010000000000000006': No Such File or Directory
LOG:  could not open file "pg_xlog/000000010000000000000006" (log file 0, segment 6): No Such File or Directory
LOG: redo done at 0/5000078 LOG: restored log file "000000010000000000000005" from archive cp: cannot stat `/usr/local/pgsql/arch/00000002.history': No Such File or Directory
LOG: selected new timeline ID: 2 cp: cannot stat `/usr/local/pgsql/arch/00000001.history':No Such File or Directory
LOG: archive recovery complete 
LOG: autovacuum launcher started
LOG: database system is ready to accept connections
复制代码

因为我的创建table test 的动作虽然计入了 redo log中,但是由于突然的崩溃,没有来得及放入archive 日志中,所以,没有得到恢复:

复制代码
[postgres@pg201 pgsql]$ ./bin/psql
psql (9.1.2)
Type "help" for help.

postgres=# select * from test;
ERROR:  relation "test" does not exist
LINE 1: select * from test;
                      ^
postgres=# 
复制代码

此时,看Log,就可以清晰地看到TimeLine的痕迹了:

复制代码
[postgres@pg201 pgsql]$ ls -lrt ./data/pg_xlog
total 49220
-rw------- 1 postgres postgres       56 Aug  6 15:11 00000002.history
-rw------- 1 postgres postgres 16777216 Aug  6 15:11 000000020000000000000005
-rw------- 1 postgres postgres 16777216 Aug  6 15:17 000000020000000000000006
drwx------ 2 postgres postgres     4096 Aug  6 15:17 archive_status
-rw------- 1 postgres postgres 16777216 Aug  6 15:17 000000020000000000000007
[postgres@pg201 pgsql]$ 
复制代码
复制代码
[postgres@pg201 pgsql]$ ls -lrt ./arch
total 114836
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000001
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002
-rw------- 1 postgres postgres      267 Aug  6 14:20 000000010000000000000003.00000020.backup
-rw------- 1 postgres postgres 16777216 Aug  6 14:20 000000010000000000000003
-rw------- 1 postgres postgres 16777216 Aug  6 14:25 000000010000000000000004
-rw------- 1 postgres postgres 16777216 Aug  6 14:39 000000010000000000000005
-rw------- 1 postgres postgres       56 Aug  6 15:11 00000002.history
-rw------- 1 postgres postgres 16777216 Aug  6 15:17 000000020000000000000006
-rw------- 1 postgres postgres 16777216 Aug  6 15:17 000000020000000000000005
[postgres@pg201 pgsql]$ cat ./arch/00000002.history
1       000000010000000000000005        no recovery target specified
[postgres@pg201 pgsql]$ 
复制代码

这次再看清空pg_xlog目录后,补足online redolog+archive log的效果:

还要把 pg_xlog/archive_status 目录清空:

复制代码
[postgres@pg201 pgsql]$ cp ./data.bak/pg_xlog/*  ./data/pg_xlog/
...

[postgres@pg201 pg_xlog]$ ls -lrt
total 98432
-rw------- 1 postgres postgres 16777216 Aug  6 14:16 000000010000000000000004
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000002
-rw------- 1 postgres postgres 16777216 Aug  6 14:17 000000010000000000000003
-rw------- 1 postgres postgres      267 Aug  6 15:41 000000010000000000000003.00000020.backup
-rw------- 1 postgres postgres 16777216 Aug  6 15:41 000000010000000000000007
-rw------- 1 postgres postgres 16777216 Aug  6 15:41 000000010000000000000006
-rw------- 1 postgres postgres 16777216 Aug  6 15:41 000000010000000000000005
drwx------ 2 postgres postgres     4096 Aug  6 15:44 archive_status
[postgres@pg201 pg_xlog]$ 
复制代码

此时,启动 PostgreSQL:

复制代码
[postgres@pg201 pgsql]$ ./bin/pg_ctl -D ./data start
pg_ctl: another server might be running; trying to start server anyway
server starting
[postgres@pg201 pgsql]$ LOG:  database system was interrupted; last known up at 2013-08-06 14:17:53 CST
LOG:  starting archive recovery
LOG:  restored log file "000000010000000000000003" from archive
LOG:  redo starts at 0/3000078
LOG:  consistent recovery state reached at 0/4000000
LOG:  restored log file "000000010000000000000004" from archive
LOG:  restored log file "000000010000000000000005" from archive
cp: cannot stat `/usr/local/pgsql/arch/000000010000000000000006': No such file or directory
LOG:  record with zero length at 0/6011770
LOG:  redo done at 0/6011730
LOG:  last completed transaction was at log time 2013-08-06 15:01:17.757945+08
LOG:  restored log file "00000002.history" from archive
cp: cannot stat `/usr/local/pgsql/arch/00000003.history': No such file or directory
LOG:  selected new timeline ID: 3
cp: cannot stat `/usr/local/pgsql/arch/00000001.history': No such file or directory
LOG:  archive recovery complete
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
复制代码

看数据:

复制代码
[postgres@pg201 pgsql]$ ./bin/psql
psql (9.1.2)
Type "help" for help.

postgres=# select * from test;
 id 
----
  1
(1 row)

postgres=# 
复制代码






相关实践学习
使用PolarDB和ECS搭建门户网站
本场景主要介绍基于PolarDB和ECS实现搭建门户网站。
阿里云数据库产品家族及特性
阿里云智能数据库产品团队一直致力于不断健全产品体系,提升产品性能,打磨产品功能,从而帮助客户实现更加极致的弹性能力、具备更强的扩展能力、并利用云设施进一步降低企业成本。以云原生+分布式为核心技术抓手,打造以自研的在线事务型(OLTP)数据库Polar DB和在线分析型(OLAP)数据库Analytic DB为代表的新一代企业级云原生数据库产品体系, 结合NoSQL数据库、数据库生态工具、云原生智能化数据库管控平台,为阿里巴巴经济体以及各个行业的企业客户和开发者提供从公共云到混合云再到私有云的完整解决方案,提供基于云基础设施进行数据从处理、到存储、再到计算与分析的一体化解决方案。本节课带你了解阿里云数据库产品家族及特性。
目录
相关文章
|
弹性计算 网络协议 容灾
PostgreSQL 时间点恢复(PITR)在异步流复制主从模式下,如何避免主备切换后PITR恢复(备库、容灾节点、只读节点)走错时间线(timeline , history , partial , restore_command , recovery.conf)
标签 PostgreSQL , 恢复 , 时间点恢复 , PITR , restore_command , recovery.conf , partial , history , 任意时间点恢复 , timeline , 时间线 背景 政治正确非常重要,对于数据库来说亦如此,一个基于流复制的HA架构的集群,如果还有一堆只读节点,当HA集群发生了主备切换后,这些只读节点能否与新的主节点保持
1673 0
|
12月前
|
运维 关系型数据库 MySQL
企业运维训练营之数据库原理与实践— 数据管理DMS—实验:如何快速连接云数据库RDS MySQL(上)
企业运维训练营之数据库原理与实践— 数据管理DMS—实验:如何快速连接云数据库RDS MySQL(上)
178 0
|
12月前
|
SQL 运维 数据可视化
企业运维训练营之数据库原理与实践— 数据管理DMS—实验:如何快速连接云数据库RDS MySQL(中)
企业运维训练营之数据库原理与实践— 数据管理DMS—实验:如何快速连接云数据库RDS MySQL(中)
132 0
|
12月前
|
SQL 运维 数据可视化
企业运维训练营之数据库原理与实践— 数据管理DMS—实验:如何快速连接云数据库RDS MySQL(下)
企业运维训练营之数据库原理与实践— 数据管理DMS—实验:如何快速连接云数据库RDS MySQL(下)
119 0
|
12月前
|
弹性计算 运维 关系型数据库
企业运维训练营之数据库原理与实践—RDS常见问题排除及DAS自动弹性伸缩—实验:RDS MySQL大促场景下的智能化弹性
企业运维训练营之数据库原理与实践—RDS常见问题排除及DAS自动弹性伸缩—实验:RDS MySQL大促场景下的智能化弹性
136 0
|
弹性计算 容灾 关系型数据库
PostgreSQL PITR 任意时间点恢复过程中如何手工得到recovery需要的下一个WAL文件名 - 默认情况下restore_command自动获取
标签 PostgreSQL , recovery , recovery.conf , restore_command , timeline , 时间线 , next wal , PITR , 时间点恢复 背景 PostgreSQL数据库支持PITR时间点恢复。默认情况下,只需要配置目标是时间点,resotre_command即可,PG会自动调用resotre_command去找需要的WA
1372 0
|
关系型数据库 数据库 PostgreSQL
【独家直播】 德哥PG系列课程15讲—PostgreSQL 多场景 沙箱实验从入门到精通
为了能够让用户可以更快的上手PostgreSQL,德哥与云栖团队的小伙伴制作了一系列阿里云RDS PostgreSQL沙箱实验。所有实验内容都结合了应用场景,非常具有代表性,兴趣性与一定的挑战性。
5263 0
|
监控 关系型数据库 Shell
PostgreSQL 10.1 手册_部分 III. 服务器管理_第 25 章 备份和恢复_25.3. 连续归档和时间点恢复(PITR)
25.3. 连续归档和时间点恢复(PITR) 25.3.1. 建立WAL归档 25.3.2. 制作一个基础备份 25.3.3. 使用低级API制作一个基础备份 25.3.4. 使用一个连续归档备份进行恢复 25.3.5. 时间线 25.3.6. 建议和例子 25.3.7. 警告 在任何时间,PostgreSQL在数据集簇目录的pg_wal/子目录下都保持有一个预写式日志(WAL)。
1863 0
|
关系型数据库 数据库 PostgreSQL
对PostgreSQL数据库的hstore类型建立GisT索引的实验
磨砺技术珠矶,践行数据之道,追求卓越价值回到上一级页面:PostgreSQL基础知识与基本操作索引页    回到顶级页面:PostgreSQL索引页[作者 高健@博客园  luckyjackgao@gmail.
2438 0