pgbackrest 简要使用说明

简介:

测试环境:

CentOS Linux release 7.4.1708 (Core) X64

PostgreSQL: 10.1(Source)

pgbackrest: pgbackrest-release-1.27.tar.gz

 

1.Test-Simple-1.302120.tar.gz

# perl Makefile.PL

# make

# make test

......

......

All tests successful.

Files=220, Tests=2514, 29 wallclock secs ( 1.10 usr  0.60 sys + 17.97 cusr  4.61 csys = 24.28 CPU)

Result: PASS

#

# make install



2.DBI-1.637.tar.gz


# tar zxvf DBI-1.637.tar.gz

# cd DBI-1.637

# perl Makefile.PL

# make

# make test

......

......

All tests successful.

Files=194, Tests=9005, 113 wallclock secs ( 2.57 usr  0.80 sys + 94.07 cusr 12.95 csys = 110.39 CPU)

Result: PASS

PERL_DL_NONLAZY=1 /usr/bin/perl "-Iblib/lib" "-Iblib/arch" test.pl

test.pl 

DBI test application $Revision$

Switch: DBI 1.637 by Tim Bunce, 1.637

Available Drivers: DBM, ExampleP, File, Gofer, Proxy, Sponge

dbi:ExampleP:: testing 3 sets of 20 connections:

Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 

Disconnecting...

Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 

Disconnecting...

Connecting... 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 

Disconnecting...

connect 20 and disconnect them, 3 times: 0.0040s / 60 = 0.0001s

Testing handle creation speed...

51282 NullP sth/s perl 5.016003 x86_64-linux-thread-multi (gcc 4.8.5 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic) 0.000019s


test.pl done

#

# make install


3.version-0.9918.tar.gz

# perl Makefile.PL

# make

# make test

......

......


All tests successful.


Test Summary Report

-------------------

t/07locale.t    (Wstat: 0 Tests: 8 Failed: 0)

  TODO passed:   7

Files=12, Tests=2502,  2 wallclock secs ( 0.45 usr  0.12 sys +  1.05 cusr  1.13 csys =  2.75 CPU)

Result: PASS

make[1]: Entering directory `/home/soft/dbd-pg/version-0.9918/vutil'

No tests defined for version::vxs extension.

make[1]: Leaving directory `/home/soft/dbd-pg/version-0.9918/vutil'

# make install


4.DBD-Pg-3.7.0.tar.gz


export PATH=/opt/pgsql/10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin

export LD_LIBRARY_PATH=/opt/pgsql/10.1/lib

export MANPATH=/opt/pgsql/10.1/share/man:$MANPATH



# perl Makefile.PL 

Configuring DBD::Pg 3.7.0

PostgreSQL version: 100001 (default port: 5432)

POSTGRES_HOME: (not set)

POSTGRES_INCLUDE: /opt/pgsql/10.1/include

POSTGRES_LIB: /opt/pgsql/10.1/lib

OS: linux

Checking if your kit is complete...

Looks good

Using DBI 1.637 (for perl 5.016003 on x86_64-linux-thread-multi) installed in /usr/local/lib64/perl5/auto/DBI/

Writing Makefile for DBD::Pg

#


# make

cp lib/Bundle/DBD/Pg.pm blib/lib/Bundle/DBD/Pg.pm

cp Pg.pm blib/lib/DBD/Pg.pm

/usr/bin/perl -p -e "s/~DRIVER~/Pg/g; s/^do\(/dontdo\(/" /usr/local/lib64/perl5/auto/DBI/Driver.xst > Pg.xsi

/usr/bin/perl /usr/share/perl5/vendor_perl/ExtUtils/xsubpp  -typemap /usr/share/perl5/ExtUtils/typemap  Pg.xs > Pg.xsc && mv Pg.xsc Pg.c

gcc -c  -I/opt/pgsql/10.1/include -I/usr/local/lib64/perl5/auto/DBI -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPGLIBVERSION=100001 -DPGDEFPORT=5432 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic  -DPERL_EXTMALLOC_DEF -Dmalloc=Perl_malloc -Dfree=Perl_mfree -Drealloc=Perl_realloc -Dcalloc=Perl_calloc -DVERSION=\"3.7.0\" -DXS_VERSION=\"3.7.0\" -fPIC "-I/usr/lib64/perl5/CORE"   Pg.c

gcc -c  -I/opt/pgsql/10.1/include -I/usr/local/lib64/perl5/auto/DBI -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPGLIBVERSION=100001 -DPGDEFPORT=5432 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic  -DPERL_EXTMALLOC_DEF -Dmalloc=Perl_malloc -Dfree=Perl_mfree -Drealloc=Perl_realloc -Dcalloc=Perl_calloc -DVERSION=\"3.7.0\" -DXS_VERSION=\"3.7.0\" -fPIC "-I/usr/lib64/perl5/CORE"   dbdimp.c

gcc -c  -I/opt/pgsql/10.1/include -I/usr/local/lib64/perl5/auto/DBI -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPGLIBVERSION=100001 -DPGDEFPORT=5432 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic  -DPERL_EXTMALLOC_DEF -Dmalloc=Perl_malloc -Dfree=Perl_mfree -Drealloc=Perl_realloc -Dcalloc=Perl_calloc -DVERSION=\"3.7.0\" -DXS_VERSION=\"3.7.0\" -fPIC "-I/usr/lib64/perl5/CORE"   quote.c

In file included from Pg.h:59:0,

                 from quote.c:10:

/usr/local/lib64/perl5/auto/DBI/DBIXS.h:491:25: warning: ‘dbi_get_state’ defined but not used [-Wunused-function]

     static dbistate_t** dbi_get_state(pTHX) {                               \

                         ^

/usr/local/lib64/perl5/auto/DBI/DBIXS.h:504:27: note: in expansion of macro ‘_DBISTATE_DECLARE_COMMON’

 # define DBISTATE_DECLARE _DBISTATE_DECLARE_COMMON

                           ^

Pg.h:65:1: note: in expansion of macro ‘DBISTATE_DECLARE’

 DBISTATE_DECLARE;

 ^

gcc -c  -I/opt/pgsql/10.1/include -I/usr/local/lib64/perl5/auto/DBI -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPGLIBVERSION=100001 -DPGDEFPORT=5432 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic  -DPERL_EXTMALLOC_DEF -Dmalloc=Perl_malloc -Dfree=Perl_mfree -Drealloc=Perl_realloc -Dcalloc=Perl_calloc -DVERSION=\"3.7.0\" -DXS_VERSION=\"3.7.0\" -fPIC "-I/usr/lib64/perl5/CORE"   types.c

In file included from Pg.h:59:0,

                 from types.c:15:

/usr/local/lib64/perl5/auto/DBI/DBIXS.h:491:25: warning: ‘dbi_get_state’ defined but not used [-Wunused-function]

     static dbistate_t** dbi_get_state(pTHX) {                               \

                         ^

/usr/local/lib64/perl5/auto/DBI/DBIXS.h:504:27: note: in expansion of macro ‘_DBISTATE_DECLARE_COMMON’

 # define DBISTATE_DECLARE _DBISTATE_DECLARE_COMMON

                           ^

Pg.h:65:1: note: in expansion of macro ‘DBISTATE_DECLARE’

 DBISTATE_DECLARE;

 ^

Running Mkbootstrap for DBD::Pg ()

chmod 644 Pg.bs

rm -f blib/arch/auto/DBD/Pg/Pg.so

gcc  -shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Wl,-z,relro  Pg.o dbdimp.o quote.o types.o  -o blib/arch/auto/DBD/Pg/Pg.so       \

   -L/opt/pgsql/10.1/lib -lpq -lm       \

  

chmod 755 blib/arch/auto/DBD/Pg/Pg.so

cp Pg.bs blib/arch/auto/DBD/Pg/Pg.bs

chmod 644 blib/arch/auto/DBD/Pg/Pg.bs

Manifying blib/man3/Bundle::DBD::Pg.3pm

Manifying blib/man3/DBD::Pg.3pm

[root@localhost DBD-Pg-3.7.0]# make test

PGINITDB="/opt/pgsql/10.1/bin/initdb" PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t

t/00_signature.t .... skipped: Set the environment variable TEST_SIGNATURE to enable this test

t/00basic.t ......... ok   

t/01connect.t ....... 1/15 # 

# DBI                         Version 1.637

# DBD::Pg                     Version 3.7.0

# Perl                        Version 5.16.3

# OS                          linux

# PostgreSQL (compiled)       100001

# PostgreSQL (target)         100001

# PostgreSQL (reported)       PostgreSQL 10.1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16), 64-bit

# Default port                5432

# DBI_DSN                     dbi:Pg:

# DBI_USER                    postgres

# Test schema                 dbd_pg_testschema

# LANG                        en_US.UTF-8

# array_nulls                 on

# backslash_quote             safe_encoding

# client_encoding             UTF8

# server_encoding             UTF8

# standard_conforming_strings on

# Adjusted:                   DBI_DSN 

t/01connect.t ....... ok     

t/01constants.t ..... ok     

t/02attribs.t ....... ok       

t/03dbmethod.t ...... ok       

t/03smethod.t ....... ok       

t/04misc.t .......... ok     

t/06bytea.t ......... ok     

t/07copy.t .......... ok     

t/08async.t ......... ok     

t/09arrays.t ........ ok       

t/12placeholders.t .. ok       

t/20savepoints.t .... ok   

t/30unicode.t ....... ok     

t/99cleanup.t ....... ok   

All tests successful.

Files=16, Tests=2129, 26 wallclock secs ( 0.57 usr  0.13 sys +  3.78 cusr  4.35 csys =  8.83 CPU)

Result: PASS

[root@localhost DBD-Pg-3.7.0]# make install

Files found in blib/arch: installing files in blib/lib into architecture dependent library tree

Installing /usr/local/lib64/perl5/auto/DBD/Pg/Pg.so

Installing /usr/local/lib64/perl5/auto/DBD/Pg/Pg.bs

Installing /usr/local/lib64/perl5/DBD/Pg.pm

Installing /usr/local/lib64/perl5/Bundle/DBD/Pg.pm

Installing /usr/local/share/man/man3/Bundle::DBD::Pg.3pm

Installing /usr/local/share/man/man3/DBD::Pg.3pm

Appending installation info to /usr/lib64/perl5/perllocal.pod

#


5.other perl module


# yum install -y perl-IO-Socket-SSL perl-XML-LibXML openssl-devel perl-Digest-SHA perl-JSON-PP


6.pgbackrest


# tar zxvf pgbackrest-release-1.27.tar.gz

# cd pgbackrest-release-1.27


######  lib ######

# cp -r lib/pgBackRest /usr/share/perl5

# find /usr/share/perl5/pgBackRest -type f -exec chmod 644 {} +

# find /usr/share/perl5/pgBackRest -type d -exec chmod 755 {} +


######  bin ###### 

# cp bin/pgbackrest /usr/bin/pgbackrest

# chmod 755 /usr/bin/pgbackrest


######  log ###### 

# mkdir -m 770 /var/log/pgbackrest

# chown postgres:postgres /var/log/pgbackrest


###### conf ######

# touch /etc/pgbackrest.conf

# chmod 640 /etc/pgbackrest.conf

# chown postgres:postgres /etc/pgbackrest.conf


######  Build and Install C Library ######


# cd libc/

# perl Makefile.PL INSTALLMAN1DIR=none INSTALLMAN3DIR=none

Writing Makefile for pgBackRest::LibC

#

# make test

......

......

All tests successful.

Files=1, Tests=4,  1 wallclock secs ( 0.02 usr  0.01 sys +  0.12 cusr  0.03 csys =  0.18 CPU)

Result: PASS


# make install

Files found in blib/arch: installing files in blib/lib into architecture dependent library tree

Installing /usr/local/lib64/perl5/auto/pgBackRest/LibC/LibC.so

Installing /usr/local/lib64/perl5/auto/pgBackRest/LibC/LibC.bs

Installing /usr/local/lib64/perl5/pgBackRest/LibC.pm

Installing /usr/local/lib64/perl5/pgBackRest/LibCAuto.pm

Installing /usr/local/lib64/perl5/auto/pgBackRest/LibC/autosplit.ix

Appending installation info to /usr/lib64/perl5/perllocal.pod



# su - postgres

Last login: Thu Dec 21 15:50:54 CST 2017 on pts/1

[postgres@localhost ~]$ pgbackrest --help

Unknown option: help


pgBackRest 1.27 - General help


Usage:

    pgbackrest [options] [command]


Commands:

    archive-get     Get a WAL segment from the archive.

    archive-push    Push a WAL segment to the archive.

    backup          Backup a database cluster.

    check           Check the configuration.

    expire          Expire backups that exceed retention.

    help            Get help.

    info            Retrieve information about backups.

    restore         Restore a database cluster.

    stanza-create   Create the required stanza data.

    stanza-upgrade  Upgrade a stanza.

    start           Allow pgBackRest processes to run.

    stop            Stop pgBackRest processes from running.

    version         Get version.


Use 'pgbackrest help [command]' for more information.

[postgres@localhost ~]$ 



###### root ######

# mkdir /var/lib/pgbackrest

# chmod 750 /var/lib/pgbackrest

# chown postgres:postgres /var/lib/pgbackrest


# su - postgres


$ vi postgresql.conf

listen_addresses = '*'

wal_level = logical

archive_mode = on

archive_command = 'pgbackrest --stanza=db archive-push %p'

#log_line_prefix = '%m [%p] '

#max_wal_senders = 10

wal_keep_segments = 5000 

#hot_standby = on


$ openssl rand -base64 48

lFhhFeDQPazx59LPvztsWu5leUqsiOwhI8JT7gMAmACMVX+Kcwak5qDOVNiiNkqX



$ vi /etc/pgbackrest.conf 

# 仅包含字母,不能为db01

[db]

db-path=/pgdata10


[global]

repo-cipher-pass=lFhhFeDQPazx59LPvztsWu5leUqsiOwhI8JT7gMAmACMVX+Kcwak5qDOVNiiNkqX

repo-cipher-type=aes-256-cbc

repo-path=/var/lib/pgbackrest

retention-full=2

process-max=3

start-fast=y

stop-auto=y



$ vi .bash_profile


# User specific environment and startup programs


export LD_LIBRARY_PATH=/opt/pgsql/10.1/lib

export MANPATH=/opt/pgsql/10.1/share/man:$MANPATH

export PATH=/opt/pgsql/10.1/bin:$PATH:$HOME/.local/bin:$HOME/bin



1).create repository


$ pgbackrest --stanza=db --log-level-console=info stanza-create

2017-12-21 16:14:25.331 P00   INFO: stanza-create command begin 1.27: --db1-path=/pgdata10 --log-level-console=info --repo-path=/var/lib/pgbackrest --stanza=db01

2017-12-21 16:14:25.801 P00   INFO: stanza-create command end: completed successfully


$ pgbackrest --stanza=db --log-level-console=info check

2017-12-21 16:46:45.886 P00   INFO: check command begin 1.27: --db1-path=/pgdata10 --log-level-console=info --repo-path=/var/lib/pgbackrest --stanza=db

2017-12-21 16:46:48.243 P00   INFO: WAL segment 000000010000000000000005 successfully stored in the archive at '/var/lib/pgbackrest/archive/db/10-1/0000000100000000/000000010000000000000005-a85cb846cee3a5a47b487415a2570ce7b5fbf2f8.gz'

2017-12-21 16:46:48.244 P00   INFO: check command end: completed successfully


2).备份操作


###### full ######


pgbackrest --stanza=db --type=full --log-level-console=info backup


###### incremental ######


$ pgbackrest --stanza=db --type=incr --log-level-console=info backup


###### diff ######


pgbackrest --stanza=db --type=diff --log-level-console=info backup


###### display backup ######  


$ pgbackrest --stanza=db --log-level-console=info info


###### restore(--delta, PGDATA base exist part file,miss some files ) ######


$ pgbackrest --stanza=db --delta --log-level-console=info restore


###### restore(--delta, PGDATA none file) ######

$ pgbackrest --stanza=db --log-level-console=info restore




本文转自 pgmia 51CTO博客,原文链接:http://blog.51cto.com/heyiyi/2053039
相关实践学习
使用PolarDB和ECS搭建门户网站
本场景主要介绍如何基于PolarDB和ECS实现搭建门户网站。
阿里云数据库产品家族及特性
阿里云智能数据库产品团队一直致力于不断健全产品体系,提升产品性能,打磨产品功能,从而帮助客户实现更加极致的弹性能力、具备更强的扩展能力、并利用云设施进一步降低企业成本。以云原生+分布式为核心技术抓手,打造以自研的在线事务型(OLTP)数据库Polar DB和在线分析型(OLAP)数据库Analytic DB为代表的新一代企业级云原生数据库产品体系, 结合NoSQL数据库、数据库生态工具、云原生智能化数据库管控平台,为阿里巴巴经济体以及各个行业的企业客户和开发者提供从公共云到混合云再到私有云的完整解决方案,提供基于云基础设施进行数据从处理、到存储、再到计算与分析的一体化解决方案。本节课带你了解阿里云数据库产品家族及特性。
相关文章
|
API
阿里云的ascm上如何使用api接口
阿里云的ascm上如何使用api接口
2870 1
|
12月前
|
存储 关系型数据库 分布式数据库
PolarDB开源数据库进阶课18 通过pg_bulkload适配pfs实现批量导入提速
本文介绍了如何修改 `pg_bulkload` 工具以适配 PolarDB 的 PFS(Polar File System),从而加速批量导入数据。实验环境依赖于 Docker 容器中的 loop 设备模拟共享存储。通过对 `writer_direct.c` 文件的修改,替换了一些标准文件操作接口为 PFS 对应接口,实现了对 PolarDB 15 版本的支持。测试结果显示,使用 `pg_bulkload` 导入 1000 万条数据的速度是 COPY 命令的三倍多。此外,文章还提供了详细的步骤和代码示例,帮助读者理解和实践这一过程。
546 0
|
6月前
|
算法 Java 定位技术
迷宫问题
迷宫问题是指在给定区域内寻找从起点到终点的可行路径。可以使用回溯算法解决,通过不断尝试四个方向(上下左右)移动,若无法前进则回退,直到找到终点或遍历所有可能路径。文中还给出了C语言、Java和Python的实现代码,并展示了运行结果。
266 0
|
6月前
|
存储 关系型数据库 数据库
【赵渝强老师】PostgreSQL数据库的WAL日志与数据写入的过程
PostgreSQL中的WAL(预写日志)是保证数据完整性的关键技术。在数据修改前,系统会先将日志写入WAL,确保宕机时可通过日志恢复数据。它减少了磁盘I/O,提升了性能,并支持手动切换日志文件。WAL文件默认存储在pg_wal目录下,采用16进制命名规则。此外,PostgreSQL提供pg_waldump工具解析日志内容。
599 0
|
11月前
|
人工智能 安全 搜索推荐
如何使用DeepSeek提高工作效率和生活质量?
普通工作者可通过DeepSeek显著提升效率和生活质量。工作方面,3秒生成文档、10分钟完成会议管理、数据处理自动化;生活方面,规划旅行、制定食谱、即时学习助手。使用技巧如“角色+任务+具体要求”提问公式,每天节省2小时,逐步培养“AI优先”思维,让琐事时间用于自我提升或陪伴家人。
522 0
|
SQL 关系型数据库 数据库
【一文搞懂PGSQL】4.逻辑备份和物理备份 pg_dump/ pg_basebackup
本文介绍了PostgreSQL数据库的备份与恢复方法,包括数据和归档日志的备份,以及使用`pg_dump`和`pg_basebackup`工具进行逻辑备份和物理备份的具体操作。通过示例展示了单库和单表的备份与恢复过程,并提供了错误处理方案。此外,还详细描述了如何利用物理备份工具进行数据损坏修复及特定时间点恢复(PITR)的操作步骤,以应对误操作导致的数据丢失问题。
|
机器学习/深度学习 人工智能 自然语言处理
《零基础实践深度学习》1.4.1飞桨产业级深度学习开源开放平台介绍
这篇文章详细介绍了飞桨(PaddlePaddle)这一产业级深度学习开源开放平台,阐述了其在深度学习领域的广泛应用和重要性,以及飞桨平台的核心框架、开发套件、工具组件和预训练模型等,旨在帮助开发者快速实现AI想法并推动产业智能化升级。
|
存储 Oracle 安全
Oracle 12c的不可见字段:数据的“隐形斗篷”
【4月更文挑战第19天】Oracle 12c引入了不可见字段,这是一种隐藏数据列的特性,用于增强数据安全性和实现业务逻辑。不可见字段在常规查询中不显示,但参与数据操作,适合存储敏感信息或内部元数据。创建时只需在列定义中指定“INVISIBLE”属性。虽然有其限制,如特定查询可能暴露,但正确使用能有效提升数据安全性。了解和利用这一功能对数据管理员至关重要。
|
Kubernetes 监控 安全
eBPF ,让观测性走向神坛
Hello folks,我是 Luga,今天我们来介绍一下“下一代”可观测性工具 - eBPF,作为一种强大的内核技术,eBPF 启用了全新类别的可观测性模型,除此之外, 其程序能够无缝地与各种内核关联,以收集有关正在发生的事件数据。
603 1