PostgreSQL cheat functions - (内存上下文\planner内容\memory context等常用函数)

本文涉及的产品
云原生数据库 PolarDB 分布式版,标准版 2核8GB
RDS PostgreSQL Serverless,0.5-4RCU 50GB 3个月
推荐场景:
对影评进行热评分析
RDS SQL Server Serverless,2-4RCU 50GB 3个月
推荐场景:
简介: 标签PostgreSQL , memory context , plan , pid signal , ...背景https://github.com/MasaoFujii/pg_cheat_funcsFujii Masao MasaoFujii PostgreSQL comm...

标签

PostgreSQL , memory context , plan , pid signal , ...


背景

https://github.com/MasaoFujii/pg_cheat_funcs

Fujii Masao  
MasaoFujii  
PostgreSQL committer, DBA at NTT DATA, maybe good husband & father.  

这个插件汇集了常用的PG扩展函数(将来也许会引入到PG社区中),比如

1、查看内存上下文,

2、查看SQL的绑定变量的通用成本,

3、plan成本状态(算法见《PostgreSQL 12 preview - plan_cache_mode参数控制强制使用plan cache或强制custom plan (force_custom_plan and force_generic_plan)》 《执行计划选择算法 与 绑定变量 - PostgreSQL prepared statement: SPI_prepare, prepare|execute COMMAND, PL/pgsql STYLE: custom & generic plan cache》 ),

4、给PID发信号,

5、设置进程优先级,

6、设置nextxid,nextoid,

7、字符串转换,

8、进制转换,

9、压缩,解压缩等。

10、打印内存。

https://github.com/postgrespro/memstat

Module adds statistic report about memory contexts in local and all backends.


    Functions:

setof (name text, level integer, nblocks bigint, freechunks bigint, totalspace bigint, freespace bigint) local_memory_stats()

prints memory context's statistic for current backend

setof (pid integer, name text, level integer, nblocks bigint, freechunks bigint, totalspace bigint, freespace bigint) instance_memory_stats()

prints memory context's statistic for all alive backend, works if library was preloaded via shared_preload_libraries.




    view:

memory_stats

prints per backend summary memory statistics




To use instance_memory_stats() it's needed to add memstat library to shared_preload_libraries. And it should be last in that list!



GUC variable:

memstat.period = 10 # seconds

Module collects memory statistics at a begining of each query and it could be expensive on highloaded instances, so, this variable set minimal time between statistic obtaining.

例子

. ./env11.sh  
  
git clone https://github.com/MasaoFujii/pg_cheat_funcs  
  
cd pg_cheat_funcs/  
  
USE_PGXS=1 make  
  
USE_PGXS=1 make install  
postgres=# create extension pg_cheat_funcs ;  
CREATE EXTENSION  

1、打印内存上下文

postgres=# select * from pg_stat_get_memory_context();  
           name           |       parent       | level | total_bytes | total_nblocks | free_bytes | free_chunks | used_bytes   
--------------------------+--------------------+-------+-------------+---------------+------------+-------------+------------  
 TopMemoryContext         |                    |     0 |      312552 |            11 |      40520 |          21 |     272032  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |       1456 |           0 |       6736  
 TopTransactionContext    | TopMemoryContext   |     1 |        8192 |             1 |       7744 |           1 |        448  
 PL/pgSQL function        | TopMemoryContext   |     1 |       16384 |             2 |       7176 |           1 |       9208  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |        560 |           0 |       7632  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |        560 |           0 |       7632  
 dynahash                 | TopMemoryContext   |     1 |       24528 |             2 |       2624 |           0 |      21904  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |       2096 |           0 |       6096  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |       1584 |           0 |       6608  
 dynahash                 | TopMemoryContext   |     1 |       24576 |             2 |      10760 |           3 |      13816  
 RowDescriptionContext    | TopMemoryContext   |     1 |        8192 |             1 |       6896 |           0 |       1296  
 MessageContext           | TopMemoryContext   |     1 |       32768 |             3 |      10904 |           1 |      21864  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |        560 |           0 |       7632  
 dynahash                 | TopMemoryContext   |     1 |       32768 |             3 |      16832 |           8 |      15936  
 TransactionAbortContext  | TopMemoryContext   |     1 |       32768 |             1 |      32512 |           0 |        256  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |        560 |           0 |       7632  
 TopPortalContext         | TopMemoryContext   |     1 |        8192 |             1 |       7664 |           0 |        528  
 PortalContext            | TopPortalContext   |     2 |        1024 |             1 |        592 |           0 |        432  
 ExecutorState            | PortalContext      |     3 |       49208 |             4 |      15576 |           3 |      33632  
 printtup                 | ExecutorState      |     4 |        8192 |             1 |       7936 |           0 |        256  
 Table function arguments | ExecutorState      |     4 |        8192 |             1 |       7936 |           0 |        256  
 ExprContext              | ExecutorState      |     4 |        8192 |             1 |       4536 |           0 |       3656  
 dynahash                 | TopMemoryContext   |     1 |       16384 |             2 |       3512 |           2 |      12872  
 CacheMemoryContext       | TopMemoryContext   |     1 |      524288 |             7 |      20960 |          26 |     503328  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        624 |           1 |       1424  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        624 |           1 |       1424  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        624 |           1 |       1424  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        624 |           1 |       1424  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        680 |           1 |       1368  
 EventTriggerCache        | CacheMemoryContext |     2 |        8192 |             1 |       7936 |           2 |        256  
 dynahash                 | EventTriggerCache  |     3 |        8192 |             1 |       2624 |           0 |       5568  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 relation rules           | CacheMemoryContext |     2 |       16384 |             5 |       7352 |           0 |       9032  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        648 |           2 |       1400  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        704 |           3 |       1344  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         16 |           0 |       1008  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        680 |           2 |       1368  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        592 |           3 |       1456  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        680 |           2 |       1368  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        3072 |             2 |       1136 |           2 |       1936  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        704 |           3 |       1344  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         16 |           0 |       1008  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        3072 |             2 |       1136 |           2 |       1936  
 index info               | CacheMemoryContext |     2 |        3072 |             2 |       1136 |           2 |       1936  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        704 |           3 |       1344  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        680 |           2 |       1368  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        728 |           1 |       1320  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        624 |           2 |       1424  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        672 |           3 |       1376  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        704 |           3 |       1344  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        680 |           2 |       1368  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        624 |           2 |       1424  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         16 |           0 |       1008  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        680 |           2 |       1368  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        3072 |             2 |       1136 |           2 |       1936  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        728 |           1 |       1320  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        728 |           1 |       1320  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        680 |           2 |       1368  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        728 |           1 |       1320  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         48 |           0 |        976  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        704 |           3 |       1344  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         16 |           0 |       1008  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        760 |           2 |       1288  
 index info               | CacheMemoryContext |     2 |        1024 |             1 |         16 |           0 |       1008  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        728 |           1 |       1320  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 index info               | CacheMemoryContext |     2 |        2048 |             2 |        952 |           1 |       1096  
 WAL record construction  | TopMemoryContext   |     1 |       49768 |             2 |       6368 |           0 |      43400  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |       2624 |           0 |       5568  
 MdSmgr                   | TopMemoryContext   |     1 |        8192 |             1 |       7432 |           0 |        760  
 dynahash                 | TopMemoryContext   |     1 |        8192 |             1 |        560 |           0 |       7632  
 dynahash                 | TopMemoryContext   |     1 |      104120 |             2 |       2624 |           0 |     101496  
 ErrorContext             | TopMemoryContext   |     1 |        8192 |             1 |       7936 |           0 |        256  
(121 rows)  

2、文本压缩

  
postgres=# select pglz_compress(repeat(md5(random()::text),1024));                                                                                                                                                      pglz_compress                                                                                                        
-------------------------------------------------------------------------------------------------------------------------------------  
 \x00800040006135663437636166003162353066626137006637356161363639003532316233336139ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f  
20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f2  
0ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f  
20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ffff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20ff0f20e7  
(1 row)  

3、解压缩

pglz_decompress(bytea)  
  
postgres=# select pglz_compress(repeat(md5(random()::text),8));  
                                       pglz_compress                                          
--------------------------------------------------------------------------------------------  
 \x00010040003236666162656631003262323262636230006462656638616330003536343161383937010f20ce  
(1 row)  
  
postgres=# select pglz_decompress('\x00010040003236666162656631003262323262636230006462656638616330003536343161383937010f20ce');  
             pglz_decompress      
------------------------------------------------------------------------------  
 26fabef12b22bcb0dbef8ac05641a89726fabef12b22bcb0dbef8ac05641a89726fabef12b22bcb0dbef8ac05641a89726fabef12b22bcb0dbef8ac05641a89726fabef12b22bcb0dbef8ac05641a89726fabef12b22bcb0dbef8ac05641a89726fabef12b22bcb0dbef8ac05641a89726fabef12b22  
bcb0dbef8ac05641a897  
(1 row)  

4、查看绑定变量QUERY的PLAN COSE。

postgres=# prepare a (name) as select * from pg_class where relname=$1;
PREPARE

postgres=# select * from pg_cached_plan_source('a');
 generic_cost | total_custom_cost | num_custom_plans | force_generic | force_custom 
--------------+-------------------+------------------+---------------+--------------
           -1 |                 0 |                0 | f             | f
(1 row)

postgres=# execute a('abc');
 relname | relnamespace | reltype | reloftype | relowner | relam | relfilenode | reltablespace | relpages | reltuples | relallvisible | reltoastrelid | relhasindex | relisshared | relpersistence | relkind | relnatts | relchecks | relhaso
ids | relhasrules | relhastriggers | relhassubclass | relrowsecurity | relforcerowsecurity | relispopulated | relreplident | relispartition | relrewrite | relfrozenxid | relminmxid | relacl | reloptions | relpartbound 
---------+--------------+---------+-----------+----------+-------+-------------+---------------+----------+-----------+---------------+---------------+-------------+-------------+----------------+---------+----------+-----------+--------
----+-------------+----------------+----------------+----------------+---------------------+----------------+--------------+----------------+------------+--------------+------------+--------+------------+--------------
(0 rows)

postgres=# select * from pg_cached_plan_source('a');
 generic_cost | total_custom_cost | num_custom_plans | force_generic | force_custom 
--------------+-------------------+------------------+---------------+--------------
           -1 |  4.61208554676785 |                1 | f             | f
(1 row)

postgres=# execute a('abc');
 relname | relnamespace | reltype | reloftype | relowner | relam | relfilenode | reltablespace | relpages | reltuples | relallvisible | reltoastrelid | relhasindex | relisshared | relpersistence | relkind | relnatts | relchecks | relhaso
ids | relhasrules | relhastriggers | relhassubclass | relrowsecurity | relforcerowsecurity | relispopulated | relreplident | relispartition | relrewrite | relfrozenxid | relminmxid | relacl | reloptions | relpartbound 
---------+--------------+---------+-----------+----------+-------+-------------+---------------+----------+-----------+---------------+---------------+-------------+-------------+----------------+---------+----------+-----------+--------
----+-------------+----------------+----------------+----------------+---------------------+----------------+--------------+----------------+------------+--------------+------------+--------+------------+--------------
(0 rows)

postgres=# select * from pg_cached_plan_source('a');
 generic_cost | total_custom_cost | num_custom_plans | force_generic | force_custom 
--------------+-------------------+------------------+---------------+--------------
           -1 |  9.22417109353571 |                2 | f             | f
(1 row)


参考

https://github.com/MasaoFujii/pg_cheat_funcs

https://github.com/postgrespro/memstat

相关实践学习
使用PolarDB和ECS搭建门户网站
本场景主要介绍基于PolarDB和ECS实现搭建门户网站。
阿里云数据库产品家族及特性
阿里云智能数据库产品团队一直致力于不断健全产品体系,提升产品性能,打磨产品功能,从而帮助客户实现更加极致的弹性能力、具备更强的扩展能力、并利用云设施进一步降低企业成本。以云原生+分布式为核心技术抓手,打造以自研的在线事务型(OLTP)数据库Polar DB和在线分析型(OLAP)数据库Analytic DB为代表的新一代企业级云原生数据库产品体系, 结合NoSQL数据库、数据库生态工具、云原生智能化数据库管控平台,为阿里巴巴经济体以及各个行业的企业客户和开发者提供从公共云到混合云再到私有云的完整解决方案,提供基于云基础设施进行数据从处理、到存储、再到计算与分析的一体化解决方案。本节课带你了解阿里云数据库产品家族及特性。
目录
相关文章
|
2月前
|
关系型数据库 Serverless 定位技术
PostgreSQL GIS函数判断两条线有交点的函数是什么?
PostgreSQL GIS函数判断两条线有交点的函数是什么?
249 60
|
3月前
|
SQL 自然语言处理 关系型数据库
在 PostgreSQL 中使用 `REPLACE` 函数
【8月更文挑战第8天】
888 9
在 PostgreSQL 中使用 `REPLACE` 函数
|
2月前
|
SQL 关系型数据库 C语言
PostgreSQL SQL扩展 ---- C语言函数(三)
可以用C(或者与C兼容,比如C++)语言编写用户自定义函数(User-defined functions)。这些函数被编译到动态可加载目标文件(也称为共享库)中并被守护进程加载到服务中。“C语言函数”与“内部函数”的区别就在于动态加载这个特性,二者的实际编码约定本质上是相同的(因此,标准的内部函数库为用户自定义C语言函数提供了丰富的示例代码)
|
3月前
|
关系型数据库 PostgreSQL
PostgreSQL的null值函数
【8月更文挑战第20天】PostgreSQL的null值函数
93 3
|
3月前
|
SQL 关系型数据库 MySQL
SQL Server、MySQL、PostgreSQL:主流数据库SQL语法异同比较——深入探讨数据类型、分页查询、表创建与数据插入、函数和索引等关键语法差异,为跨数据库开发提供实用指导
【8月更文挑战第31天】SQL Server、MySQL和PostgreSQL是当今最流行的关系型数据库管理系统,均使用SQL作为查询语言,但在语法和功能实现上存在差异。本文将比较它们在数据类型、分页查询、创建和插入数据以及函数和索引等方面的异同,帮助开发者更好地理解和使用这些数据库。尽管它们共用SQL语言,但每个系统都有独特的语法规则,了解这些差异有助于提升开发效率和项目成功率。
386 0
|
4月前
|
SQL Oracle 关系型数据库
|
4月前
|
关系型数据库 BI 数据处理
|
5月前
|
JSON 关系型数据库 数据库
PostgreSQL中json_to_record函数的神秘面纱
`json_to_record`是PostgreSQL中的函数,用于将JSON数据转换为RECORD类型,便于查询和分析。基本用法是传入JSON数据,如`SELECT json_to_record('{"name": "张三", "age": 30}'::json);`。还可结合FUNCTION创建自定义函数,实现复杂功能。在实际应用中,它简化了对JSON格式数据的处理,例如筛选年龄大于30的用户。了解并善用此函数能提升数据库操作效率。本文由木头左分享,期待你的点赞和收藏,下次见!
PostgreSQL中json_to_record函数的神秘面纱
|
5月前
|
SQL 关系型数据库 数据库
PostgreSQL 常用函数分享
PostgreSQL 常用函数分享
48 0
|
6月前
|
关系型数据库 PostgreSQL
postgresql日程排程函数的编写实例
postgresql日程排程函数的编写实例

相关产品

  • 云原生数据库 PolarDB
  • 云数据库 RDS PostgreSQL 版
  • 下一篇
    无影云桌面