PostgreSQL 语法树分析之前需要了解到知识

本文涉及的产品
云原生数据库 PolarDB MySQL 版,通用型 2核8GB 50GB
云原生数据库 PolarDB PostgreSQL 版,标准版 2核4GB 50GB
简介:

为了少走弯路,有必要了解如下概念:

http://www.postgresql.org/docs/9.2/static/querytree.html

The rule system is located between the parser and the planner. It takes the output of the parser, one query tree, and the user-defined rewrite rules, which are also query trees with some extra information, and creates zero or more query trees as result. So its input and output are always things the parser itself could have produced and thus, anything it sees is basically representable as an SQL statement.

parser -- rule sytem -- planner

难道说,PostgreSQL的 执行计划,是基于规则的?

parser 输出的 是  query tree,加入了 用户定义的 rewrite rule后,planner 输出的是 一个或多个 query tree。

Now what is a query tree? It is an internal representation of an SQL statement where the single parts that it is built from are stored separately. These query trees can be shown in the server log if you set the configuration parameters debug_print_parse, debug_print_rewritten, or debug_print_plan. The rule actions are also stored as query trees, in the system catalog pg_rewrite. They are not formatted like the log output, but they contain exactly the same information.

可以通过  debug_print_parse 、debug_print_rewritten、debug_print_plan 来从 服务器Log 输出 plan。

我如果设置了  debug_print_plan后,如下的SQL文,可以得到如下的log:

"WITH test(x) AS (SELECT 1 UNION SELECT 2)
SELECT *  FROM test;

 

复制代码
[postgres@lex pg_log]$ cat postgresql-2013-06-06_095516.csv
2013-06-06 09:55:16.125 CST,,,2875,,51afec04.b3b,1,,2013-06-06 09:55:16 CST,,0,LOG,00000,"database system was shut down at 2013-06-06 09:47:45 CST",,,,,,,,,""
2013-06-06 09:55:16.187 CST,,,2879,,51afec04.b3f,1,,2013-06-06 09:55:16 CST,,0,LOG,00000,"autovacuum launcher started",,,,,,,,,""
2013-06-06 09:55:16.188 CST,,,2873,,51afec03.b39,1,,2013-06-06 09:55:15 CST,,0,LOG,00000,"database system is ready to accept connections",,,,,,,,,""
2013-06-06 09:56:05.261 CST,"postgres","postgres",2938,"[local]",51afec12.b7a,1,"SELECT",2013-06-06 09:55:30 CST,2/2,0,LOG,00000,"plan:","   {PLANNEDSTMT 
   :commandType 1 
   :queryId 0 
   :hasReturning false 
   :hasModifyingCTE false 
   :canSetTag true 
   :transientPlan false 
   :planTree 
      {CTESCAN 
      :startup_cost 0.06 
      :total_cost 0.10 
      :plan_rows 2 
      :plan_width 4 
      :targetlist (
         {TARGETENTRY 
         :expr 
            {VAR 
            :varno 1 
            :varattno 1 
            :vartype 23 
            :vartypmod -1 
            :varcollid 0 
            :varlevelsup 0 
            :varnoold 1 
            :varoattno 1 
            :location -1
            }
         :resno 1 
         :resname x 
         :ressortgroupref 0 
         :resorigtbl 0 
         :resorigcol 0 
         :resjunk false
         }
      )
      :qual <> 
      :lefttree <> 
      :righttree <> 
      :initPlan (
         {SUBPLAN 
         :subLinkType 6 
         :testexpr <> 
         :paramIds <> 
         :plan_id 1 
         :plan_name CTE\ test 
         :firstColType 23 
         :firstColTypmod -1 
         :firstColCollation 0 
         :useHashTable false 
         :unknownEqFalse false 
         :setParam (i 0)
         :parParam <> 
         :args <> 
         :startup_cost 0.00 
         :per_call_cost 0.06
         }
      )
      :extParam (b)
      :allParam (b 0)
      :scanrelid 1 
      :ctePlanId 1 
      :cteParam 0
      }
   :rtable (
      {RTE 
      :alias <> 
      :eref 
         {ALIAS 
         :aliasname test 
         :colnames (""x"")
         }
      :rtekind 5 
      :ctename test 
      :ctelevelsup 0 
      :self_reference false 
      :ctecoltypes <> 
      :ctecoltypmods <> 
      :ctecolcollations <> 
      :inh false 
      :inFromCl true 
      :requiredPerms 2 
      :checkAsUser 0 
      :selectedCols (b)
      :modifiedCols (b)
      }
      {RTE 
      :alias 
         {ALIAS 
         :aliasname *SELECT*\ 1 
         :colnames <>
         }
      :eref 
         {ALIAS 
         :aliasname *SELECT*\ 1 
         :colnames (""?column?"")
         }
      :rtekind 1 
      :subquery <> 
      :security_barrier false 
      :inh false 
      :inFromCl false 
      :requiredPerms 0 
      :checkAsUser 0 
      :selectedCols (b)
      :modifiedCols (b)
      }
      {RTE 
      :alias 
         {ALIAS 
         :aliasname *SELECT*\ 2 
         :colnames <>
         }
      :eref 
         {ALIAS 
         :aliasname *SELECT*\ 2 
         :colnames (""?column?"")
         }
      :rtekind 1 
      :subquery <> 
      :security_barrier false 
      :inh false 
      :inFromCl false 
      :requiredPerms 0 
      :checkAsUser 0 
      :selectedCols (b)
      :modifiedCols (b)
      }
   )
   :resultRelations <> 
   :utilityStmt <> 
   :subplans (
      {UNIQUE 
      :startup_cost 0.05 
      :total_cost 0.06 
      :plan_rows 2 
      :plan_width 0 
      :targetlist (
         {TARGETENTRY 
         :expr 
            {VAR 
            :varno 65001 
            :varattno 1 
            :vartype 23 
            :vartypmod -1 
            :varcollid 0 
            :varlevelsup 0 
            :varnoold 1 
            :varoattno 1 
            :location -1
            }
         :resno 1 
         :resname ?column? 
         :ressortgroupref 1 
         :resorigtbl 0 
         :resorigcol 0 
         :resjunk false
         }
      )
      :qual <> 
      :lefttree 
         {SORT 
         :startup_cost 0.05 
         :total_cost 0.06 
         :plan_rows 2 
         :plan_width 0 
         :targetlist (
            {TARGETENTRY 
            :expr 
               {VAR 
               :varno 65001 
               :varattno 1 
               :vartype 23 
               :vartypmod -1 
               :varcollid 0 
               :varlevelsup 0 
               :varnoold 1 
               :varoattno 1 
               :location -1
               }
            :resno 1 
            :resname ?column? 
            :ressortgroupref 1 
            :resorigtbl 0 
            :resorigcol 0 
            :resjunk false
            }
         )
         :qual <> 
         :lefttree 
            {APPEND 
            :startup_cost 0.00 
            :total_cost 0.04 
            :plan_rows 2 
            :plan_width 0 
            :targetlist (
               {TARGETENTRY 
               :expr 
                  {VAR 
                  :varno 65001 
                  :varattno 1 
                  :vartype 23 
                  :vartypmod -1 
                  :varcollid 0 
                  :varlevelsup 0 
                  :varnoold 1 
                  :varoattno 1 
                  :location -1
                  }
               :resno 1 
               :resname ?column? 
               :ressortgroupref 1 
               :resorigtbl 0 
               :resorigcol 0 
               :resjunk false
               }
            )
            :qual <> 
            :lefttree <> 
            :righttree <> 
            :initPlan <> 
            :extParam (b)
            :allParam (b)
            :appendplans (
               {RESULT 
               :startup_cost 0.00 
               :total_cost 0.01 
               :plan_rows 1 
               :plan_width 0 
               :targetlist (
                  {TARGETENTRY 
                  :expr 
                     {CONST 
                     :consttype 23 
                     :consttypmod -1 
                     :constcollid 0 
                     :constlen 4 
                     :constbyval true 
                     :constisnull false 
                     :location 24 
                     :constvalue 4 [ 1 0 0 0 0 0 0 0 ]
                     }
                  :resno 1 
                  :resname ?column? 
                  :ressortgroupref 0 
                  :resorigtbl 0 
                  :resorigcol 0 
                  :resjunk false
                  }
               )
               :qual <> 
               :lefttree <> 
               :righttree <> 
               :initPlan <> 
               :extParam (b)
               :allParam (b)
               :resconstantqual <>
               }
               {RESULT 
               :startup_cost 0.00 
               :total_cost 0.01 
               :plan_rows 1 
               :plan_width 0 
               :targetlist (
                  {TARGETENTRY 
                  :expr 
                     {CONST 
                     :consttype 23 
                     :consttypmod -1 
                     :constcollid 0 
                     :constlen 4 
                     :constbyval true 
                     :constisnull false 
                     :location 39 
                     :constvalue 4 [ 2 0 0 0 0 0 0 0 ]
                     }
                  :resno 1 
                  :resname ?column? 
                  :ressortgroupref 0 
                  :resorigtbl 0 
                  :resorigcol 0 
                  :resjunk false
                  }
               )
               :qual <> 
               :lefttree <> 
               :righttree <> 
               :initPlan <> 
               :extParam (b)
               :allParam (b)
               :resconstantqual <>
               }
            )
            }
         :righttree <> 
         :initPlan <> 
         :extParam (b)
         :allParam (b)
         :numCols 1 
         :sortColIdx 1 
         :sortOperators 97 
         :collations 0 
         :nullsFirst false
         }
      :righttree <> 
      :initPlan <> 
      :extParam (b)
      :allParam (b)
      :numCols 1 
      :uniqColIdx 1 
      :uniqOperators 96
      }
   )
   :rewindPlanIDs (b)
   :rowMarks <> 
   :relationOids <> 
   :invalItems <> 
   :nParamExec 1
   }
",,,,,"WITH test(x) AS (SELECT 1 UNION SELECT 2)
SELECT *  FROM test;",,,"psql"
[postgres@lex pg_log]$ 
复制代码

重要的是,要了解以下几个概念:

When reading the SQL representations of the query trees in this chapter it is necessary to be able to identify the parts the statement is broken into when it is in the query tree structure. The parts of a query tree are

 command type: 命令的类型

the command type
This is a simple value telling which command (SELECT, INSERT, UPDATE, DELETE) produced the query tree.

range table: 范围表 ,在SQL 的 select 而言,就是 from 后面跟的表名称的列表

复制代码
the range table
The range table is a list of relations that are used in the query. In a SELECT statement these are the relations given after the FROM key word.

Every range table entry identifies a table or view and tells by which name it is called in the other parts of the query. In the query tree, the range table entries are referenced by number rather than by name, so here it doesn't matter if there are duplicate names as it would in an SQL statement. This can happen after the range tables of rules have been merged in. The examples in this chapter will not have this situation.
复制代码

the result relation:结果表,对 SQL的 select 而言,可以无视

复制代码
the result relation
This is an index into the range table that identifies the relation where the results of the query go.

SELECT queries don't have a result relation. (The special case of SELECT INTO is mostly identical to CREATE TABLE followed by INSERT ... SELECT, and is not discussed separately here.)

For INSERT, UPDATE, and DELETE commands, the result relation is the table (or view!) where the changes are to take effect.
复制代码

target list: 目标列表,对SQL的 select 而言,就是  select 和 from 之间的东西

复制代码
the target list
The target list is a list of expressions that define the result of the query. In the case of a SELECT, these expressions are the ones that build the final output of the query. They correspond to the expressions between the key words SELECT and FROM. (* is just an abbreviation for all the column names of a relation. It is expanded by the parser into the individual columns, so the rule system never sees it.)

DELETE commands don't need a normal target list because they don't produce any result. Instead, the rule system adds a special CTID entry to the empty target list, to allow the executor to find the row to be deleted. (CTID is added when the result relation is an ordinary table. If it is a view, a whole-row variable is added instead, as described in Section 37.2.4.)

For INSERT commands, the target list describes the new rows that should go into the result relation. It consists of the expressions in the VALUES clause or the ones from the SELECT clause in INSERT ... SELECT. The first step of the rewrite process adds target list entries for any columns that were not assigned to by the original command but have defaults. Any remaining columns (with neither a given value nor a default) will be filled in by the planner with a constant null expression.

For UPDATE commands, the target list describes the new rows that should replace the old ones. In the rule system, it contains just the expressions from the SET column = expression part of the command. The planner will handle missing columns by inserting expressions that copy the values from the old row into the new one. Just as for DELETE, the rule system adds a CTID or whole-row variable so that the executor can identify the old row to be updated.

Every entry in the target list contains an expression that can be a constant value, a variable pointing to a column of one of the relations in the range table, a parameter, or an expression tree made of function calls, constants, variables, operators, etc.
复制代码

the qualification: 对 sql 而言,就是 where 条件

the qualification
The query's qualification is an expression much like one of those contained in the target list entries. The result value of this expression is a Boolean that tells whether the operation (INSERT, UPDATE, DELETE, or SELECT) for the final result row should be executed or not. It corresponds to the WHERE clause of an SQL statement.

the join tree: 对简单查询而言,和 rangetable 类似。对复杂的join 而言,就需要 join tree了。

复制代码
the join tree
The query's join tree shows the structure of the FROM clause. For a simple query like SELECT ... FROM a, b, c, the join tree is just a list of the FROM items, because we are allowed to join them in any order. But when JOIN expressions, particularly outer joins, are used, we have to join in the order shown by the joins. In that case, the join tree shows the structure of the JOIN expressions. The restrictions associated with particular JOIN clauses (from ON or USING expressions) are stored as qualification expressions attached to those join-tree nodes. It turns out to be convenient to store the top-level WHERE expression as a qualification attached to the top-level join-tree item, too. So really the join tree represents both the FROM and WHERE clauses of a SELECT.
复制代码

 再来看看比较简单的SQL 查询得到的 plan tree:

复制代码
2013-06-06 10:45:17.784 CST,,,2896,,51aff7bd.b50,1,,2013-06-06 10:45:17 CST,,0,LOG,00000,"database system is ready to accept connections",,,,,,,,,""
2013-06-06 10:45:48.335 CST,"postgres","postgres",2961,"[local]",51aff7d1.b91,1,"SELECT",2013-06-06 10:45:37 CST,2/2,0,LOG,00000,"plan:","   {PLANNEDSTMT 
   :commandType 1 
   :queryId 0 
   :hasReturning false 
   :hasModifyingCTE false 
   :canSetTag true 
   :transientPlan false 
   :planTree 
      {SEQSCAN 
      :startup_cost 0.00 
      :total_cost 36.75 
      :plan_rows 713 
      :plan_width 4 
      :targetlist (
         {TARGETENTRY 
         :expr 
            {VAR 
            :varno 1 
            :varattno 1 
            :vartype 23 
            :vartypmod -1 
            :varcollid 0 
            :varlevelsup 0 
            :varnoold 1 
            :varoattno 1 
            :location 7
            }
         :resno 1 
         :resname id 
         :ressortgroupref 0 
         :resorigtbl 16393 
         :resorigcol 1 
         :resjunk false
         }
      )
      :qual (
         {OPEXPR 
         :opno 521 
         :opfuncid 147 
         :opresulttype 16 
         :opretset false 
         :opcollid 0 
         :inputcollid 0 
         :args (
            {VAR 
            :varno 1 
            :varattno 1 
            :vartype 23 
            :vartypmod -1 
            :varcollid 0 
            :varlevelsup 0 
            :varnoold 1 
            :varoattno 1 
            :location 27
            }
            {CONST 
            :consttype 23 
            :consttypmod -1 
            :constcollid 0 
            :constlen 4 
            :constbyval true 
            :constisnull false 
            :location 30 
            :constvalue 4 [ 3 0 0 0 0 0 0 0 ]
            }
         )
         :location 29
         }
      )
      :lefttree <> 
      :righttree <> 
      :initPlan <> 
      :extParam (b)
      :allParam (b)
      :scanrelid 1
      }
   :rtable (
      {RTE 
      :alias <> 
      :eref 
         {ALIAS 
         :aliasname tst04 
         :colnames (""id"" ""val"")
         }
      :rtekind 0 
      :relid 16393 
      :relkind r 
      :inh false 
      :inFromCl true 
      :requiredPerms 2 
      :checkAsUser 0 
      :selectedCols (b 9)
      :modifiedCols (b)
      }
   )
   :resultRelations <> 
   :utilityStmt <> 
   :subplans <> 
   :rewindPlanIDs (b)
   :rowMarks <> 
   :relationOids (o 16393)
   :invalItems <> 
   :nParamExec 0
   }
",,,,,"select id from tst04 where id>3;",,,"psql"
[postgres@lex pg_log]$ 
复制代码
相关实践学习
使用PolarDB和ECS搭建门户网站
本场景主要介绍如何基于PolarDB和ECS实现搭建门户网站。
阿里云数据库产品家族及特性
阿里云智能数据库产品团队一直致力于不断健全产品体系,提升产品性能,打磨产品功能,从而帮助客户实现更加极致的弹性能力、具备更强的扩展能力、并利用云设施进一步降低企业成本。以云原生+分布式为核心技术抓手,打造以自研的在线事务型(OLTP)数据库Polar DB和在线分析型(OLAP)数据库Analytic DB为代表的新一代企业级云原生数据库产品体系, 结合NoSQL数据库、数据库生态工具、云原生智能化数据库管控平台,为阿里巴巴经济体以及各个行业的企业客户和开发者提供从公共云到混合云再到私有云的完整解决方案,提供基于云基础设施进行数据从处理、到存储、再到计算与分析的一体化解决方案。本节课带你了解阿里云数据库产品家族及特性。
目录
相关文章
|
SQL Oracle 关系型数据库
实时计算 Flink版操作报错之往GREENPLUM 6 写数据,用postgresql-42.2.9.jar 报 ON CONFLICT (uuid) DO UPDATE SET 语法有问题。怎么解决
在使用实时计算Flink版过程中,可能会遇到各种错误,了解这些错误的原因及解决方法对于高效排错至关重要。针对具体问题,查看Flink的日志是关键,它们通常会提供更详细的错误信息和堆栈跟踪,有助于定位问题。此外,Flink社区文档和官方论坛也是寻求帮助的好去处。以下是一些常见的操作报错及其可能的原因与解决策略。
|
关系型数据库 物联网 PostgreSQL
沉浸式学习PostgreSQL|PolarDB 11: 物联网(IoT)、监控系统、应用日志、用户行为记录等场景 - 时序数据高吞吐存取分析
物联网场景, 通常有大量的传感器(例如水质监控、气象监测、新能源汽车上的大量传感器)不断探测最新数据并上报到数据库. 监控系统, 通常也会有采集程序不断的读取被监控指标(例如CPU、网络数据包转发、磁盘的IOPS和BW占用情况、内存的使用率等等), 同时将监控数据上报到数据库. 应用日志、用户行为日志, 也就有同样的特征, 不断产生并上报到数据库. 以上数据具有时序特征, 对数据库的关键能力要求如下: 数据高速写入 高速按时间区间读取和分析, 目的是发现异常, 分析规律. 尽量节省存储空间
1009 1
|
4月前
|
SQL 存储 关系型数据库
PostgreSQL窗口函数避坑指南:如何让复杂分析查询提速300%?
本文基于真实企业级案例,深入剖析PostgreSQL窗口函数的执行原理与性能陷阱,提供8大优化策略。通过定制索引、分区裁剪、内存调优及并行处理等手段,将分钟级查询压缩至秒级响应。结合CTE分阶段计算与物化视图技术,解决海量数据分析中的瓶颈问题。某金融客户实践表明,风险分析查询从47秒降至0.8秒,效率提升5800%。文章附带代码均在PostgreSQL 15中验证,助您高效优化SQL性能。
198 0
|
Oracle NoSQL 关系型数据库
主流数据库对比:MySQL、PostgreSQL、Oracle和Redis的优缺点分析
主流数据库对比:MySQL、PostgreSQL、Oracle和Redis的优缺点分析
2184 3
|
关系型数据库 定位技术 分布式数据库
沉浸式学习PostgreSQL|PolarDB 18: 通过GIS轨迹相似伴随|时态分析|轨迹驻点识别等技术对拐卖、诱骗场景进行侦查
本文主要教大家怎么用好数据库, 而不是怎么运维管理数据库、怎么开发数据库内核.
1525 1
|
SQL 关系型数据库 MySQL
SQL Server、MySQL、PostgreSQL:主流数据库SQL语法异同比较——深入探讨数据类型、分页查询、表创建与数据插入、函数和索引等关键语法差异,为跨数据库开发提供实用指导
【8月更文挑战第31天】SQL Server、MySQL和PostgreSQL是当今最流行的关系型数据库管理系统,均使用SQL作为查询语言,但在语法和功能实现上存在差异。本文将比较它们在数据类型、分页查询、创建和插入数据以及函数和索引等方面的异同,帮助开发者更好地理解和使用这些数据库。尽管它们共用SQL语言,但每个系统都有独特的语法规则,了解这些差异有助于提升开发效率和项目成功率。
1403 0
|
关系型数据库 数据库 PostgreSQL
PostgreSQL数据库的字符串拼接语法使用说明
【6月更文挑战第11天】PostgreSQL数据库的字符串拼接语法使用说明
1307 1
|
存储 关系型数据库 MySQL
TiDB与MySQL、PostgreSQL等数据库的比较分析
【2月更文挑战第25天】本文将对TiDB、MySQL和PostgreSQL等数据库进行详细的比较分析,探讨它们各自的优势和劣势。TiDB作为一款分布式关系型数据库,在扩展性、并发性能等方面表现突出;MySQL以其易用性和成熟性受到广泛应用;PostgreSQL则在数据完整性、扩展性等方面具有优势。通过对比这些数据库的特点和适用场景,帮助企业更好地选择适合自己业务需求的数据库系统。
2175 4
|
关系型数据库 PostgreSQL
postgresql字符串拼接语法
【5月更文挑战第6天】postgresql字符串拼接语法
598 0
|
SQL 关系型数据库 MySQL
PostgreSQL【异常 01】java.io.IOException:Tried to send an out-of-range integer as a 2-byte value 分析+解决
PostgreSQL【异常 01】java.io.IOException:Tried to send an out-of-range integer as a 2-byte value 分析+解决
919 1

热门文章

最新文章

推荐镜像

更多