PostgreSQL在何处处理 sql查询之三十七

本文涉及的产品
云原生数据库 PolarDB PostgreSQL 版,标准版 2核4GB 50GB
云原生数据库 PolarDB MySQL 版,通用型 2核8GB 50GB
简介:

看:

复制代码
TupleTableSlot *
ExecProcNode(PlanState *node)
{
    TupleTableSlot *result;

    CHECK_FOR_INTERRUPTS();

    if (node->chgParam != NULL) /* something changed */
        ExecReScan(node);        /* let ReScan handle this */

    if (node->instrument)
        InstrStartNode(node->instrument);

    switch (nodeTag(node))
    {
            /*
             * control nodes
             */
        case T_ResultState:
            fprintf(stderr,"T_ResultState\n");
            result = ExecResult((ResultState *) node);
            break;

        case T_ModifyTableState:
            fprintf(stderr,"T_ModifyTableState\n");
            result = ExecModifyTable((ModifyTableState *) node);
            break;

        case T_AppendState:
            fprintf(stderr,"T_AppendState\n");
            result = ExecAppend((AppendState *) node);
            break;

        case T_MergeAppendState:
            fprintf(stderr,"T_MergeAppendState\n");
            result = ExecMergeAppend((MergeAppendState *) node);
            break;

        case T_RecursiveUnionState:
            fprintf(stderr,"T_RecursiveUnionState\n");
            result = ExecRecursiveUnion((RecursiveUnionState *) node);
            break;

            /* BitmapAndState does not yield tuples */

            /* BitmapOrState does not yield tuples */

            /*
             * scan nodes
             */
        case T_SeqScanState:
            fprintf(stderr,"T_SeqScanState\n");
            result = ExecSeqScan((SeqScanState *) node);
            break;

        case T_IndexScanState:
            fprintf(stderr,"T_IndexScanState\n");
            result = ExecIndexScan((IndexScanState *) node);
            break;

        case T_IndexOnlyScanState:
            fprintf(stderr,"T_IndexOnlyScanState\n");
            result = ExecIndexOnlyScan((IndexOnlyScanState *) node);
            break;

            /* BitmapIndexScanState does not yield tuples */

        case T_BitmapHeapScanState:
            fprintf(stderr,"T_BitmapHeapScanState\n");
            result = ExecBitmapHeapScan((BitmapHeapScanState *) node);
            break;

        case T_TidScanState:
            fprintf(stderr,"T_TidScanState\n");
            result = ExecTidScan((TidScanState *) node);
            break;

        case T_SubqueryScanState:
            fprintf(stderr,"T_SubqueryScanState\n");
            result = ExecSubqueryScan((SubqueryScanState *) node);
            break;

        case T_FunctionScanState:
            fprintf(stderr,"T_FunctionScanState\n");
            result = ExecFunctionScan((FunctionScanState *) node);
            break;

        case T_ValuesScanState:
            fprintf(stderr,"T_ValuesScanState\n");
            result = ExecValuesScan((ValuesScanState *) node);
            break;

        case T_CteScanState:
            fprintf(stderr,"T_CteScanState\n");
            result = ExecCteScan((CteScanState *) node);
            break;

        case T_WorkTableScanState:
            fprintf(stderr,"T_WorkTableScanState\n");
            result = ExecWorkTableScan((WorkTableScanState *) node);
            break;

        case T_ForeignScanState:
            fprintf(stderr,"T_ForeignScanState\n");
            result = ExecForeignScan((ForeignScanState *) node);
            break;

            /*
             * join nodes
             */
        case T_NestLoopState:
            fprintf(stderr,"T_NestLoopState\n");
            result = ExecNestLoop((NestLoopState *) node);
            break;

        case T_MergeJoinState:
            fprintf(stderr,"T_MergeJoinState\n");
            result = ExecMergeJoin((MergeJoinState *) node);
            break;

        case T_HashJoinState:
            fprintf(stderr,"T_HashJoinState\n");
            result = ExecHashJoin((HashJoinState *) node);
            break;

            /*
             * materialization nodes
             */
        case T_MaterialState:
            fprintf(stderr,"T_MaterialState\n");
            result = ExecMaterial((MaterialState *) node);
            break;

        case T_SortState:
            fprintf(stderr,"T_SortState\n");
            result = ExecSort((SortState *) node);
            break;

        case T_GroupState:
            fprintf(stderr,"T_GroupState\n");
            result = ExecGroup((GroupState *) node);
            break;

        case T_AggState:
            fprintf(stderr,"T_AggState\n");
            result = ExecAgg((AggState *) node);
            break;

        case T_WindowAggState:
            fprintf(stderr,"T_WindowAggState\n");
            result = ExecWindowAgg((WindowAggState *) node);
            break;

        case T_UniqueState:
            fprintf(stderr,"T_UniqueState\n");
            result = ExecUnique((UniqueState *) node);
            break;

        case T_HashState:
            fprintf(stderr,"T_HashState\n");
            result = ExecHash((HashState *) node);
            break;

        case T_SetOpState:
            fprintf(stderr,"T_SetOpState\n");
            result = ExecSetOp((SetOpState *) node);
            break;

        case T_LockRowsState:
            fprintf(stderr,"T_LockRowsState\n");
            result = ExecLockRows((LockRowsState *) node);
            break;

        case T_LimitState:
            fprintf(stderr,"T_LimitState\n");
            result = ExecLimit((LimitState *) node);
            break;

        default:
            elog(ERROR, "unrecognized node type: %d", (int) nodeTag(node));
            result = NULL;
            break;
    }

    if (node->instrument)
        InstrStopNode(node->instrument, TupIsNull(result) ? 0.0 : 1.0);

    return result;
}
复制代码

实际执行发现:我的select id from tst04 where id<3 ,执行了三次 T_SeqScanState。

将上述代码缩略:

复制代码
TupleTableSlot *
ExecProcNode(PlanState *node)
{
    TupleTableSlot *result;

    CHECK_FOR_INTERRUPTS();

    if (node->chgParam != NULL) /* something changed */
        ExecReScan(node);        /* let ReScan handle this */

    if (node->instrument)
        InstrStartNode(node->instrument);

    switch (nodeTag(node))
    {
       ...
            /*
             * scan nodes
             */
        case T_SeqScanState:
            fprintf(stderr,"T_SeqScanState\n");
            result = ExecSeqScan((SeqScanState *) node);
            break;

        ...
    }

    if (node->instrument)
        InstrStopNode(node->instrument, TupIsNull(result) ? 0.0 : 1.0);

    return result;
}
复制代码

 再看 ExecSeaScan 的执行:

复制代码
/* ----------------------------------------------------------------
 *        ExecSeqScan(node)
 *
 *        Scans the relation sequentially and returns the next qualifying
 *        tuple.
 *        We call the ExecScan() routine and pass it the appropriate
 *        access method functions.
 * ----------------------------------------------------------------
 */
TupleTableSlot *
ExecSeqScan(SeqScanState *node)
{
    return ExecScan((ScanState *) node,
                    (ExecScanAccessMtd) SeqNext,
                    (ExecScanRecheckMtd) SeqRecheck);
}
复制代码

这个就是全表扫描中的一步。

复制代码
/* ----------------------------------------------------------------
 *        ExecScan
 *
 *        Scans the relation using the 'access method' indicated and
 *        returns the next qualifying tuple in the direction specified
 *        in the global variable ExecDirection.
 *        The access method returns the next tuple and execScan() is
 *        responsible for checking the tuple returned against the qual-clause.
 *
 *        A 'recheck method' must also be provided that can check an
 *        arbitrary tuple of the relation against any qual conditions
 *        that are implemented internal to the access method.
 *
 *        Conditions:
 *          -- the "cursor" maintained by the AMI is positioned at the tuple
 *             returned previously.
 *
 *        Initial States:
 *          -- the relation indicated is opened for scanning so that the
 *             "cursor" is positioned before the first qualifying tuple.
 * ----------------------------------------------------------------
 */
TupleTableSlot *
ExecScan(ScanState *node,
         ExecScanAccessMtd accessMtd,    /* function returning a tuple */
         ExecScanRecheckMtd recheckMtd)
{
    ExprContext *econtext;
    List       *qual;
    ProjectionInfo *projInfo;
    ExprDoneCond isDone;
    TupleTableSlot *resultSlot;

    /*
     * Fetch data from node
     */
    qual = node->ps.qual;
    projInfo = node->ps.ps_ProjInfo;
    econtext = node->ps.ps_ExprContext;

    /*
     * If we have neither a qual to check nor a projection to do, just skip
     * all the overhead and return the raw scan tuple.
     */
    if (!qual && !projInfo)
    {
        ResetExprContext(econtext);
        return ExecScanFetch(node, accessMtd, recheckMtd);
    }

    /*
     * Check to see if we're still projecting out tuples from a previous scan
     * tuple (because there is a function-returning-set in the projection
     * expressions).  If so, try to project another one.
     */
    if (node->ps.ps_TupFromTlist)
    {
        Assert(projInfo);        /* can't get here if not projecting */
        resultSlot = ExecProject(projInfo, &isDone);
        if (isDone == ExprMultipleResult)
            return resultSlot;
        /* Done with that source tuple... */
        node->ps.ps_TupFromTlist = false;
    }

    /*
     * Reset per-tuple memory context to free any expression evaluation
     * storage allocated in the previous tuple cycle.  Note this can't happen
     * until we're done projecting out tuples from a scan tuple.
     */
    ResetExprContext(econtext);

    /*
     * get a tuple from the access method.    Loop until we obtain a tuple that
     * passes the qualification.
     */
    for (;;)
    {
        TupleTableSlot *slot;

        CHECK_FOR_INTERRUPTS();

        slot = ExecScanFetch(node, accessMtd, recheckMtd);

        /*
         * if the slot returned by the accessMtd contains NULL, then it means
         * there is nothing more to scan so we just return an empty slot,
         * being careful to use the projection result slot so it has correct
         * tupleDesc.
         */
        if (TupIsNull(slot))
        {
            if (projInfo)
                return ExecClearTuple(projInfo->pi_slot);
            else
                return slot;
        }

        /*
         * place the current tuple into the expr context
         */
        econtext->ecxt_scantuple = slot;

        /*
         * check that the current tuple satisfies the qual-clause
         *
         * check for non-nil qual here to avoid a function call to ExecQual()
         * when the qual is nil ... saves only a few cycles, but they add up
         * ...
         */
        if (!qual || ExecQual(qual, econtext, false))
        {
            /*
             * Found a satisfactory scan tuple.
             */
            if (projInfo)
            {
                /*
                 * Form a projection tuple, store it in the result tuple slot
                 * and return it --- unless we find we can project no tuples
                 * from this scan tuple, in which case continue scan.
                 */
                resultSlot = ExecProject(projInfo, &isDone);
                if (isDone != ExprEndResult)
                {
                    node->ps.ps_TupFromTlist = (isDone == ExprMultipleResult);
                    return resultSlot;
                }
            }
            else
            {
                /*
                 * Here, we aren't projecting, so just return scan tuple.
                 */
                return slot;
            }
        }
        else
            InstrCountFiltered1(node, 1);

        /*
         * Tuple fails qual, so free per-tuple memory and try again.
         */
        ResetExprContext(econtext);
    }
}
复制代码

现在抛开细节不谈,谈谈自己的观感:

梳理一下:

对于一个特定的查询而言,一旦执行计划指定了seqscan,那么接下来:

ExecutePlan --> ExecProcNode --> ExecSeqScan-->ExecScan

复制代码
static void
ExecutePlan(EState *estate,
            PlanState *planstate,
            CmdType operation,
            bool sendTuples,
            long numberTuples,
            ScanDirection direction,
            DestReceiver *dest)
{
    ...
    /*
     * Loop until we've processed the proper number of tuples from the plan.
     */
    for (;;)
    {
        ...
        /*
         * Execute the plan and obtain a tuple
         */
        slot = ExecProcNode(planstate);
        ...

        /*
         * if the tuple is null, then we assume there is nothing more to
         * process so we just end the loop...
         */
        if (TupIsNull(slot))
            break;
        ...
/* * check our tuple count.. if we've processed the proper number then * quit, else loop again and process more tuples. Zero numberTuples * means no limit. */ current_tuple_count++; if (numberTuples && numberTuples == current_tuple_count) break; } }
复制代码

ExecProcNode:

复制代码
/* ----------------------------------------------------------------
 *        ExecProcNode
 *
 *        Execute the given node to return a(nother) tuple.
 * ----------------------------------------------------------------
 */
TupleTableSlot *
ExecProcNode(PlanState *node)
{
    ...

    switch (nodeTag(node))
    {
            /*
             * control nodes
             */
        case T_ResultState:
            
            result = ExecResult((ResultState *) node);
            break;
        ...
            /*
             * scan nodes
             */
        case T_SeqScanState:
            //fprintf(stderr,"T_SeqScanState\n");
            result = ExecSeqScan((SeqScanState *) node);
            break;

        case T_IndexScanState:
            fprintf(stderr,"T_IndexScanState\n");
            result = ExecIndexScan((IndexScanState *) node);
            break;
        ...
    }
...
return result; }
复制代码

ExecSeqScan:

复制代码
TupleTableSlot *
ExecSeqScan(SeqScanState *node)
{
    return ExecScan((ScanState *) node,
                    (ExecScanAccessMtd) SeqNext,
                    (ExecScanRecheckMtd) SeqRecheck);
}
复制代码

ExecScan:

复制代码
/* ----------------------------------------------------------------
 *        ExecScan
 *
 *        Scans the relation using the 'access method' indicated and
 *        returns the next qualifying tuple in the direction specified
 *        in the global variable ExecDirection.
 *        The access method returns the next tuple and execScan() is
 *        responsible for checking the tuple returned against the qual-clause.
 *
 *        A 'recheck method' must also be provided that can check an
 *        arbitrary tuple of the relation against any qual conditions
 *        that are implemented internal to the access method.
 *
 *        Conditions:
 *          -- the "cursor" maintained by the AMI is positioned at the tuple
 *             returned previously.
 *
 *        Initial States:
 *          -- the relation indicated is opened for scanning so that the
 *             "cursor" is positioned before the first qualifying tuple.
 * ----------------------------------------------------------------
 */
TupleTableSlot *
ExecScan(ScanState *node,
         ExecScanAccessMtd accessMtd,    /* function returning a tuple */
         ExecScanRecheckMtd recheckMtd)
{
    ...

    /*
     * get a tuple from the access method.    Loop until we obtain a tuple that
     * passes the qualification.
     */
    for (;;)
    {
        ...

        slot = ExecScanFetch(node, accessMtd, recheckMtd);

        ...
    }
}
复制代码

可以看到,全表扫描的时候,是一条条地往下捋,完全没有任何并发的可能。

所以,可以想象,遇到大表,Index立马变得十分重要。如果遇到大表和大表 Hash Join 之类的,那就自求多福吧。

我现在看到的是,加入调试信息后,对于如下的表:

复制代码
postgres=# select * from tst04;
 id | val 
----+-----
  4 | 400
  1 | 100
  3 | 300
  2 | 200
(4 rows)
复制代码

然后这样来查询:

复制代码
postgres=# select * from tst04 where id>1;
 id | val 
----+-----
  4 | 400
  3 | 300
  2 | 200
(3 rows)
复制代码

可以看到执行情况:

复制代码
In Standard_ExecutorRun, count is 0
In ExecutePlan ... for loop by process 7365
In ExecScan... for loop ... by process 7365
In ExecutePlan ... for loop by process 7365
In ExecScan... for loop ... by process 7365
In ExecScan... for loop ... by process 7365
In ExecutePlan ... for loop by process 7365
In ExecScan... for loop ... by process 7365
In ExecutePlan ... for loop by process 7365
In ExecScan... for loop ... by process 7365
复制代码

对为何执行这么多次,还不是很理解。







本文转自健哥的数据花园博客园博客,原文链接:http://www.cnblogs.com/gaojian/archive/2013/05/31/3108539.html,如需转载请自行联系原作者

相关实践学习
使用PolarDB和ECS搭建门户网站
本场景主要介绍基于PolarDB和ECS实现搭建门户网站。
阿里云数据库产品家族及特性
阿里云智能数据库产品团队一直致力于不断健全产品体系,提升产品性能,打磨产品功能,从而帮助客户实现更加极致的弹性能力、具备更强的扩展能力、并利用云设施进一步降低企业成本。以云原生+分布式为核心技术抓手,打造以自研的在线事务型(OLTP)数据库Polar DB和在线分析型(OLAP)数据库Analytic DB为代表的新一代企业级云原生数据库产品体系, 结合NoSQL数据库、数据库生态工具、云原生智能化数据库管控平台,为阿里巴巴经济体以及各个行业的企业客户和开发者提供从公共云到混合云再到私有云的完整解决方案,提供基于云基础设施进行数据从处理、到存储、再到计算与分析的一体化解决方案。本节课带你了解阿里云数据库产品家族及特性。
目录
相关文章
PostgreSQL窗口函数避坑指南:如何让复杂分析查询提速300%?
本文基于真实企业级案例,深入剖析PostgreSQL窗口函数的执行原理与性能陷阱,提供8大优化策略。通过定制索引、分区裁剪、内存调优及并行处理等手段,将分钟级查询压缩至秒级响应。结合CTE分阶段计算与物化视图技术,解决海量数据分析中的瓶颈问题。某金融客户实践表明,风险分析查询从47秒降至0.8秒,效率提升5800%。文章附带代码均在PostgreSQL 15中验证,助您高效优化SQL性能。
第三篇:高级 SQL 查询与多表操作
本文深入讲解高级SQL查询技巧,涵盖多表JOIN操作、聚合函数、分组查询、子查询及视图索引等内容。适合已掌握基础SQL的学习者,通过实例解析INNER/LEFT/RIGHT/FULL JOIN用法,以及COUNT/SUM/AVG等聚合函数的应用。同时探讨复杂WHERE条件、子查询嵌套,并介绍视图简化查询与索引优化性能的方法。最后提供实践建议与学习资源,助你提升SQL技能以应对实际数据处理需求。
161 1
SQL查询太慢?实战讲解YashanDB SQL调优思路
本文是Meetup第十期“调优实战专场”的第二篇技术文章,上一篇《高效查询秘诀,解码YashanDB优化器分组查询优化手段》中,我们揭秘了YashanDB分组查询优化秘诀,本文将通过一个案例,助你快速上手YashanDB慢日志功能,精准定位“慢SQL”后进行优化。
CTE vs 子查询:深入拆解PostgreSQL复杂SQL的隐藏性能差异
本文深入探讨了PostgreSQL中CTE(公共表表达式)与子查询的选择对SQL性能的影响。通过分析两者底层机制,揭示CTE的物化特性及子查询的优化融合优势,并结合多场景案例对比执行效率。最终给出决策指南,帮助开发者根据数据量、引用次数和复杂度选择最优方案,同时提供高级优化技巧和版本演进建议,助力SQL性能调优。
【YashanDB知识库】字段加上索引后,SQL查询不到结果
【YashanDB知识库】字段加上索引后,SQL查询不到结果
凌晨2点报警群炸了:一条sql 执行200秒!搞定之后,我总结了一个慢SQL查询、定位分析解决的完整套路
凌晨2点报警群炸了:一条sql 执行200秒!搞定之后,我总结了一个慢SQL查询、定位分析解决的完整套路
凌晨2点报警群炸了:一条sql 执行200秒!搞定之后,我总结了一个慢SQL查询、定位分析解决的完整套路
OmniSQL:开源文本到SQL神器!自然语言秒转查询到复杂多表连接等SQL需求
OmniSQL是开源的文本到SQL转换模型,通过创新的数据合成框架生成250万条高质量样本,支持7B/14B/32B三种模型版本,能处理从简单查询到复杂多表连接等各种SQL需求。
357 16
OmniSQL:开源文本到SQL神器!自然语言秒转查询到复杂多表连接等SQL需求
玩转大数据:从零开始掌握SQL查询基础
玩转大数据:从零开始掌握SQL查询基础
204 35
如何优化SQL查询以提高数据库性能?
这篇文章以生动的比喻介绍了优化SQL查询的重要性及方法。它首先将未优化的SQL查询比作在自助餐厅贪多嚼不烂的行为,强调了只获取必要数据的必要性。接着,文章详细讲解了四种优化策略:**精简选择**(避免使用`SELECT *`)、**专业筛选**(利用`WHERE`缩小范围)、**高效联接**(索引和限制数据量)以及**使用索引**(加速搜索)。此外,还探讨了如何避免N+1查询问题、使用分页限制结果、理解执行计划以及定期维护数据库健康。通过这些技巧,可以显著提升数据库性能,让查询更高效流畅。
云原生数据仓库AnalyticDB PostgreSQL同一个SQL可以实现向量索引、全文索引GIN、普通索引BTREE混合查询,简化业务实现逻辑、提升查询性能
本文档介绍了如何在AnalyticDB for PostgreSQL中创建表、向量索引及混合检索的实现步骤。主要内容包括:创建`articles`表并设置向量存储格式,创建ANN向量索引,为表增加`username`和`time`列,建立BTREE索引和GIN全文检索索引,并展示了查询结果。参考文档提供了详细的SQL语句和配置说明。
125 2

热门文章

最新文章

推荐镜像

更多
AI助理

你好,我是AI助理

可以解答问题、推荐解决方案等

登录插画

登录以查看您的控制台资源

管理云资源
状态一览
快捷访问