postgres_up_个人页

个人头像照片 postgres_up
0
11
0

个人介绍

一枚PGer

擅长的技术

获得更多能力
通用技术能力:

暂时未有相关通用技术能力~

云产品技术能力:

暂时未有相关云产品技术能力~

阿里云技能认证

详细说明
  • 提交了问题 2016-01-24

    greenplum使用gpfdist导入数据性能问题

  • 提交了问题 2016-01-24

    关于GPfdist数据入库性能问题

  • 提交了问题 2016-01-08

    (急)跌跌撞撞前行:greenplum FATAL: DTM initialization: failure during startup recovery, retry failed, check segment status (cdbtm.c:1603)

  • 提交了问题 2016-01-08

    greenplum安装gpinitsystem中:File "/gpadmin/greenplum/bin/gpstop", line 9, in <module> from gppylib.mainUtils import *

  • 提交了问题 2016-01-07

    greenplum数据节点segment个数问题

  • 提交了问题 2016-01-05

    Greeenplum安装gpinitsystem时报Failed to complete obtain psql count Master gp_segment_configuration

暂无更多信息
正在加载, 请稍后...
暂无更多信息
  • 提交了问题 2016-01-24

    greenplum使用gpfdist导入数据性能问题

  • 提交了问题 2016-01-24

    关于GPfdist数据入库性能问题

  • 回答了问题 2019-07-17

    请教GP自带入库程序gpload问题

    请教下,你们600G的数据入库在不出错的情况下,大概要入多久,多少节点的集群?配置如何
    踩0 评论0
  • 回答了问题 2019-07-17

    (急)跌跌撞撞前行:greenplum FATAL: DTM initialization: failure during startup recovery, retry failed, check segment status (cdbtm.c:1603)

    找到问题的答案了,原来shared_buffers确实是不需要设置那么大的,摘一段放这,想看详细的请点连接。https://support.pivotal.io/hc/communities/public/questions/205239098-Unable-to-set-shared-buffers-to-2-GBMichael,While pg_tune is a good utility for a standalone database - it does not apply to the MPP setup used by Greenplum which needs overhead to operate multiple standalone databases communicating on a private network.You cannot set the shared buffers to a higher value, because it is set as an integer -- which caps out at 2GB, and that would be overflow.For tuning, you really want to focus on the gp_vmem_protect_limt, statement_memory and resource queues.And while you want to squeeze every bit of performance out of your cluster - you need to leave room for overhead and things that aren't calculated in the memory usage - simply because greenplum or the OS has to do them -- like unpacking the plan.An OS setting of vm.overcommit_memory = 2 with the default vm.overcommit_ratio being .5 limits the amount of available OS Virtual Memory to: ( swap + ( vm.overcommit_ratio * RAM ) )In your case, this would be a segment max of (Swap + (64 * .5))Your gp_vmem_protect_limit should be set to: ( OS Virtual Memory *.9 / Number of Primary Segments Per Server )This leaves 10% overhead, but does not account for a saturated system with failover... so you could set it for less for stability.For example, if you have mirroring enabled (recommended) and you have failover, one or more nodes would now be carrying one or more additional primary segments -- so if you were tuned for 4 primaries per node and failed over under heavy load, you increase the possibility of the segments running out of memory.Your statment_mem should be set to:( gp_vmem_protect_limt / number of concurrent queries allowed by the resource queues)I would recommend leaving the default setting for any non-custom queue, which leaves you room to bump up the other for queues with massive queries -- though you can simply set the presql at runtime as necessary: set statement_mem=xxx; select....These settings safeguard your system from virtual memory crashes -- if they are setup correctly, the resource queue will prevent queries that would exceed your gp_vmem limit. Your gp_vmem limit should protect your OS from running out of virtual memory. Your OS virtual memory settings should prevent your OS from running out of memory...Correctly tuned, they allow your queries to run with optimal performance with overhead to manage all the moving components.GPDB will always try to use 100% of it's resources, unless otherwise capped by maximums on the resource queues.To get a better understanding of this, take a look at this short video, which does a very good job of explaining how they work effectively:https://www.youtube.com/watch?v=1b0mHsT_woUHope this helps --
    踩0 评论0
  • 提交了问题 2016-01-08

    (急)跌跌撞撞前行:greenplum FATAL: DTM initialization: failure during startup recovery, retry failed, check segment status (cdbtm.c:1603)

  • 提交了问题 2016-01-08

    greenplum安装gpinitsystem中:File "/gpadmin/greenplum/bin/gpstop", line 9, in <module> from gppylib.mainUtils import *

  • 回答了问题 2019-07-17

    greenplum安装gpinitsystem中:File "/gpadmin/greenplum/bin/gpstop", line 9, in <module> from gppylib.mainUtils import *

    在虚拟机的环境下安装没问题了,集群运行也正常,但是现在在物理机上安装就出了这么个问题。操作系统都是centos6.6 问题解决了,还是python得包装得不全。按下面的包装全就没有问题了。$ wget https://bootstrap.pypa.io/get-pip.py$ sudo python get-pip.py $ sudo pip install psi lockfile paramiko setuptools epydoc
    踩0 评论0
  • 回答了问题 2019-07-17

    greenplum数据节点segment个数问题

    ok。谢谢德哥,混子
    踩0 评论0
  • 提交了问题 2016-01-07

    greenplum数据节点segment个数问题

  • 提交了问题 2016-01-05

    Greeenplum安装gpinitsystem时报Failed to complete obtain psql count Master gp_segment_configuration

  • 回答了问题 2019-07-17

    Greeenplum安装gpinitsystem时报Failed to complete obtain psql count Master gp_segment_configuration

    @digoal 德哥,我是该参考姚总的建议,清空目录,重新编译安装喽o(╯□╰)o
    踩0 评论0
正在加载, 请稍后...
滑动查看更多
正在加载, 请稍后...
暂无更多信息