pidstat 统计进程、线程、进程组的资源消耗

本文涉及的产品
云原生数据库 PolarDB 分布式版,标准版 2核8GB
云数据库 RDS PostgreSQL,高可用版 2核4GB 50GB
云原生数据库 PolarDB MySQL 版,Serverless 5000PCU 100GB
简介:

标签

PostgreSQL , Linux , pidstat


背景

如果我们要查看Linux下面进程、进程组、线程的资源消耗的统计信息,可以使用pidstat,它可以收集并报告进程的统计信息。

pidstat实际上也是将/proc/pid下的统计信息统筹后展现给用户。

pidstat例子

1. 列出IO统计信息:-d

2. 列出内存使用统计,page fault:-r

3. CPU统计信息:-u

4. 上下文切换统计信息:-w

5. 指定输出的维度

指定命令: -C command

指定进程号:-p { pid [,...] | SELF | ALL }

将所有列输出到单行、便于导入单张表中:-h

按每个CPU核的统计:-I

按所有CPU核统计:默认

列出命令的完整内容,包括参数:-l

列出线程统计信息:-t

按进程维度单独统计、按进程全局+子进程单独统计、按进程全局、单独统计同时按子任务单独统计:-T { TASK | CHILD | ALL }

实例,统计5秒的值并输出

#pidstat -d -r -u -w -l -h -p ALL 5 1|less  


#      Time       PID    %usr %system  %guest    %CPU   CPU  minflt/s  majflt/s     VSZ    RSS   %MEM   kB_rd/s   kB_wr/s kB_ccwr/s   cswch/s nvcswch/s  Command  
 1488541252         1    0.00    0.00    0.00    0.00     0      0.00      0.00   19348   1512   0.00      0.00      0.00      0.00      0.00      0.00  /sbin/init   
 1488541252         2    0.00    0.00    0.00    0.00    50      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  kthreadd  
 1488541252         3    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.59      0.00  migration/0  
 1488541252         4    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  ksoftirqd/0  
 1488541252         5    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  migration/0  
 1488541252         6    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  watchdog/0  

 ......  

 1488541264     18523    0.00    0.20    0.00    0.20     5     10.78      0.00 137774940 3002232   0.57      0.00      0.00      0.00      1.96      0.20  /home/digoal/pgsql9.6/bin/postgres   
 1488541264     18549    0.00    0.00    0.00    0.00    61      0.00      0.00  140892   2028   0.00      0.00      0.00      0.00      0.00      0.00  postgres: logger process               
 1488541264     18552    0.00    0.00    0.00    0.00     0      0.00      0.00 137779116 5509520   1.04      0.00      0.00      0.00      0.00      0.00  postgres: checkpointer process         
 1488541264     18554    0.00    0.00    0.00    0.00    20      0.00      0.00 137776328 1058572   0.20      0.00      0.00      0.00      3.92      0.00  postgres: writer process               
 1488541264     18556    0.00    0.00    0.00    0.00    15      0.00      0.00 137774940  19120   0.00      0.00      0.00      0.00      3.92      0.00  postgres: wal writer process           
 1488541264     18557    0.00    0.00    0.00    0.00    21      2.94      0.00 137779092   3404   0.00      0.00      0.00      0.00      3.92      0.00  postgres: autovacuum launcher process     
 1488541264     18559    0.00    0.00    0.00    0.00    11      0.00      0.00  142988   2092   0.00      0.00      0.00      0.00      0.00      0.00  postgres: archiver process   last was 0000000100000030000000B2  
 1488541264     18561    0.00    0.00    0.00    0.00    53      4.90      0.00  143556   2624   0.00      0.00    317.65    317.65      4.12      0.20  postgres: stats collector process      

参考

man pidstat

PIDSTAT(1)                    Linux User’s Manual                   PIDSTAT(1)  

NAME  
       pidstat - Report statistics for Linux tasks.  

SYNOPSIS  
       pidstat [ -C comm ] [ -d ] [ -h ] [ -I ] [ -l ] [ -p { pid [,...] | SELF | ALL } ] [ -r ] [ -t ] [ -T { TASK | CHILD | ALL } ] [ -u ] [ -V ] [ -w ] [ interval [ count ] ]  

DESCRIPTION  
       The  pidstat  command  is  used for monitoring individual tasks currently being managed by the Linux kernel.  It writes to standard output activities for every task selected with option -p or for every task  
       managed by the Linux kernel if option -p ALL has been used. Not selecting any tasks is equivalent to specifying -p ALL but only active tasks (tasks with  non-zero  statistics  values)  will  appear  in  the  
       report.  

       The pidstat command can also be used for monitoring the child processes of selected tasks.  Read about option -T below.  

       The  interval  parameter  specifies  the  amount  of time in seconds between each report.  A value of 0 (or no parameters at all) indicates that tasks statistics are to be reported for the time since system  
       startup (boot).  The count parameter can be specified in conjunction with the interval parameter if this one is not set to zero. The value of count determines the number of  reports  generated  at  interval  
       seconds apart. If the interval parameter is specified without the count parameter, the pidstat command generates reports continuously.  

       You can select information about specific task activities using flags.  Not specifying any flags selects only CPU activity.  

OPTIONS  
       -C comm  
              Display only tasks whose command name includes the string comm.  

       -d     Report I/O statistics (kernels 2.6.20 and later only).  The following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              kB_rd/s  
                     Number of kilobytes the task has caused to be read from disk per second.  

              kB_wr/s  
                     Number of kilobytes the task has caused, or shall cause to be written to disk per second.  

              kB_ccwr/s  
                     Number  of  kilobytes  whose  writing  to  disk  has  been cancelled by the task. This may occur when the task truncates some dirty pagecache. In this case, some IO which another task has been  
                     accounted for will not be happening.  

              Command  
                     The command name of the task.  

       -h     Display all activities horizontally on a single line. This is intended to make it easier to be parsed by other programs.  

       -I     In an SMP environment, indicate that tasks CPU usage (as displayed by option -u ) should be divided by the total number of processors.  

       -l     Display the process command name and all its arguments.  

       -p { pid [,...] | SELF | ALL }  
              Select tasks (processes) for which statistics are to be reported.  pid is the process identification number. The SELF keyword indicates that statistics are to be  reported  for  the  pidstat  process  
              itself, whereas the ALL keyword indicates that statistics are to be reported for all the tasks managed by the system.  

       -r     Report page faults and memory utilization.  

              When reporting statistics for individual tasks, the following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              minflt/s  
                     Total number of minor faults the task has made per second, those which have not required loading a memory page from disk.  

              majflt/s  
                     Total number of major faults the task has made per second, those which have required loading a memory page from disk.  

              VSZ  
                     Virtual Size: The virtual memory usage of entire task in kilobytes.  

              RSS  
                     Resident Set Size: The non-swapped physical memory used by the task in kilobytes.  

              Command  
                     The command name of the task.  

              When reporting global statistics for tasks and all their children, the following values are displayed:  

              PID  
                     The identification number of the task which is being monitored together with its children.  

              minflt-nr  
                     Total number of minor faults made by the task and all its children, and collected during the interval of time.  

              majflt-nr  
                     Total number of major faults made by the task and all its children, and collected during the interval of time.  

              Command  
                     The command name of the task which is being monitored together with its children.  

       -t     Also display statistics for threads associated with selected tasks.  

              This option adds the following values to the reports:  

              TGID  
                     The identification number of the thread group leader.  

              TID  
                     The identification number of the thread being monitored.  

       -T { TASK | CHILD | ALL }  
              This  option  specifies  what  has  to be monitored by the pidstat command. The TASK keyword indicates that statistics are to be reported for individual tasks (this is the default option) whereas the  
              CHILD keyword indicates that statistics are to be globally reported for the selected tasks and all their children. The ALL keyword indicates that statistics are to be reported  for  individual  tasks  
              and globally for the selected tasks and their children.  

              Note:  Global statistics for tasks and all their children are not available for all options of pidstat.  Also these statistics are not necessarily relevant to current time interval: The statistics of  
              a child process are collected only when it finishes or it is killed.  

       -u     Report CPU utilization.  

              When reporting statistics for individual tasks, the following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              %usr  
                     Percentage of CPU used by the task while executing at the user level (application), with or without nice priority. Note that this field does NOT include time spent running a virtual processor.  

              %system  
                     Percentage of CPU used by the task while executing at the system level (kernel).  

              %guest  
                     Percentage of CPU spent by the task in virtual machine (running a virtual processor).  

              %CPU  
                     Total percentage of CPU time used by the task. In an SMP environment, the task’s CPU usage will be divided by the total number of CPU’s if option -I has been entered on the command line.  

              CPU  
                     Processor number to which the task is attached.  

              Command  
                     The command name of the task.  

              When reporting global statistics for tasks and all their children, the following values are displayed:  

              PID  
                     The identification number of the task which is being monitored together with its children.  

              usr-ms  
                     Total  number  of milliseconds spent by the task and all its children while executing at the user level (application), with or without nice priority, and collected during the interval of time.  
                     Note that this field does NOT include time spent running a virtual processor.  

              system-ms  
                     Total number of milliseconds spent by the task and all its children while executing at the system level (kernel), and collected during the interval of time.  

              guest-ms  
                     Total number of milliseconds spent by the task and all its children in virtual machine (running a virtual processor).  

              Command  
                     The command name of the task which is being monitored together with its children.  

       -V     Print version number then exit.  

       -w     Report task switching activity (kernels 2.6.23 and later only).  The following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              cswch/s  
                     Total number of voluntary context switches the task made per second.  A voluntary context switch occurs when a task blocks because it requires a resource that is unavailable.  

              nvcswch/s  
                     Total number of non voluntary context switches the task made per second.  A involuntary context switch takes place when a task executes for the duration of its time slice and then is forced to  
                     relinquish the processor.  

              Command  
                     The command name of the task.  

ENVIRONMENT  
       The pidstat command takes into account the following environment variable:  

       S_TIME_FORMAT  
              If  this  variable  exists  and  its  value  is ISO then the current locale will be ignored when printing the date in the report header.  The pidstat command will use the ISO 8601 format (YYYY-MM-DD)  
              instead.  

EXAMPLES  
       pidstat 2 5  
              Display five reports of CPU statistics for every active task in the system at two second intervals.  

       pidstat -r -p 1643 2 5  
              Display five reports of page faults and memory statistics for PID 1643 at two second intervals.  

       pidstat -T CHILD -r 2 5  
              Display five reports of page faults statistics at two second intervals for the child processes of all tasks in the system. Only child processes with non-zero statistics values are displayed.  

BUGS  
       /proc filesystem must be mounted for the pidstat command to work.  

FILES  
       /proc contains various files with system statistics.  

AUTHOR  
       Sebastien Godard (sysstat <at> orange.fr)  

SEE ALSO  
       sar(1), top(1), ps(1), mpstat(1), iostat(1), vmstat(8)  

       http://pagesperso-orange.fr/sebastien.godard/  

Linux                            DECEMBER 2008                      PIDSTAT(1)  

http://linoxide.com/linux-command/linux-pidstat-monitor-statistics-procesess/

http://www.cnblogs.com/ggjucheng/archive/2013/01/13/2858874.html

http://www.2cto.com/os/201306/217190.html

其他

mpstat  

vmstat  

top  

sar  

iostat  

dstat  

ps  
相关实践学习
基于Redis实现在线游戏积分排行榜
本场景将介绍如何基于Redis数据库实现在线游戏中的游戏玩家积分排行榜功能。
云数据库 Redis 版使用教程
云数据库Redis版是兼容Redis协议标准的、提供持久化的内存数据库服务,基于高可靠双机热备架构及可无缝扩展的集群架构,满足高读写性能场景及容量需弹性变配的业务需求。 产品详情:https://www.aliyun.com/product/kvstore &nbsp; &nbsp; ------------------------------------------------------------------------- 阿里云数据库体验:数据库上云实战 开发者云会免费提供一台带自建MySQL的源数据库&nbsp;ECS 实例和一台目标数据库&nbsp;RDS实例。跟着指引,您可以一步步实现将ECS自建数据库迁移到目标数据库RDS。 点击下方链接,领取免费ECS&amp;RDS资源,30分钟完成数据库上云实战!https://developer.aliyun.com/adc/scenario/51eefbd1894e42f6bb9acacadd3f9121?spm=a2c6h.13788135.J_3257954370.9.4ba85f24utseFl
相关文章
|
18天前
|
安全 Python
告别低效编程!Python线程与进程并发技术详解,让你的代码飞起来!
【7月更文挑战第9天】Python并发编程提升效率:**理解并发与并行,线程借助`threading`模块处理IO密集型任务,受限于GIL;进程用`multiprocessing`实现并行,绕过GIL限制。示例展示线程和进程创建及同步。选择合适模型,注意线程安全,利用多核,优化性能,实现高效并发编程。
28 3
|
18天前
|
安全 数据安全/隐私保护 数据中心
Python并发编程大挑战:线程安全VS进程隔离,你的选择影响深远!
【7月更文挑战第9天】Python并发:线程共享内存,高效但需处理线程安全(GIL限制并发),适合IO密集型;进程独立内存,安全但通信复杂,适合CPU密集型。使用`threading.Lock`保证线程安全,`multiprocessing.Queue`实现进程间通信。选择取决于任务性质和性能需求。
31 1
|
18天前
|
Python
解锁Python并发新世界:线程与进程的并行艺术,让你的应用性能翻倍!
【7月更文挑战第9天】并发编程**是同时执行多个任务的技术,提升程序效率。Python的**threading**模块支持多线程,适合IO密集型任务,但受GIL限制。**multiprocessing**模块允许多进程并行,绕过GIL,适用于CPU密集型任务。例如,计算平方和,多线程版本使用`threading`分割工作并同步结果;多进程版本利用`multiprocessing.Pool`分块计算再合并。正确选择能优化应用性能。
|
2天前
|
Java
如何使用jstack命令查看Java进程的线程栈
如何使用jstack命令查看Java进程的线程栈?
10 2
|
13天前
|
消息中间件 安全 数据处理
Python中的并发编程:理解多线程与多进程的区别与应用
在Python编程中,理解并发编程是提高程序性能和响应速度的关键。本文将深入探讨多线程和多进程的区别、适用场景及实际应用,帮助开发者更好地利用Python进行并发编程。
|
17天前
|
数据库 数据安全/隐私保护 C++
Python并发编程实战:线程(threading)VS进程(multiprocessing),谁才是并发之王?
【7月更文挑战第10天】Python并发对比:线程轻量级,适合I/O密集型任务,但受GIL限制;进程绕过GIL,擅CPU密集型,但通信成本高。选择取决于应用场景,线程利于数据共享,进程利于多核利用。并发无“王者”,灵活运用方为上策。
|
18天前
|
安全 API 调度
深度剖析:Python并发编程中的线程与进程,那些你不可不知的使用技巧与限制!
【7月更文挑战第9天】Python并发:线程适合IO密集型任务,利用GIL下的多线程同步,如示例中使用锁。进程适用于CPU密集型,通过multiprocessing模块实现多进程,利用进程间通信如队列。线程受限于GIL,进程间通信成本高。选择取决于任务需求和性能目标。
17 2
|
15天前
|
缓存 Linux 编译器
【Linux】多线程——线程概念|进程VS线程|线程控制(下)
【Linux】多线程——线程概念|进程VS线程|线程控制(下)
28 0
|
15天前
|
存储 Linux 调度
【Linux】多线程——线程概念|进程VS线程|线程控制(上)
【Linux】多线程——线程概念|进程VS线程|线程控制(上)
34 0
|
16天前
|
存储 安全 Java
Java面试题:假设你正在开发一个Java后端服务,该服务需要处理高并发的用户请求,并且对内存使用效率有严格的要求,在多线程环境下,如何确保共享资源的线程安全?
Java面试题:假设你正在开发一个Java后端服务,该服务需要处理高并发的用户请求,并且对内存使用效率有严格的要求,在多线程环境下,如何确保共享资源的线程安全?
23 0