pidstat 统计进程、线程、进程组的资源消耗

本文涉及的产品
RDS MySQL Serverless 基础系列,0.5-2RCU 50GB
云原生数据库 PolarDB MySQL 版,Serverless 5000PCU 100GB
云原生数据库 PolarDB 分布式版,标准版 2核8GB
简介:

标签

PostgreSQL , Linux , pidstat


背景

如果我们要查看Linux下面进程、进程组、线程的资源消耗的统计信息,可以使用pidstat,它可以收集并报告进程的统计信息。

pidstat实际上也是将/proc/pid下的统计信息统筹后展现给用户。

pidstat例子

1. 列出IO统计信息:-d

2. 列出内存使用统计,page fault:-r

3. CPU统计信息:-u

4. 上下文切换统计信息:-w

5. 指定输出的维度

指定命令: -C command

指定进程号:-p { pid [,...] | SELF | ALL }

将所有列输出到单行、便于导入单张表中:-h

按每个CPU核的统计:-I

按所有CPU核统计:默认

列出命令的完整内容,包括参数:-l

列出线程统计信息:-t

按进程维度单独统计、按进程全局+子进程单独统计、按进程全局、单独统计同时按子任务单独统计:-T { TASK | CHILD | ALL }

实例,统计5秒的值并输出

#pidstat -d -r -u -w -l -h -p ALL 5 1|less  


#      Time       PID    %usr %system  %guest    %CPU   CPU  minflt/s  majflt/s     VSZ    RSS   %MEM   kB_rd/s   kB_wr/s kB_ccwr/s   cswch/s nvcswch/s  Command  
 1488541252         1    0.00    0.00    0.00    0.00     0      0.00      0.00   19348   1512   0.00      0.00      0.00      0.00      0.00      0.00  /sbin/init   
 1488541252         2    0.00    0.00    0.00    0.00    50      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  kthreadd  
 1488541252         3    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.59      0.00  migration/0  
 1488541252         4    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  ksoftirqd/0  
 1488541252         5    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  migration/0  
 1488541252         6    0.00    0.00    0.00    0.00     0      0.00      0.00       0      0   0.00      0.00      0.00      0.00      0.00      0.00  watchdog/0  

 ......  

 1488541264     18523    0.00    0.20    0.00    0.20     5     10.78      0.00 137774940 3002232   0.57      0.00      0.00      0.00      1.96      0.20  /home/digoal/pgsql9.6/bin/postgres   
 1488541264     18549    0.00    0.00    0.00    0.00    61      0.00      0.00  140892   2028   0.00      0.00      0.00      0.00      0.00      0.00  postgres: logger process               
 1488541264     18552    0.00    0.00    0.00    0.00     0      0.00      0.00 137779116 5509520   1.04      0.00      0.00      0.00      0.00      0.00  postgres: checkpointer process         
 1488541264     18554    0.00    0.00    0.00    0.00    20      0.00      0.00 137776328 1058572   0.20      0.00      0.00      0.00      3.92      0.00  postgres: writer process               
 1488541264     18556    0.00    0.00    0.00    0.00    15      0.00      0.00 137774940  19120   0.00      0.00      0.00      0.00      3.92      0.00  postgres: wal writer process           
 1488541264     18557    0.00    0.00    0.00    0.00    21      2.94      0.00 137779092   3404   0.00      0.00      0.00      0.00      3.92      0.00  postgres: autovacuum launcher process     
 1488541264     18559    0.00    0.00    0.00    0.00    11      0.00      0.00  142988   2092   0.00      0.00      0.00      0.00      0.00      0.00  postgres: archiver process   last was 0000000100000030000000B2  
 1488541264     18561    0.00    0.00    0.00    0.00    53      4.90      0.00  143556   2624   0.00      0.00    317.65    317.65      4.12      0.20  postgres: stats collector process      

参考

man pidstat

PIDSTAT(1)                    Linux User’s Manual                   PIDSTAT(1)  

NAME  
       pidstat - Report statistics for Linux tasks.  

SYNOPSIS  
       pidstat [ -C comm ] [ -d ] [ -h ] [ -I ] [ -l ] [ -p { pid [,...] | SELF | ALL } ] [ -r ] [ -t ] [ -T { TASK | CHILD | ALL } ] [ -u ] [ -V ] [ -w ] [ interval [ count ] ]  

DESCRIPTION  
       The  pidstat  command  is  used for monitoring individual tasks currently being managed by the Linux kernel.  It writes to standard output activities for every task selected with option -p or for every task  
       managed by the Linux kernel if option -p ALL has been used. Not selecting any tasks is equivalent to specifying -p ALL but only active tasks (tasks with  non-zero  statistics  values)  will  appear  in  the  
       report.  

       The pidstat command can also be used for monitoring the child processes of selected tasks.  Read about option -T below.  

       The  interval  parameter  specifies  the  amount  of time in seconds between each report.  A value of 0 (or no parameters at all) indicates that tasks statistics are to be reported for the time since system  
       startup (boot).  The count parameter can be specified in conjunction with the interval parameter if this one is not set to zero. The value of count determines the number of  reports  generated  at  interval  
       seconds apart. If the interval parameter is specified without the count parameter, the pidstat command generates reports continuously.  

       You can select information about specific task activities using flags.  Not specifying any flags selects only CPU activity.  

OPTIONS  
       -C comm  
              Display only tasks whose command name includes the string comm.  

       -d     Report I/O statistics (kernels 2.6.20 and later only).  The following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              kB_rd/s  
                     Number of kilobytes the task has caused to be read from disk per second.  

              kB_wr/s  
                     Number of kilobytes the task has caused, or shall cause to be written to disk per second.  

              kB_ccwr/s  
                     Number  of  kilobytes  whose  writing  to  disk  has  been cancelled by the task. This may occur when the task truncates some dirty pagecache. In this case, some IO which another task has been  
                     accounted for will not be happening.  

              Command  
                     The command name of the task.  

       -h     Display all activities horizontally on a single line. This is intended to make it easier to be parsed by other programs.  

       -I     In an SMP environment, indicate that tasks CPU usage (as displayed by option -u ) should be divided by the total number of processors.  

       -l     Display the process command name and all its arguments.  

       -p { pid [,...] | SELF | ALL }  
              Select tasks (processes) for which statistics are to be reported.  pid is the process identification number. The SELF keyword indicates that statistics are to be  reported  for  the  pidstat  process  
              itself, whereas the ALL keyword indicates that statistics are to be reported for all the tasks managed by the system.  

       -r     Report page faults and memory utilization.  

              When reporting statistics for individual tasks, the following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              minflt/s  
                     Total number of minor faults the task has made per second, those which have not required loading a memory page from disk.  

              majflt/s  
                     Total number of major faults the task has made per second, those which have required loading a memory page from disk.  

              VSZ  
                     Virtual Size: The virtual memory usage of entire task in kilobytes.  

              RSS  
                     Resident Set Size: The non-swapped physical memory used by the task in kilobytes.  

              Command  
                     The command name of the task.  

              When reporting global statistics for tasks and all their children, the following values are displayed:  

              PID  
                     The identification number of the task which is being monitored together with its children.  

              minflt-nr  
                     Total number of minor faults made by the task and all its children, and collected during the interval of time.  

              majflt-nr  
                     Total number of major faults made by the task and all its children, and collected during the interval of time.  

              Command  
                     The command name of the task which is being monitored together with its children.  

       -t     Also display statistics for threads associated with selected tasks.  

              This option adds the following values to the reports:  

              TGID  
                     The identification number of the thread group leader.  

              TID  
                     The identification number of the thread being monitored.  

       -T { TASK | CHILD | ALL }  
              This  option  specifies  what  has  to be monitored by the pidstat command. The TASK keyword indicates that statistics are to be reported for individual tasks (this is the default option) whereas the  
              CHILD keyword indicates that statistics are to be globally reported for the selected tasks and all their children. The ALL keyword indicates that statistics are to be reported  for  individual  tasks  
              and globally for the selected tasks and their children.  

              Note:  Global statistics for tasks and all their children are not available for all options of pidstat.  Also these statistics are not necessarily relevant to current time interval: The statistics of  
              a child process are collected only when it finishes or it is killed.  

       -u     Report CPU utilization.  

              When reporting statistics for individual tasks, the following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              %usr  
                     Percentage of CPU used by the task while executing at the user level (application), with or without nice priority. Note that this field does NOT include time spent running a virtual processor.  

              %system  
                     Percentage of CPU used by the task while executing at the system level (kernel).  

              %guest  
                     Percentage of CPU spent by the task in virtual machine (running a virtual processor).  

              %CPU  
                     Total percentage of CPU time used by the task. In an SMP environment, the task’s CPU usage will be divided by the total number of CPU’s if option -I has been entered on the command line.  

              CPU  
                     Processor number to which the task is attached.  

              Command  
                     The command name of the task.  

              When reporting global statistics for tasks and all their children, the following values are displayed:  

              PID  
                     The identification number of the task which is being monitored together with its children.  

              usr-ms  
                     Total  number  of milliseconds spent by the task and all its children while executing at the user level (application), with or without nice priority, and collected during the interval of time.  
                     Note that this field does NOT include time spent running a virtual processor.  

              system-ms  
                     Total number of milliseconds spent by the task and all its children while executing at the system level (kernel), and collected during the interval of time.  

              guest-ms  
                     Total number of milliseconds spent by the task and all its children in virtual machine (running a virtual processor).  

              Command  
                     The command name of the task which is being monitored together with its children.  

       -V     Print version number then exit.  

       -w     Report task switching activity (kernels 2.6.23 and later only).  The following values are displayed:  

              PID  
                     The identification number of the task being monitored.  

              cswch/s  
                     Total number of voluntary context switches the task made per second.  A voluntary context switch occurs when a task blocks because it requires a resource that is unavailable.  

              nvcswch/s  
                     Total number of non voluntary context switches the task made per second.  A involuntary context switch takes place when a task executes for the duration of its time slice and then is forced to  
                     relinquish the processor.  

              Command  
                     The command name of the task.  

ENVIRONMENT  
       The pidstat command takes into account the following environment variable:  

       S_TIME_FORMAT  
              If  this  variable  exists  and  its  value  is ISO then the current locale will be ignored when printing the date in the report header.  The pidstat command will use the ISO 8601 format (YYYY-MM-DD)  
              instead.  

EXAMPLES  
       pidstat 2 5  
              Display five reports of CPU statistics for every active task in the system at two second intervals.  

       pidstat -r -p 1643 2 5  
              Display five reports of page faults and memory statistics for PID 1643 at two second intervals.  

       pidstat -T CHILD -r 2 5  
              Display five reports of page faults statistics at two second intervals for the child processes of all tasks in the system. Only child processes with non-zero statistics values are displayed.  

BUGS  
       /proc filesystem must be mounted for the pidstat command to work.  

FILES  
       /proc contains various files with system statistics.  

AUTHOR  
       Sebastien Godard (sysstat <at> orange.fr)  

SEE ALSO  
       sar(1), top(1), ps(1), mpstat(1), iostat(1), vmstat(8)  

       http://pagesperso-orange.fr/sebastien.godard/  

Linux                            DECEMBER 2008                      PIDSTAT(1)  

http://linoxide.com/linux-command/linux-pidstat-monitor-statistics-procesess/

http://www.cnblogs.com/ggjucheng/archive/2013/01/13/2858874.html

http://www.2cto.com/os/201306/217190.html

其他

mpstat  

vmstat  

top  

sar  

iostat  

dstat  

ps  
相关实践学习
基于Redis实现在线游戏积分排行榜
本场景将介绍如何基于Redis数据库实现在线游戏中的游戏玩家积分排行榜功能。
云数据库 Redis 版使用教程
云数据库Redis版是兼容Redis协议标准的、提供持久化的内存数据库服务,基于高可靠双机热备架构及可无缝扩展的集群架构,满足高读写性能场景及容量需弹性变配的业务需求。 产品详情:https://www.aliyun.com/product/kvstore &nbsp; &nbsp; ------------------------------------------------------------------------- 阿里云数据库体验:数据库上云实战 开发者云会免费提供一台带自建MySQL的源数据库&nbsp;ECS 实例和一台目标数据库&nbsp;RDS实例。跟着指引,您可以一步步实现将ECS自建数据库迁移到目标数据库RDS。 点击下方链接,领取免费ECS&amp;RDS资源,30分钟完成数据库上云实战!https://developer.aliyun.com/adc/scenario/51eefbd1894e42f6bb9acacadd3f9121?spm=a2c6h.13788135.J_3257954370.9.4ba85f24utseFl
相关文章
|
15天前
|
消息中间件 存储 缓存
【嵌入式软件工程师面经】Linux系统编程(线程进程)
【嵌入式软件工程师面经】Linux系统编程(线程进程)
26 1
|
4天前
|
分布式计算 JavaScript 前端开发
多线程、多进程、协程的概念、区别与联系
多线程、多进程、协程的概念、区别与联系
17 1
|
11天前
|
存储 网络协议 算法
【进程与线程】最好懂的讲解
【进程与线程】最好懂的讲解
17 1
|
3天前
|
安全 Java Python
GIL是Python解释器的锁,确保单个进程中字节码执行的串行化,以保护内存管理,但限制了多线程并行性。
【6月更文挑战第20天】GIL是Python解释器的锁,确保单个进程中字节码执行的串行化,以保护内存管理,但限制了多线程并行性。线程池通过预创建线程池来管理资源,减少线程创建销毁开销,提高效率。示例展示了如何使用Python实现一个简单的线程池,用于执行多个耗时任务。
15 6
|
3天前
|
Java
死锁是线程间争夺资源造成的无限等待现象,Java示例展示了两个线程各自持有资源并等待对方释放,导致死锁。`
【6月更文挑战第20天】死锁是线程间争夺资源造成的无限等待现象,Java示例展示了两个线程各自持有资源并等待对方释放,导致死锁。`volatile`保证变量的可见性和部分原子性,确保多线程环境中值的即时更新。与`synchronized`相比,`volatile`作用于单个变量,不保证原子操作,同步范围有限,但开销较小。`synchronized`提供更全面的内存语义,保证原子性和可见性,适用于复杂并发控制。
13 3
|
2天前
|
存储 调度 C++
【操作系统】进程与线程的区别及总结(非常非常重要,面试必考题,其它文章可以不看,但这篇文章最后的总结你必须要看,满满的全是干货......)
【操作系统】进程与线程的区别及总结(非常非常重要,面试必考题,其它文章可以不看,但这篇文章最后的总结你必须要看,满满的全是干货......)
22 1
|
6天前
|
数据挖掘 调度 开发者
Python并发编程的艺术:掌握线程、进程与协程的同步技巧
并发编程在Python中涵盖线程、进程和协程,用于优化IO操作和响应速度。`threading`模块支持线程,`multiprocessing`处理进程,而`asyncio`则用于协程。线程通过Lock和Condition Objects同步,进程使用Queue和Pipe通信。协程利用异步事件循环避免上下文切换。了解并发模型及同步技术是提升Python应用性能的关键。
26 5
|
5天前
|
Java 程序员
Java多线程编程是指在一个进程中创建并运行多个线程,每个线程执行不同的任务,并行地工作,以达到提高效率的目的
【6月更文挑战第18天】Java多线程提升效率,通过synchronized关键字、Lock接口和原子变量实现同步互斥。synchronized控制共享资源访问,基于对象内置锁。Lock接口提供更灵活的锁管理,需手动解锁。原子变量类(如AtomicInteger)支持无锁的原子操作,减少性能影响。
18 3
|
5天前
|
数据采集 自然语言处理 调度
【干货】python多进程和多线程谁更快
【干货】python多进程和多线程谁更快
12 2
|
9天前
|
消息中间件 分布式计算 物联网
深入理解操作系统之进程与线程管理
操作系统的核心职责之一是进程与线程管理,它关乎系统的效率和稳定性。本文将剖析进程与线程的基本概念、生命周期以及它们在现代操作系统中的实现机制。通过对比分析,我们将揭示进程与线程的区别、优势及其适用场景,并探讨它们对系统性能的具体影响。进一步,文章将讨论进程间通信(IPC)的几种方式,以及同步和异步处理在多任务环境中的重要性。最后,我们将展望未来操作系统在进程与线程管理方面可能的发展趋势。