如何分析D状态进程

在使用top查看进程状态时，我们有时候会看到D状态的进程。

       w: S  --  Process Status
          The status of the task which can be one of:
             ’D’ = uninterruptible sleep
             ’R’ = running
             ’S’ = sleeping
             ’T’ = traced or stopped
             ’Z’ = zombie

D是一种不可中断的sleep，如果你发现大量的D状态的进程，这个时候这些进程实际上是没有在处理业务逻辑的。
例如使用PostgreSQL时，批量的往数据库导入数据，如果导入的数据量大到OS脏页回写的速度赶不上写入的速度时，并且用户刷dirty page的阈值到达，用户进程会需要主动刷脏页。

vm.dirty_background_ratio = 10
vm.dirty_background_bytes = 0
vm.dirty_ratio = 20
vm.dirty_bytes = 0
vm.dirty_writeback_centisecs = 50
vm.dirty_expire_centisecs = 6000  

例如以上配置，OS脏页超过20%时，用户调write也需要主动的刷脏页，就会看到进程处于D状态，直到脏页水位下降到10%以下。
当然还有其他的原因会导致进程进入D状态，我们需要观察进程的stack，看看它处于什么状态。
例如处于R状态的COPY PostgreSQL进程，它的stack是什么样的？

cat /proc/17944/status ; echo -e "\n"; cat /proc/17944/stack
Name:   postgres
State:  R (running)
Tgid:   17944
Pid:    17944
PPid:   57925
TracerPid:      0
Uid:    123293  123293  123293  123293
Gid:    100     100     100     100
Utrace: 0
FDSize: 64
Groups: 100 19001 
VmPeak: 272294920 kB
VmSize:   119788 kB
VmLck:         0 kB
VmHWM:      3244 kB
VmRSS:      2812 kB
VmData:     2140 kB
VmStk:       152 kB
VmExe:      5852 kB
VmLib:      2400 kB
VmPTE:        64 kB
VmSwap:        0 kB
Threads:        1
SigQ:   0/4131614
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001301800
SigCgt: 0000000180006287
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
Cpus_allowed:   ffffffff,ffffffff
Cpus_allowed_list:      0-63
Mems_allowed:   00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001
Mems_allowed_list:      0
voluntary_ctxt_switches:        55758
nonvoluntary_ctxt_switches:     103995


[<ffffffff8121b40a>] sys_semtimedop+0x81a/0x840
[<ffffffffffffffff>] 0xffffffffffffffff

例如处于D状态的COPY PostgreSQL进程，它的stack是什么样的？
可以看到它处于刷脏页速度受限的状态，与ext4内核有关。

cat /proc/17944/status ; echo -e "\n"; cat /proc/17944/stack
Name:   postgres
State:  D (disk sleep)
Tgid:   17944
Pid:    17944
PPid:   57925
TracerPid:      0
Uid:    123293  123293  123293  123293
Gid:    100     100     100     100
Utrace: 0
FDSize: 64
Groups: 100 19001 
VmPeak: 272294920 kB
VmSize:   119788 kB
VmLck:         0 kB
VmHWM:      3244 kB
VmRSS:      2812 kB
VmData:     2140 kB
VmStk:       152 kB
VmExe:      5852 kB
VmLib:      2400 kB
VmPTE:        64 kB
VmSwap:        0 kB
Threads:        1
SigQ:   0/4131614
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000001301800
SigCgt: 0000000180006287
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
Cpus_allowed:   ffffffff,ffffffff
Cpus_allowed_list:      0-63
Mems_allowed:   00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001
Mems_allowed_list:      0
voluntary_ctxt_switches:        55922
nonvoluntary_ctxt_switches:     104189


[<ffffffff81133cf0>] balance_dirty_pages_ratelimited_nr+0x2d0/0x9a0
[<ffffffff8111f19a>] generic_file_buffered_write+0x1da/0x2e0
[<ffffffff81120fe0>] __generic_file_aio_write+0x260/0x490
[<ffffffff81121298>] generic_file_aio_write+0x88/0x100
[<ffffffffa00b9463>] ext4_file_write+0x43/0xe0 [ext4]
[<ffffffff8118863a>] do_sync_write+0xfa/0x140
[<ffffffff81188938>] vfs_write+0xb8/0x1a0
[<ffffffff81189231>] sys_write+0x51/0x90
[<ffffffff8100c072>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff