１.启动 Profiling
[root@192_168_174_68 ~]# glustergluster> volume profile gv_KVM start
Starting volume profile on gv_KVM has been successful
２.查看I/0 状态
[root@192_168_174_68 ~]# gluster
gluster> volume profile gv_KVM info
Brick: 192.168.174.68:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 11 598 261
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1261 121 85
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 757
No. of Writes: 39 30 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.00 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.01 34.00 us 34.00 us 34.00 us 1 LOOKUP
0.03 66.00 us 23.00 us 109.00 us 2 READDIR
0.14 152.25 us 136.00 us 192.00 us 4 READ
0.17 84.22 us 42.00 us 117.00 us 9 GETXATTR
0.33 104.86 us 85.00 us 156.00 us 14 FINODELK
0.62 196.36 us 149.00 us 245.00 us 14 FXATTROP
98.71 23198.74 us 17434.00 us 32511.00 us 19 WRITE
Duration: 5528 seconds
Data Read: 99221504 bytes
Data Written: 255271424 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 11 598 261
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1261 121 85
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 757
No. of Writes: 39 30 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.00 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.01 34.00 us 34.00 us 34.00 us 1 LOOKUP
0.03 66.00 us 23.00 us 109.00 us 2 READDIR
0.14 152.25 us 136.00 us 192.00 us 4 READ
0.17 84.22 us 42.00 us 117.00 us 9 GETXATTR
0.33 104.86 us 85.00 us 156.00 us 14 FINODELK
0.62 196.36 us 149.00 us 245.00 us 14 FXATTROP
98.71 23198.74 us 17434.00 us 32511.00 us 19 WRITE
Duration: 5528 seconds
Data Read: 99221504 bytes
Data Written: 255271424 bytes
Brick: 192.168.174.81:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 10 606 277
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1158 139 62
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 767
No. of Writes: 53 26 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.01 2.00 us 2.00 us 2.00 us 1 OPENDIR
0.20 67.00 us 67.00 us 67.00 us 1 LOOKUP
0.52 89.50 us 73.00 us 106.00 us 2 READDIR
2.99 114.00 us 71.00 us 167.00 us 9 GETXATTR
7.11 174.14 us 134.00 us 272.00 us 14 READ
8.17 116.75 us 86.00 us 163.00 us 24 FINODELK
13.47 192.38 us 154.00 us 222.00 us 24 FXATTROP
67.53 826.68 us 605.00 us 1028.00 us 28 WRITE
Duration: 5528 seconds
Data Read: 100532224 bytes
Data Written: 254659072 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 10 606 277
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1158 139 62
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 767
No. of Writes: 53 26 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.01 2.00 us 2.00 us 2.00 us 1 OPENDIR
0.20 67.00 us 67.00 us 67.00 us 1 LOOKUP
0.52 89.50 us 73.00 us 106.00 us 2 READDIR
2.99 114.00 us 71.00 us 167.00 us 9 GETXATTR
7.11 174.14 us 134.00 us 272.00 us 14 READ
8.17 116.75 us 86.00 us 163.00 us 24 FINODELK
13.47 192.38 us 154.00 us 222.00 us 24 FXATTROP
67.53 826.68 us 605.00 us 1028.00 us 28 WRITE
Duration: 5528 seconds
Data Read: 100532224 bytes
Data Written: 254659072 bytes
Brick: 192.168.174.80:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 40 512 260
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1405 168 53
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 59 21 1849
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 1 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.01 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.31 121.00 us 121.00 us 121.00 us 1 LOOKUP
0.63 124.50 us 96.00 us 153.00 us 2 READDIR
2.89 126.67 us 24.00 us 170.00 us 9 GETXATTR
6.34 108.83 us 73.00 us 165.00 us 23 FINODELK
11.77 202.04 us 152.00 us 492.00 us 23 FXATTROP
78.06 616.40 us 212.00 us 911.00 us 50 WRITE
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 256275968 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 40 512 260
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1405 168 53
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 59 21 1849
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 1 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.01 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.31 121.00 us 121.00 us 121.00 us 1 LOOKUP
0.63 124.50 us 96.00 us 153.00 us 2 READDIR
2.89 126.67 us 24.00 us 170.00 us 9 GETXATTR
6.34 108.83 us 73.00 us 165.00 us 23 FINODELK
11.77 202.04 us 152.00 us 492.00 us 23 FXATTROP
78.06 616.40 us 212.00 us 911.00 us 50 WRITE
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 256275968 bytes
Brick: 192.168.167.58:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 7 393 258
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1409 120 49
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 754
No. of Writes: 55 24 1837
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 2 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.02 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.17 29.00 us 29.00 us 29.00 us 1 LOOKUP
0.59 51.00 us 24.00 us 78.00 us 2 GETXATTR
0.93 80.00 us 24.00 us 136.00 us 2 READDIR
3.42 197.33 us 154.00 us 241.00 us 3 READ
6.71 145.00 us 44.00 us 432.00 us 8 FINODELK
8.84 191.12 us 157.00 us 262.00 us 8 FXATTROP
79.32 1246.91 us 548.00 us 3541.00 us 11 WRITE
Duration: 5528 seconds
Data Read: 98828288 bytes
Data Written: 254124032 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 7 393 258
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1409 120 49
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 754
No. of Writes: 55 24 1837
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 2 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.02 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.17 29.00 us 29.00 us 29.00 us 1 LOOKUP
0.59 51.00 us 24.00 us 78.00 us 2 GETXATTR
0.93 80.00 us 24.00 us 136.00 us 2 READDIR
3.42 197.33 us 154.00 us 241.00 us 3 READ
6.71 145.00 us 44.00 us 432.00 us 8 FINODELK
8.84 191.12 us 157.00 us 262.00 us 8 FXATTROP
79.32 1246.91 us 548.00 us 3541.00 us 11 WRITE
Duration: 5528 seconds
Data Read: 98828288 bytes
Data Written: 254124032 bytes
Brick: 192.168.167.52:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 11 598 261
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1261 121 85
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 39 30 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.00 2.00 us 2.00 us 2.00 us 1 OPENDIR
0.00 50.00 us 50.00 us 50.00 us 1 LOOKUP
0.01 62.00 us 37.00 us 87.00 us 2 READDIR
0.02 72.00 us 32.00 us 97.00 us 3 GETXATTR
0.07 66.50 us 41.00 us 106.00 us 14 FINODELK
45.46 30512.58 us 18533.00 us 139514.00 us 19 WRITE
54.43 49580.00 us 124.00 us 691933.00 us 14 FXATTROP
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 255271424 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 11 598 261
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1261 121 85
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 39 30 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.00 2.00 us 2.00 us 2.00 us 1 OPENDIR
0.00 50.00 us 50.00 us 50.00 us 1 LOOKUP
0.01 62.00 us 37.00 us 87.00 us 2 READDIR
0.02 72.00 us 32.00 us 97.00 us 3 GETXATTR
0.07 66.50 us 41.00 us 106.00 us 14 FINODELK
45.46 30512.58 us 18533.00 us 139514.00 us 19 WRITE
54.43 49580.00 us 124.00 us 691933.00 us 14 FXATTROP
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 255271424 bytes
Brick: 192.168.174.79:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 40 512 260
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1405 168 53
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 855
No. of Writes: 59 21 1849
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 1 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.01 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.15 75.00 us 75.00 us 75.00 us 1 LOOKUP
0.36 91.50 us 72.00 us 111.00 us 2 READDIR
1.99 113.56 us 50.00 us 169.00 us 9 GETXATTR
2.66 151.89 us 107.00 us 225.00 us 9 READ
5.52 123.39 us 90.00 us 145.00 us 23 FINODELK
9.43 210.65 us 177.00 us 249.00 us 23 FXATTROP
79.90 821.30 us 324.00 us 1139.00 us 50 WRITE
Duration: 5528 seconds
Data Read: 112066560 bytes
Data Written: 256275968 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 40 512 260
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1405 168 53
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 855
No. of Writes: 59 21 1849
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 1 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.01 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.15 75.00 us 75.00 us 75.00 us 1 LOOKUP
0.36 91.50 us 72.00 us 111.00 us 2 READDIR
1.99 113.56 us 50.00 us 169.00 us 9 GETXATTR
2.66 151.89 us 107.00 us 225.00 us 9 READ
5.52 123.39 us 90.00 us 145.00 us 23 FINODELK
9.43 210.65 us 177.00 us 249.00 us 23 FXATTROP
79.90 821.30 us 324.00 us 1139.00 us 50 WRITE
Duration: 5528 seconds
Data Read: 112066560 bytes
Data Written: 256275968 bytes
Brick: 192.168.167.76:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 7 393 258
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1409 120 49
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 55 24 1837
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 2 RELEASE
0.00 0.00 us 0.00 us 0.00 us 50 RELEASEDIR
0.00 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.04 90.00 us 90.00 us 90.00 us 1 LOOKUP
0.22 269.50 us 35.00 us 504.00 us 2 READDIR
0.24 66.11 us 23.00 us 118.00 us 9 GETXATTR
0.31 77.40 us 35.00 us 136.00 us 10 FINODELK
0.75 186.50 us 153.00 us 280.00 us 10 FXATTROP
98.45 20465.25 us 17265.00 us 24661.00 us 12 WRITE
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 254124032 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 7 393 258
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1409 120 49
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 55 24 1837
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 2 RELEASE
0.00 0.00 us 0.00 us 0.00 us 50 RELEASEDIR
0.00 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.04 90.00 us 90.00 us 90.00 us 1 LOOKUP
0.22 269.50 us 35.00 us 504.00 us 2 READDIR
0.24 66.11 us 23.00 us 118.00 us 9 GETXATTR
0.31 77.40 us 35.00 us 136.00 us 10 FINODELK
0.75 186.50 us 153.00 us 280.00 us 10 FXATTROP
98.45 20465.25 us 17265.00 us 24661.00 us 12 WRITE
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 254124032 bytes
Brick: 192.168.174.67:/opt/brick/gv_KVM
---------------------------------------
Cumulative Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 10 606 277
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1158 139 62
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 53 26 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.00 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.01 83.00 us 83.00 us 83.00 us 1 LOOKUP
0.02 102.00 us 76.00 us 128.00 us 2 READDIR
0.07 138.20 us 56.00 us 241.00 us 5 GETXATTR
0.28 123.75 us 71.00 us 161.00 us 24 FINODELK
3.52 1531.75 us 175.00 us 18628.00 us 24 FXATTROP
96.10 35840.75 us 19924.00 us 52197.00 us 28 WRITE
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 254659072 bytes
Interval 0 Stats:
Block Size: 512b+ 1024b+ 2048b+
No. of Reads: 0 0 0
No. of Writes: 10 606 277
Block Size: 4096b+ 8192b+ 16384b+
No. of Reads: 0 0 0
No. of Writes: 1158 139 62
Block Size: 32768b+ 65536b+ 131072b+
No. of Reads: 0 0 0
No. of Writes: 53 26 1843
%-latency Avg-latency Min-Latency Max-Latency No. of calls Fop
--------- ----------- ----------- ----------- ------------ ----
0.00 0.00 us 0.00 us 0.00 us 3 RELEASE
0.00 0.00 us 0.00 us 0.00 us 49 RELEASEDIR
0.00 3.00 us 3.00 us 3.00 us 1 OPENDIR
0.01 83.00 us 83.00 us 83.00 us 1 LOOKUP
0.02 102.00 us 76.00 us 128.00 us 2 READDIR
0.07 138.20 us 56.00 us 241.00 us 5 GETXATTR
0.28 123.75 us 71.00 us 161.00 us 24 FINODELK
3.52 1531.75 us 175.00 us 18628.00 us 24 FXATTROP
96.10 35840.75 us 19924.00 us 52197.00 us 28 WRITE
Duration: 5528 seconds
Data Read: 0 bytes
Data Written: 254659072 bytes
gluster>
３.使用GlusterFS Volume top 命令
GlusterFS Volume Top 命令允许查看 glusterfs每个 bricks的性能值,比如 read, write, file open calls, file read calls, file write calls, directory open calls, and directory real calls. Top命令最多显示100个结果.
- Viewing Open fd Count and Maximum fd Count
- Viewing Highest File Read Calls
- Viewing Highest File Write Calls
- Viewing Highest Open Calls on Directories
- Viewing Highest Read Calls on Directory
- Viewing List of Read Performance on each Brick
- Viewing List of Write Performance on each Brick
gluster> volume top gv_KVM open
Brick: 192.168.174.68:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 3, Max openfd time: 2015-10-29 09:41:37.030604
Count filename
=======================
3 /template_centos7.img
Brick: 192.168.167.52:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 3, Max openfd time: 2015-10-29 09:41:37.034485
Count filename
=======================
3 /template_centos7.img
Brick: 192.168.174.79:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 1, Max openfd time: 2015-10-29 09:36:34.160011
Count filename
=======================
1 /template_centos7.img
Brick: 192.168.167.58:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 2, Max openfd time: 2015-10-29 09:41:34.015363
Count filename
=======================
2 /template_centos7.img
Brick: 192.168.174.80:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 1, Max openfd time: 2015-10-29 09:36:34.122131
Count filename
=======================
1 /template_centos7.img
Brick: 192.168.174.67:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 3, Max openfd time: 2015-10-29 09:41:32.004093
Count filename
=======================
3 /template_centos7.img
Brick: 192.168.174.81:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 3, Max openfd time: 2015-10-29 09:41:31.965186
Count filename
=======================
3 /template_centos7.img
Brick: 192.168.167.76:/opt/brick/gv_KVM
Current open fds: 1, Max open fds: 2, Max openfd time: 2015-10-29 09:41:34.015011
Count filename
=======================
2 /template_centos7.img
gluster>
