备案控制台
开发者社区 开发与运维 文章 正文

3天玩转shell--12.实战编写nginx日志统计脚本【终结篇】

2023-02-16 416
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
简介： 本节课是shell教程系列的最后一篇文章，希望这12篇文章对shell的学习者有帮助。往后有空会发表一些关于golang编程的实战系列文章、或者是k8s相关的文章。

一、功能介绍：

【1】统计各种状态码的数量
【2】统计访问最多的referer
【3】统计访问最高的uri
【4】统计访问最多的 ip和ua
【5】统计每分钟的请求数、流量、请求时间、状态码等

二、通过代码进行讲解

本节是通过编写一段统计访问日志，统计日志的状态码、浏览器类型、URL、流量、状态码分布等信息，这对于网站的维护人员，能很好的掌握网站的运行情况。将附件保存12.example.txt 
#!/bin/bash
#12.sh v1
PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"
export LANG=zh_CN.UTF-8
export PATH

#统计状态码并排序
GroupByStatusCode(){
  file=$1
  cat $file |awk -F "|" '{Arry[$5]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计IP并排序
GroupByRemoteAddress(){
  file=$1
  cat $file |awk -F "|" '{Arry[$1]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计referer并排序
GroupByReferer(){
  file=$1
  cat $file |awk -F "|" '{Arry[$7]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计浏览器并排序
GroupByUserAgent(){
  file=$1
  cat $file |awk -F "|" '{Arry[$8]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计URi不含参数，并排序
GroupByUri(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," |?");Arry[aa[3]]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计URL并排序
GroupByUrl(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," ");Arry[aa[2]]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计请求方法并排序
GroupByMethod(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," |?");Arry[aa[2]]+=1;total++}END{for(s in Arry){printf "%s\t%d\t%.4f\n",s,Arry[s],Arry[s]/total}}'|sort -nr -k 2
}

#统计请求时间并排序
GroupByRequestTime(){
  file=$1
  #cat $file |awk -F "|" '{if($14<1){reqs["<1"]++}else if($14>1 && $14<5){reqs["1-5"]++}else if($14>5 && $14<10){reqs["5-10"]++}else{reqs[">10"]++};total++}END{printf "<1\t%.4f\t%d\n1-5\t%.4f\t%d\n5-10\t%.4f\t%d\n>10\t%.4f\t%d\n",reqs["<
1"]/total,reqs["<1"],reqs["1-5"]/total,reqs["1-5"],reqs["5-10"]/total,reqs["5-10"],reqs[">10"]/total,reqs[">10"]}'
  cat $file |awk -F "|" '{
  if($14<1){reqs["<1"]++}
  else if($14>1 && $14<5) {reqs["1-5"]++}
  else if($14>5 && $14<10){reqs["5-10"]++}
  else {reqs[">10"]++};
  total++
  }END{
   printf "<1\t%.4f\t%d\n1-5\t%.4f\t%d\n5-10\t%.4f\t%d\n>10\t%.4f\t%d\n",reqs["<1"]/total,reqs["<1"],reqs["1-5"]/total,reqs["1-5"],reqs["5-10"]/total,reqs["5-10"],reqs[">10"]/total,reqs[">10"]
  }'
}

#统计每分钟请求的数量
TrendTimesCount(){
 file=$1
 cat $file|awk -F "|" '{tms[substr($2,15,5)]+=1}END{for(t in tms)print t" "tms[t]}'
}

#统计每分钟的流量
TrendTimesNetFlow(){
 file=$1
 cat $file|awk -F "|" '{tms[substr($2,15,5)]+=$6}END{for(t in tms)printf "%s\t%.4fMB\n",t,tms[t]/1024/1024}'
}

#统计每分钟的数量、流量、状态码分布、平均请求时间
TrendTimesAll(){
 file=$1
 #cat $file|awk -F "|" 'BEGIN{printf "时间\t数量\t流量[MB]\t请求时间\t20x\t30x\t40x\t50x\t60x\n"}{tm=substr($2,15,5);tms[tm]+=$6;cnt[tm]+=1;reqt[tm]+=$14;if($5 ~ /20/){sc20x[tm]++}else if($5 ~ /30/){sc30x[tm]++}else if($5 ~ /40/){sc40x[
tm]++}else if($5 ~ /50/){sc50x[tm]++}else{sc60x[tm]++};}END{for(t in tms)printf "%s\t%d\t%.4f\t\t%.4f\t\t%d\t%d\t%d\t%d\t%d\n",t,cnt[t],tms[t]/1024/1024,reqt[t]/cnt[t],sc20x[t],sc30x[t],sc40x[t],sc50x[t],sc60x[t]}'
 cat $file|awk -F "|" 'BEGIN{
      printf "时间\t数量\t流量[MB]\t请求时间\t20x\t30x\t40x\t50x\t60x\n"
  }{
      tm=substr($2,15,5);
      tms[tm]+=$6;
      cnt[tm]+=1;
      reqt[tm]+=$14;
      if($5 ~ /20/) {sc20x[tm]++}
      else if($5 ~ /30/) {sc30x[tm]++}
      else if($5 ~ /40/) {sc40x[tm]++}
      else if($5 ~ /50/) {sc50x[tm]++}
      else {sc60x[tm]++};
   }END{
   for(t in tms)
        printf "%s\t%d\t%.4f\t\t%.4f\t\t%d\t%d\t%d\t%d\t%d\n",t,cnt[t],tms[t]/1024/1024,reqt[t]/cnt[t],sc20x[t],sc30x[t],sc40x[t],sc50x[t],sc60x[t]
   }'
}

#帮助函数
Help(){
 echo "Help:"
 echo "\$1 args \$2 logfile"
 echo "s :groupby statuscode"
 echo "h :groupby remoteaddress"
 echo "e :groupby referer"
 echo "a :groupby useragent"
 echo "i :groupby uri"
 echo "r :groupby url"
 echo "t :groupby requestTime"
 echo "m :groupby method"
 echo "tc :trend count request by minute"
 echo "tn :trend count netflow by minute"
 echo "ta :trend count request,netflow,requesTime,statuscode by minute"
 echo "UseAge Example:"
 echo "$0 s access.log"
 echo "$0 h access.log"
 echo "$0 i access.log"
 echo "$0 tn access.log"
 echo "$0 ta access.log"
}

if [ $# -ne 2 ];then
   Help
   exit 1
else
   if [ ! -s $2 ];then
      echo "$2 file does not exist"
      exit 1
   fi
fi

case $1 in
    s|statuscode)
      GroupByStatusCode "$2";;
    h|remoteaddress)
      GroupByRemoteAddress $2;;
    e|referer)
      GroupByReferer $2;;
    a|useragent)
      GroupByUserAgent $2;;
    i|uri)
      GroupByUri $2;;
    r|url)
      GroupByUrl $2;;
    t|requesttime)
      GroupByRequestTime $2;;
    m|method)
      GroupByMethod $2;;
    tc|trendocunt)
      TrendTimesCount $2;;
    tn|trendnet)
      TrendTimesNetFlow $2;;
    ta|trendall)
      TrendTimesAll $2;;
    *)
      Help;;
esac

三.脚本执行结果：

1.打印帮助信息：
help.png
2.统计状态码和IP并排序：
code.png
3.统计请求方法、统计响应时间分布、统计uri
m.png
4.统计每分钟的数量、每分钟的流量、每分钟的状态码走势
t.png

四.附件

113.96.140.246 | [28/May/2020:15:32:55 +0800] | www.shell.com.cn | "GET /pcedu/ios/1106/2448633_1.html HTTP/1.1" | 200 | 8289 | "-" | "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" | "-" | - | - | - | 172.16.239.5 | 0.234 | https  | 47585
172.16.239.156 | [28/May/2020:15:32:55 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 1.232 | http  | 35949
172.16.239.158 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.982 | http  | 58399
172.16.246.51 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.545 | http  | 52384
113.96.140.246 | [28/May/2020:15:32:56 +0800] | office.shell.com.cn | "GET /1282/12825949_3.html HTTP/1.1" | 200 | 31173 | "https://office.shell.com.cn/1282/12825949_2.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" | "-" | - | - | - | 172.16.239.5 | 0.645 | https  | 47791
58.55.123.88 | [28/May/2020:15:32:56 +0800] | www1.shell.com.cn | "GET /2015/dl/client/images/btn_client.png HTTP/1.1" | 304 | 0 | "https://js.shell.com/bashTest/2015/dl/css/client.css" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.358 | https  | 26877
172.16.239.174 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.435 | http  | 40954
172.16.238.72 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /pcclub/itbbsindex/1702/intf17885.html HTTP/1.1" | 200 | 1766 | "-" | "NodeJs" | "-" | - | - | - | 172.16.239.5 | 0.519 | http  | 55833
172.16.240.137 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /product/fz/2015/1511/intf11701.html HTTP/1.1" | 200 | 0 | "-" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.905 | http  | 46830
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/409990.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.715 | https  | 47574
229.137.244.63 | [28/May/2020:15:32:56 +0800] | www1.shell.com.cn | "GET /zt/gz20140905/404/css/404.css HTTP/1.0" | 200 | 4429 | "-" | "-" | "-" | - | - | - | 172.16.239.5 | 0.926 | http  | 43571
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/385364.html" | "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36 2345Explorer/6.1.0.7934" | "-" | - | - | - | 172.16.239.5 | 0.445 | https  | 47574
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/385364.html" | "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36 2345Explorer/6.1.0.7934" | "-" | - | - | - | 172.16.239.5 | 0.506 | https  | 48098
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/63718.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.68 Safari/537.36 Core/1.53.4033.400 QQBrowser/9.6.12624.400" | "-" | - | - | - | 172.16.239.5 | 0.371 | https  | 48096
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/63718.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.68 Safari/537.36 Core/1.53.4033.400 QQBrowser/9.6.12624.400" | "-" | - | - | - | 172.16.239.5 | 0.341 | https  | 48098
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/html_2/1/121/id=1051&pn=0.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.805 | https  | 48098
114.203.12.194 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /xlab/miniapp/2005/intf23253_1.js?t=1590651164841 HTTP/1.1" | 200 | 1243 | "https://servicewechat.com/wx25cb37caf7f44a41/0/page-frame.html" | "Mozilla/5.0 (Linux; Android 8.0.0; Mi Note 2 Build/OPR1.170623.032; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/78.0.3904.62 XWEB/2353 MMWEBSDK/200401 Mobile Safari/537.36 MMWEBID/747 MicroMessenger/7.0.14.1660(0x27000E37) Process/appbrand0 NetType/WIFI Language/zh_CN ABI/arm64 WeChat/arm64" | "-" | - | - | - | 172.16.239.5 | 0.998 | https  | 44620
172.16.238.228 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.486 | http  | 50761
110.80.141.159 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /pcedu/soft/wl/brower/0311/242394.html HTTP/1.1" | 302 | 270 | "-" | "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" | "-" | - | - | - | 172.16.239.5 | 0.015 | https  | 38277
113.96.140.246 | [28/May/2020:15:32:56 +0800] | pcedu.shell.com.cn | "GET /1346/13462969.html HTTP/1.1" | 200 | 11872 | "https://www.baidu.com/link?url=4cyuuadApwd6GD33cnmpoToHdfj9zL1lQ4mJSsE43ROVRFUJ43OyJziAxJK8KPBMkYefrsIZr3CObtpXvHRG2q&wd=&eqid=c14c6a850005b8910.6600065ecf6924" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 2.265 | https  | 49769
113.96.140.246 | [28/May/2020:15:32:56 +0800] | mobile.shell.com.cn | "GET /380/3806282.html HTTP/1.1" | 200 | 28100 | "-" | "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" | "-" | - | - | - | 172.16.239.5 | 0.181 | https  | 31225
172.16.238.146 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.705 | http  | 53098
113.96.140.246 | [28/May/2020:15:32:57 +0800] | gps.shell.com.cn | "GET /553/5535909.html HTTP/1.1" | 200 | 13072 | "-" | "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)" | "-" | - | - | - | 172.16.239.5 | 0.255 | https  | 49801
113.96.140.245 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/1655767.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363" | "-" | - | - | - | 172.16.239.5 | 0.304 | https  | 49684
172.16.239.59 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.905 | http  | 35067
110.80.141.158 | [28/May/2020:15:32:57 +0800] | pcedu.shell.com.cn | "GET /1239/12390087.html HTTP/1.1" | 200 | 14525 | "https://www.so.com/link?m=a92lOp6pOJvedmxUx9TBeQMp4n8OlvOlrcFo88bNTlcmQHQGUIPkwTIJIGjfOGAQByXA1tYZ5GHFd7DO0SYwzXg%2Bv5NwTD1QSYf2AYR4GfD9H3jXpaj9nv5Az8Yf%2BKkCohL2N9RZGto6%2B5coh%2BJvERvajHydHfQ4YTH59HnxjqG6lwQIelamKmPceG3YQCP0LvFYjgwLSCLuER8G%2FmtJwk1RLZ6rpaNq5NlFtXw%3D%3D" | "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.941 | https  | 36322
150.138.109.205 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /2014/0121/zt4201891.html HTTP/1.1" | 301 | 290 | "-" | "Mozilla/5.0 (Linux; Android 7.0; FRD-AL00 Build/HUAWEIFRD-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/53.0.2785.49 Mobile MQQBrowser/6.2 TBS/043602 Safari/537.36 MicroMessenger/6.5.16.1120 NetType/WIFI Language/zh_CN" | "-" | - | - | - | 172.16.239.5 | 0.387 | http  | 59501
172.16.239.174 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.515 | http  | 41052
172.16.246.51 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.968 | http  | 52389
172.16.239.175 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.311 | http  | 60251
113.96.140.245 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/359460-1.html" | "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.781 | https  | 49684
112.90.135.106 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/2352841.html" | "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.878 | https  | 38873
172.16.239.22 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/jd/ac/1712/10519871.html HTTP/1.0" | 200 | 27162 | "-" | "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" | "116.179.32.71, 172.16.237.58, 172.16.238.216, 172.16.238.216" | - | - | - | 172.16.239.5 | 0.214 | http  | 10246
172.16.237.200 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/software/excel/1403/4509076.html HTTP/1.0" | 200 | 8078 | "https://www.so.com/link?m=as5SxBFhuuzOANq9bvfy5AcMPf8wMcuiS45Td%2FDwcSHQ744TpvL%2BM3iHG7CDeRYSptnK6efNNN3zWfp3mg4hDRhiUD%2BY0ss8pxOZ9tP04Xo2NGclBG9JVuB1PCXzHD1wiVOf5OUT%2F6tSjGvszO4fhPn3CHTXY%2FFRVXMHlyWGjzwsiWkkNChoxzJwhN64cseN5XBAUP9aMrrfVqFj4GtMqmToYKYc1RFwY8bkGncb1oFj4qedamsvLKGfEy6r0cdRS8y%2BZZDOjngsAZKhYw1rkUztOwVCFrdA1BtgsRC%2FuMdwjnLD6IhB1qepAnk4qZrbQ" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "113.65.30.180" | - | - | - | 172.16.239.5 | 0.199 | http  | 43433
27.128.146.74 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/2380882.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.048 | https  | 47253
27.148.141.118 | [28/May/2020:15:32:57 +0800] | hk.shell.com.cn | "GET //network.shell.com.cn/1344/13441764.html HTTP/1.1" | 302 | 270 | "-" | "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; Media Center PC 4.0; SLCC1; .NET CLR 3.0.04320)" | "-" | - | - | - | 172.16.239.5 | 0.590 | http  | 40505
172.16.237.200 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/software/dnyw/1707/9686422.html HTTP/1.0" | 200 | 8420 | "-" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" | "171.113.81.245, 222.79.64.76" | - | - | - | 172.16.239.5 | 0.634 | http  | 43442
27.148.141.119 | [28/May/2020:15:32:57 +0800] | mobile.shell.com.cn | "GET /453/4537978.html HTTP/1.1" | 200 | 13638 | "-" | "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)" | "-" | - | - | - | 172.16.239.5 | 0.538 | https  | 62475
172.16.239.54 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.670 | http  | 55058
27.128.146.76 | [28/May/2020:15:32:57 +0800] | pcedu.shell.com.cn | "GET /1055/10554305.html HTTP/1.1" | 200 | 17286 | "https://www.baidu.com/link?url=QkXeaFixVvfQKX6LyyE29rGja5mQwx_L8_ehdHNnl_NO5bJ_TAlul2OpKpen2IJQOp-bXieCVidwkjP2o2HGjK&wd=&eqid=f67b455a0003bf310.8340045ecf6926" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 2.265 | https  | 47929
27.128.146.76 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /factory_pic/883/8830500_pic.html?imgsrc=//img0.shell.com.cn/bashTest/1702/14/8830500_1.jpg&channel=309 HTTP/1.1" | 200 | 9638 | "-" | "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)" | "-" | - | - | - | 172.16.239.5 | 0.206 | https  | 47997
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/1825853.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.6788.400 QQBrowser/10.3.2776.400" | "-" | - | - | - | 172.16.239.5 | 0.220 | https  | 38873
27.148.141.117 | [28/May/2020:15:32:58 +0800] | tv.shell.com.cn | "GET /1283/12830092.html HTTP/1.1" | 200 | 32856 | "https://product.shell.com.cn/lcd_tv/sony/1156127.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.635 | https  | 37993
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/1825853.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.6788.400 QQBrowser/10.3.2776.400" | "-" | - | - | - | 172.16.239.5 | 0.058 | https  | 39865
124.236.26.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /projector/437/4375785.html HTTP/1.1" | 200 | 14556 | "-" | "Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://aspiegel.com/petalbot)" | "-" | - | - | - | 172.16.239.5 | 0.119 | https  | 62999
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/2684489.html" | "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3947.100 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.286 | https  | 39865
172.16.238.61 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /global/navibar_html/index.html HTTP/1.1" | 200 | 13418 | "-" | "NodeJs" | "-" | - | - | - | 172.16.239.5 | 0.182 | http  | 50513
172.16.236.29 | [28/May/2020:15:32:58 +0800] | picture.shell.com.cn | "GET /diy/1112/2619852_6.html HTTP/1.0" | 200 | 4446 | "-" | "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)" | "46.229.168.143, 172.16.237.171, 172.16.237.171, 172.16.239.12" | - | - | - | 172.16.239.5 | 0.321 | http  | 33494
172.16.239.59 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.683 | http  | 35075
172.16.239.158 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.898 | http  | 58592
文章标签：
日志服务
Go
Shell
容器
应用服务中间件
Kubernetes
nginx
关键词：
Nginx日志
日志服务nginx
Shell脚本
日志服务脚本
日志服务统计
相关实践学习
日志服务之使用Nginx模式采集日志
本文介绍如何通过日志服务控制台创建Nginx模式的Logtail配置快速采集Nginx日志并进行多维度分析。
运维包工头
目录
相关文章
GG2020gg
|
19天前
|
运维 监控 应用服务中间件
LNMP详解（十五）——Nginx日志分析实战
LNMP详解（十五）——Nginx日志分析实战
GG2020gg
39 0
魏红斌
|
19天前
|
弹性计算 应用服务中间件 Shell
切割 Nginx 日志文件
【4月更文挑战第28天】
魏红斌
25 0
玉米侠爱吃玉米
|
3天前
|
缓存 应用服务中间件 网络安全
nginx 日志，压缩，https功能介绍
nginx 日志，压缩，https功能介绍
玉米侠爱吃玉米
10 0
魏红斌
|
19天前
|
弹性计算 应用服务中间件 Shell
一键编译安装Nginx脚本
【4月更文挑战第30天】
魏红斌
32 1
魏红斌
|
19天前
|
弹性计算 应用服务中间件 Shell
切割Nginx 日志文件
【4月更文挑战第29天】
魏红斌
24 1
魏红斌
|
19天前
|
弹性计算 应用服务中间件 Shell
编写nginx 启动脚本
【4月更文挑战第29天】
魏红斌
13 1
魏红斌
|
19天前
|
存储 弹性计算 运维
统计/var/log 有多少个文件
【4月更文挑战第29天】
魏红斌
25 1
牵着猫散步的鼠鼠
|
19天前
|
存储 消息中间件 Java
Java多线程实战-异步操作日志记录解决方案(AOP+注解+多线程)
Java多线程实战-异步操作日志记录解决方案(AOP+注解+多线程)
牵着猫散步的鼠鼠
40 1
jianz123
|
19天前
|
存储 应用服务中间件 nginx
nginx日志定时切割 按年月日
nginx日志定时切割 按年月日
jianz123
22 0
jianz123
|
19天前
|
网络协议 应用服务中间件 Linux
centos7 Nginx Log日志统计分析 常用命令
centos7 Nginx Log日志统计分析 常用命令
jianz123
160 2

热门文章

最新文章

  • 1
    Redis内核基于时间点的备份恢复和基于AOF日志的增量同步机制设计
  • 2
    用Windows Live Writer发布日志到BlogBus
  • 3
    log file sync总结
  • 4
    抑制告警风暴—SLS告警智能合并发布
  • 5
    SQL Server 2014 日志传送部署（4）：SQL Server Management Studio部署日志传送
  • 6
    秋式网站日志分析器[IISLogViewer] V3版本发布
  • 7
    安卓开发-See the log file\.metadata\.log.
  • 8
    web日志分析工具
  • 9
    新浪明星日志推荐系统——爬虫爬取数据（1）
  • 10
    步步为营VS 2008 + .NET 3.5(11) - DLINQ(LINQ to SQL)之大数据量分页、延迟执行和日志记录
  • 1
    【Linux】浅谈shell命令以及运行原理
    84
  • 2
    Linux--自定义shell
    32
  • 3
    通用Shell脚本执行Spring Boot项目Jar包
    43
  • 4
    Java调用shell脚本实现数据库备份功能
    43
  • 5
    什么是计算机软件领域的 shell app
    24
  • 6
    利用Shell脚本自动化员工电脑监管软件的部署流程
    166
  • 7
    供应链投毒预警 | 恶意NPM包利用Windows反向shell后门攻击开发者
    38
  • 8
    【Linux】shell命令以及运行原理
    54
  • 9
    上传Joomal网站的Shell
    23
  • 10
    使用shell脚本给日志文件瘦身
    48

    • 相关课程

    更多
  • 日志服务SLS实现云产品可观测
  • 日志服务 SLS 可观测数据分析平台介绍
  • 大数据知识图谱系列—基于ELK+Flink日志全观测最佳实践
  • Nginx企业级Web服务实战
  • Linux Web服务器Nginx搭建与配置
  • 场景实践-基于阿里云Quick BI 对MOOC网站日志分析

    • 相关电子书

    更多
  • 大规模日志数据平台架构面临的问题与挑战
  • CentOS Nginx PHP JAVA 多语言镜像使用手
  • CentOS Nginx PHP JAVA多语言镜像使用手册

    • 相关实验场景

    更多
  • 如何快速创建插件agent
  • 使用CloudLens管理SLS日志采集
  • 使用CloudLens采集PolarDB日志并进行审计分析
  • 使用CloudLens采集RDS日志并进行审计分析
  • 基于SLS快速实现业务可观测
  • 使用日志服务SLS进行OSS可观测分析
    • 下一篇
    2024年阿里云免费云服务器及学生云服务器申请教程参考