AI 助理
备案控制台
开发者社区 开发与运维 文章 正文

3天玩转shell--12.实战编写nginx日志统计脚本【终结篇】

2023-02-16 494
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
本文涉及的产品
日志服务 SLS，月写入数据量 50GB 1个月
简介： 本节课是shell教程系列的最后一篇文章，希望这12篇文章对shell的学习者有帮助。往后有空会发表一些关于golang编程的实战系列文章、或者是k8s相关的文章。

一、功能介绍：

【1】统计各种状态码的数量
【2】统计访问最多的referer
【3】统计访问最高的uri
【4】统计访问最多的 ip和ua
【5】统计每分钟的请求数、流量、请求时间、状态码等

二、通过代码进行讲解

本节是通过编写一段统计访问日志，统计日志的状态码、浏览器类型、URL、流量、状态码分布等信息，这对于网站的维护人员，能很好的掌握网站的运行情况。将附件保存12.example.txt 
#!/bin/bash
#12.sh v1
PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"
export LANG=zh_CN.UTF-8
export PATH

#统计状态码并排序
GroupByStatusCode(){
  file=$1
  cat $file |awk -F "|" '{Arry[$5]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计IP并排序
GroupByRemoteAddress(){
  file=$1
  cat $file |awk -F "|" '{Arry[$1]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计referer并排序
GroupByReferer(){
  file=$1
  cat $file |awk -F "|" '{Arry[$7]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计浏览器并排序
GroupByUserAgent(){
  file=$1
  cat $file |awk -F "|" '{Arry[$8]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计URi不含参数，并排序
GroupByUri(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," |?");Arry[aa[3]]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计URL并排序
GroupByUrl(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," ");Arry[aa[2]]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计请求方法并排序
GroupByMethod(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," |?");Arry[aa[2]]+=1;total++}END{for(s in Arry){printf "%s\t%d\t%.4f\n",s,Arry[s],Arry[s]/total}}'|sort -nr -k 2
}

#统计请求时间并排序
GroupByRequestTime(){
  file=$1
  #cat $file |awk -F "|" '{if($14<1){reqs["<1"]++}else if($14>1 && $14<5){reqs["1-5"]++}else if($14>5 && $14<10){reqs["5-10"]++}else{reqs[">10"]++};total++}END{printf "<1\t%.4f\t%d\n1-5\t%.4f\t%d\n5-10\t%.4f\t%d\n>10\t%.4f\t%d\n",reqs["<
1"]/total,reqs["<1"],reqs["1-5"]/total,reqs["1-5"],reqs["5-10"]/total,reqs["5-10"],reqs[">10"]/total,reqs[">10"]}'
  cat $file |awk -F "|" '{
  if($14<1){reqs["<1"]++}
  else if($14>1 && $14<5) {reqs["1-5"]++}
  else if($14>5 && $14<10){reqs["5-10"]++}
  else {reqs[">10"]++};
  total++
  }END{
   printf "<1\t%.4f\t%d\n1-5\t%.4f\t%d\n5-10\t%.4f\t%d\n>10\t%.4f\t%d\n",reqs["<1"]/total,reqs["<1"],reqs["1-5"]/total,reqs["1-5"],reqs["5-10"]/total,reqs["5-10"],reqs[">10"]/total,reqs[">10"]
  }'
}

#统计每分钟请求的数量
TrendTimesCount(){
 file=$1
 cat $file|awk -F "|" '{tms[substr($2,15,5)]+=1}END{for(t in tms)print t" "tms[t]}'
}

#统计每分钟的流量
TrendTimesNetFlow(){
 file=$1
 cat $file|awk -F "|" '{tms[substr($2,15,5)]+=$6}END{for(t in tms)printf "%s\t%.4fMB\n",t,tms[t]/1024/1024}'
}

#统计每分钟的数量、流量、状态码分布、平均请求时间
TrendTimesAll(){
 file=$1
 #cat $file|awk -F "|" 'BEGIN{printf "时间\t数量\t流量[MB]\t请求时间\t20x\t30x\t40x\t50x\t60x\n"}{tm=substr($2,15,5);tms[tm]+=$6;cnt[tm]+=1;reqt[tm]+=$14;if($5 ~ /20/){sc20x[tm]++}else if($5 ~ /30/){sc30x[tm]++}else if($5 ~ /40/){sc40x[
tm]++}else if($5 ~ /50/){sc50x[tm]++}else{sc60x[tm]++};}END{for(t in tms)printf "%s\t%d\t%.4f\t\t%.4f\t\t%d\t%d\t%d\t%d\t%d\n",t,cnt[t],tms[t]/1024/1024,reqt[t]/cnt[t],sc20x[t],sc30x[t],sc40x[t],sc50x[t],sc60x[t]}'
 cat $file|awk -F "|" 'BEGIN{
      printf "时间\t数量\t流量[MB]\t请求时间\t20x\t30x\t40x\t50x\t60x\n"
  }{
      tm=substr($2,15,5);
      tms[tm]+=$6;
      cnt[tm]+=1;
      reqt[tm]+=$14;
      if($5 ~ /20/) {sc20x[tm]++}
      else if($5 ~ /30/) {sc30x[tm]++}
      else if($5 ~ /40/) {sc40x[tm]++}
      else if($5 ~ /50/) {sc50x[tm]++}
      else {sc60x[tm]++};
   }END{
   for(t in tms)
        printf "%s\t%d\t%.4f\t\t%.4f\t\t%d\t%d\t%d\t%d\t%d\n",t,cnt[t],tms[t]/1024/1024,reqt[t]/cnt[t],sc20x[t],sc30x[t],sc40x[t],sc50x[t],sc60x[t]
   }'
}

#帮助函数
Help(){
 echo "Help:"
 echo "\$1 args \$2 logfile"
 echo "s :groupby statuscode"
 echo "h :groupby remoteaddress"
 echo "e :groupby referer"
 echo "a :groupby useragent"
 echo "i :groupby uri"
 echo "r :groupby url"
 echo "t :groupby requestTime"
 echo "m :groupby method"
 echo "tc :trend count request by minute"
 echo "tn :trend count netflow by minute"
 echo "ta :trend count request,netflow,requesTime,statuscode by minute"
 echo "UseAge Example:"
 echo "$0 s access.log"
 echo "$0 h access.log"
 echo "$0 i access.log"
 echo "$0 tn access.log"
 echo "$0 ta access.log"
}

if [ $# -ne 2 ];then
   Help
   exit 1
else
   if [ ! -s $2 ];then
      echo "$2 file does not exist"
      exit 1
   fi
fi

case $1 in
    s|statuscode)
      GroupByStatusCode "$2";;
    h|remoteaddress)
      GroupByRemoteAddress $2;;
    e|referer)
      GroupByReferer $2;;
    a|useragent)
      GroupByUserAgent $2;;
    i|uri)
      GroupByUri $2;;
    r|url)
      GroupByUrl $2;;
    t|requesttime)
      GroupByRequestTime $2;;
    m|method)
      GroupByMethod $2;;
    tc|trendocunt)
      TrendTimesCount $2;;
    tn|trendnet)
      TrendTimesNetFlow $2;;
    ta|trendall)
      TrendTimesAll $2;;
    *)
      Help;;
esac

三.脚本执行结果：

1.打印帮助信息：
help.png
2.统计状态码和IP并排序：
code.png
3.统计请求方法、统计响应时间分布、统计uri
m.png
4.统计每分钟的数量、每分钟的流量、每分钟的状态码走势
t.png

四.附件

113.96.140.246 | [28/May/2020:15:32:55 +0800] | www.shell.com.cn | "GET /pcedu/ios/1106/2448633_1.html HTTP/1.1" | 200 | 8289 | "-" | "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" | "-" | - | - | - | 172.16.239.5 | 0.234 | https  | 47585
172.16.239.156 | [28/May/2020:15:32:55 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 1.232 | http  | 35949
172.16.239.158 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.982 | http  | 58399
172.16.246.51 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.545 | http  | 52384
113.96.140.246 | [28/May/2020:15:32:56 +0800] | office.shell.com.cn | "GET /1282/12825949_3.html HTTP/1.1" | 200 | 31173 | "https://office.shell.com.cn/1282/12825949_2.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" | "-" | - | - | - | 172.16.239.5 | 0.645 | https  | 47791
58.55.123.88 | [28/May/2020:15:32:56 +0800] | www1.shell.com.cn | "GET /2015/dl/client/images/btn_client.png HTTP/1.1" | 304 | 0 | "https://js.shell.com/bashTest/2015/dl/css/client.css" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.358 | https  | 26877
172.16.239.174 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.435 | http  | 40954
172.16.238.72 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /pcclub/itbbsindex/1702/intf17885.html HTTP/1.1" | 200 | 1766 | "-" | "NodeJs" | "-" | - | - | - | 172.16.239.5 | 0.519 | http  | 55833
172.16.240.137 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /product/fz/2015/1511/intf11701.html HTTP/1.1" | 200 | 0 | "-" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.905 | http  | 46830
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/409990.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.715 | https  | 47574
229.137.244.63 | [28/May/2020:15:32:56 +0800] | www1.shell.com.cn | "GET /zt/gz20140905/404/css/404.css HTTP/1.0" | 200 | 4429 | "-" | "-" | "-" | - | - | - | 172.16.239.5 | 0.926 | http  | 43571
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/385364.html" | "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36 2345Explorer/6.1.0.7934" | "-" | - | - | - | 172.16.239.5 | 0.445 | https  | 47574
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/385364.html" | "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36 2345Explorer/6.1.0.7934" | "-" | - | - | - | 172.16.239.5 | 0.506 | https  | 48098
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/63718.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.68 Safari/537.36 Core/1.53.4033.400 QQBrowser/9.6.12624.400" | "-" | - | - | - | 172.16.239.5 | 0.371 | https  | 48096
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/63718.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.68 Safari/537.36 Core/1.53.4033.400 QQBrowser/9.6.12624.400" | "-" | - | - | - | 172.16.239.5 | 0.341 | https  | 48098
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/html_2/1/121/id=1051&pn=0.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.805 | https  | 48098
114.203.12.194 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /xlab/miniapp/2005/intf23253_1.js?t=1590651164841 HTTP/1.1" | 200 | 1243 | "https://servicewechat.com/wx25cb37caf7f44a41/0/page-frame.html" | "Mozilla/5.0 (Linux; Android 8.0.0; Mi Note 2 Build/OPR1.170623.032; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/78.0.3904.62 XWEB/2353 MMWEBSDK/200401 Mobile Safari/537.36 MMWEBID/747 MicroMessenger/7.0.14.1660(0x27000E37) Process/appbrand0 NetType/WIFI Language/zh_CN ABI/arm64 WeChat/arm64" | "-" | - | - | - | 172.16.239.5 | 0.998 | https  | 44620
172.16.238.228 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.486 | http  | 50761
110.80.141.159 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /pcedu/soft/wl/brower/0311/242394.html HTTP/1.1" | 302 | 270 | "-" | "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" | "-" | - | - | - | 172.16.239.5 | 0.015 | https  | 38277
113.96.140.246 | [28/May/2020:15:32:56 +0800] | pcedu.shell.com.cn | "GET /1346/13462969.html HTTP/1.1" | 200 | 11872 | "https://www.baidu.com/link?url=4cyuuadApwd6GD33cnmpoToHdfj9zL1lQ4mJSsE43ROVRFUJ43OyJziAxJK8KPBMkYefrsIZr3CObtpXvHRG2q&wd=&eqid=c14c6a850005b8910.6600065ecf6924" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 2.265 | https  | 49769
113.96.140.246 | [28/May/2020:15:32:56 +0800] | mobile.shell.com.cn | "GET /380/3806282.html HTTP/1.1" | 200 | 28100 | "-" | "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" | "-" | - | - | - | 172.16.239.5 | 0.181 | https  | 31225
172.16.238.146 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.705 | http  | 53098
113.96.140.246 | [28/May/2020:15:32:57 +0800] | gps.shell.com.cn | "GET /553/5535909.html HTTP/1.1" | 200 | 13072 | "-" | "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)" | "-" | - | - | - | 172.16.239.5 | 0.255 | https  | 49801
113.96.140.245 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/1655767.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363" | "-" | - | - | - | 172.16.239.5 | 0.304 | https  | 49684
172.16.239.59 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.905 | http  | 35067
110.80.141.158 | [28/May/2020:15:32:57 +0800] | pcedu.shell.com.cn | "GET /1239/12390087.html HTTP/1.1" | 200 | 14525 | "https://www.so.com/link?m=a92lOp6pOJvedmxUx9TBeQMp4n8OlvOlrcFo88bNTlcmQHQGUIPkwTIJIGjfOGAQByXA1tYZ5GHFd7DO0SYwzXg%2Bv5NwTD1QSYf2AYR4GfD9H3jXpaj9nv5Az8Yf%2BKkCohL2N9RZGto6%2B5coh%2BJvERvajHydHfQ4YTH59HnxjqG6lwQIelamKmPceG3YQCP0LvFYjgwLSCLuER8G%2FmtJwk1RLZ6rpaNq5NlFtXw%3D%3D" | "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.941 | https  | 36322
150.138.109.205 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /2014/0121/zt4201891.html HTTP/1.1" | 301 | 290 | "-" | "Mozilla/5.0 (Linux; Android 7.0; FRD-AL00 Build/HUAWEIFRD-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/53.0.2785.49 Mobile MQQBrowser/6.2 TBS/043602 Safari/537.36 MicroMessenger/6.5.16.1120 NetType/WIFI Language/zh_CN" | "-" | - | - | - | 172.16.239.5 | 0.387 | http  | 59501
172.16.239.174 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.515 | http  | 41052
172.16.246.51 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.968 | http  | 52389
172.16.239.175 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.311 | http  | 60251
113.96.140.245 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/359460-1.html" | "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.781 | https  | 49684
112.90.135.106 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/2352841.html" | "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.878 | https  | 38873
172.16.239.22 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/jd/ac/1712/10519871.html HTTP/1.0" | 200 | 27162 | "-" | "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" | "116.179.32.71, 172.16.237.58, 172.16.238.216, 172.16.238.216" | - | - | - | 172.16.239.5 | 0.214 | http  | 10246
172.16.237.200 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/software/excel/1403/4509076.html HTTP/1.0" | 200 | 8078 | "https://www.so.com/link?m=as5SxBFhuuzOANq9bvfy5AcMPf8wMcuiS45Td%2FDwcSHQ744TpvL%2BM3iHG7CDeRYSptnK6efNNN3zWfp3mg4hDRhiUD%2BY0ss8pxOZ9tP04Xo2NGclBG9JVuB1PCXzHD1wiVOf5OUT%2F6tSjGvszO4fhPn3CHTXY%2FFRVXMHlyWGjzwsiWkkNChoxzJwhN64cseN5XBAUP9aMrrfVqFj4GtMqmToYKYc1RFwY8bkGncb1oFj4qedamsvLKGfEy6r0cdRS8y%2BZZDOjngsAZKhYw1rkUztOwVCFrdA1BtgsRC%2FuMdwjnLD6IhB1qepAnk4qZrbQ" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "113.65.30.180" | - | - | - | 172.16.239.5 | 0.199 | http  | 43433
27.128.146.74 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/2380882.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.048 | https  | 47253
27.148.141.118 | [28/May/2020:15:32:57 +0800] | hk.shell.com.cn | "GET //network.shell.com.cn/1344/13441764.html HTTP/1.1" | 302 | 270 | "-" | "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; Media Center PC 4.0; SLCC1; .NET CLR 3.0.04320)" | "-" | - | - | - | 172.16.239.5 | 0.590 | http  | 40505
172.16.237.200 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/software/dnyw/1707/9686422.html HTTP/1.0" | 200 | 8420 | "-" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" | "171.113.81.245, 222.79.64.76" | - | - | - | 172.16.239.5 | 0.634 | http  | 43442
27.148.141.119 | [28/May/2020:15:32:57 +0800] | mobile.shell.com.cn | "GET /453/4537978.html HTTP/1.1" | 200 | 13638 | "-" | "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)" | "-" | - | - | - | 172.16.239.5 | 0.538 | https  | 62475
172.16.239.54 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.670 | http  | 55058
27.128.146.76 | [28/May/2020:15:32:57 +0800] | pcedu.shell.com.cn | "GET /1055/10554305.html HTTP/1.1" | 200 | 17286 | "https://www.baidu.com/link?url=QkXeaFixVvfQKX6LyyE29rGja5mQwx_L8_ehdHNnl_NO5bJ_TAlul2OpKpen2IJQOp-bXieCVidwkjP2o2HGjK&wd=&eqid=f67b455a0003bf310.8340045ecf6926" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 2.265 | https  | 47929
27.128.146.76 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /factory_pic/883/8830500_pic.html?imgsrc=//img0.shell.com.cn/bashTest/1702/14/8830500_1.jpg&channel=309 HTTP/1.1" | 200 | 9638 | "-" | "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)" | "-" | - | - | - | 172.16.239.5 | 0.206 | https  | 47997
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/1825853.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.6788.400 QQBrowser/10.3.2776.400" | "-" | - | - | - | 172.16.239.5 | 0.220 | https  | 38873
27.148.141.117 | [28/May/2020:15:32:58 +0800] | tv.shell.com.cn | "GET /1283/12830092.html HTTP/1.1" | 200 | 32856 | "https://product.shell.com.cn/lcd_tv/sony/1156127.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.635 | https  | 37993
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/1825853.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.6788.400 QQBrowser/10.3.2776.400" | "-" | - | - | - | 172.16.239.5 | 0.058 | https  | 39865
124.236.26.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /projector/437/4375785.html HTTP/1.1" | 200 | 14556 | "-" | "Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://aspiegel.com/petalbot)" | "-" | - | - | - | 172.16.239.5 | 0.119 | https  | 62999
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/2684489.html" | "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3947.100 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.286 | https  | 39865
172.16.238.61 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /global/navibar_html/index.html HTTP/1.1" | 200 | 13418 | "-" | "NodeJs" | "-" | - | - | - | 172.16.239.5 | 0.182 | http  | 50513
172.16.236.29 | [28/May/2020:15:32:58 +0800] | picture.shell.com.cn | "GET /diy/1112/2619852_6.html HTTP/1.0" | 200 | 4446 | "-" | "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)" | "46.229.168.143, 172.16.237.171, 172.16.237.171, 172.16.239.12" | - | - | - | 172.16.239.5 | 0.321 | http  | 33494
172.16.239.59 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.683 | http  | 35075
172.16.239.158 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.898 | http  | 58592
文章标签：
日志服务
Go
Shell
容器
应用服务中间件
Kubernetes
nginx
关键词：
Nginx日志
Nginx实战
日志服务统计
实战日志服务
Shell脚本
相关实践学习
日志服务之使用Nginx模式采集日志
本文介绍如何通过日志服务控制台创建Nginx模式的Logtail配置快速采集Nginx日志并进行多维度分析。
运维包工头
目录
相关文章
瓴羊Dataphin
|
1月前
|
存储 运维 监控
API明细日志及运维统计日志全面提升API可运维性
在数字化转型的大潮中，数据已成为企业最宝贵的资产之一。而数据服务API可快速为数据应用提供数据接口。面对越来越多的API以及越来越多的应用调用，如何快速查看API的服务情况、异常情况及影响范围，以及查看API的调用详情，进行API的性能优化、错误排查变得越来越重要，本文将介绍如何配置和开通API运维统计及明细日志，以及如何查看日志进行介绍。
瓴羊Dataphin
75 0
LKIDTI数据
|
2月前
|
Shell
一个用于添加/删除定时任务的shell脚本
一个用于添加/删除定时任务的shell脚本
LKIDTI数据
114 1
liuyunshengsir
|
19天前
|
监控 应用服务中间件 定位技术
要统计Nginx的客户端IP，可以通过分析Nginx的访问日志文件来实现
要统计Nginx的客户端IP，可以通过分析Nginx的访问日志文件来实现
liuyunshengsir
47 3
冷冻工厂
|
1月前
|
Shell Linux 测试技术
6种方法打造出色的Shell脚本
6种方法打造出色的Shell脚本
冷冻工厂
66 2
6种方法打造出色的Shell脚本
蓝易云
|
1月前
|
XML JSON 监控
Shell脚本要点和难点以及具体应用和优缺点介绍
Shell脚本在系统管理和自动化任务中扮演着重要角色。尽管存在调试困难、可读性差等问题，但其简洁高效、易于学习和强大的功能使其在许多场景中不可或缺。通过掌握Shell脚本的基本语法、常用命令和函数，并了解其优缺点，开发者可以编写出高效的脚本来完成各种任务，提高工作效率。希望本文能为您在Shell脚本编写和应用中提供有价值的参考和指导。
蓝易云
61 1
eisc
|
1月前
|
Ubuntu Shell 开发工具
ubuntu/debian shell 脚本自动配置 gitea git 仓库
这是一个自动配置 Gitea Git 仓库的 Shell 脚本，支持 Ubuntu 20+ 和 Debian 12+ 系统。脚本会创建必要的目录、下载并安装 Gitea，创建 Gitea 用户和服务，确保 Gitea 在系统启动时自动运行。用户可以选择从官方或小绿叶技术博客下载安装包。
eisc
51 2
eisc
|
2月前
|
监控 网络协议 Shell
ip和ip网段攻击拦截系统-绿叶结界防火墙系统shell脚本
这是一个名为“小绿叶技术博客扫段攻击拦截系统”的Bash脚本，用于监控和拦截TCP攻击。通过抓取网络数据包监控可疑IP，并利用iptables和firewalld防火墙规则对这些IP进行拦截。同时，该系统能够查询数据库中的白名单，确保合法IP不受影响。此外，它还具备日志记录功能，以便于后续分析和审计。
eisc
58 6
我不是游客20240119
|
1月前
|
运维 监控 Shell
深入理解Linux系统下的Shell脚本编程
【10月更文挑战第24天】本文将深入浅出地介绍Linux系统中Shell脚本的基础知识和实用技巧，帮助读者从零开始学习编写Shell脚本。通过本文的学习，你将能够掌握Shell脚本的基本语法、变量使用、流程控制以及函数定义等核心概念，并学会如何将这些知识应用于实际问题解决中。文章还将展示几个实用的Shell脚本例子，以加深对知识点的理解和应用。无论你是运维人员还是软件开发者，这篇文章都将为你提供强大的Linux自动化工具。
我不是游客20240119
68 1
大成无敌
|
应用服务中间件 Apache nginx
nginx自己写日志切割脚本
转自本人的博客：http://blog.teier.cn
大成无敌
1499 0
技术小阿哥
|
监控 应用服务中间件 nginx
每日一shell(八)nginx日志切割
技术小阿哥
871 0

热门文章

最新文章

  • 1
    Nginx报错403 forbidden几个解决方法
  • 2
    如何优雅打印nginx header和body
  • 3
    nginx hello world模块编译运行的问题
  • 4
    Nginx防盗链、Nginx访问控制、Nginx解析php相关配置、Nginx代理
  • 5
    Nginx的Upload上传模块
  • 6
    Nginx动静分离经典案例配置
  • 7
    nginx反向代理与微信测试全攻略
  • 8
    debian下LAMP+nginx代理+awstats+cacti+nagios(四)
  • 9
    NGINX + TOMCAT7 + MEMCACHED 实现SESSION 共享
  • 10
    nginx之yum源
  • 1
    mongoDB查看数据的插入日志
    467
  • 2
    修改开发板内核启动日志输出级别
    118
  • 3
    【MongoDB 专栏】MongoDB 的日志管理与分析
    180
  • 4
    编程日志01:个人网站更新用户头像
    53
  • 5
    编程日记02：个人站优化数据库和日志
    53
  • 6
    oracle11g SAP测试机归档日志暴增排查（二）
    338
  • 7
    oracle11g SAP测试机归档日志暴增排查（一）
    86
  • 8
    log4j:WARN Please initialize the log4j system prop
    64
  • 9
    LabVIEW崩溃后所产生的错误日志文件的位置
    203
  • 10
    关于日志的清理
    69

    • 相关课程

    更多
  • 日志服务SLS实现云产品可观测
  • 日志服务 SLS 可观测数据分析平台介绍
  • 大数据知识图谱系列—基于ELK+Flink日志全观测最佳实践
  • Nginx企业级Web服务实战
  • Linux Web服务器Nginx搭建与配置
  • 场景实践-基于阿里云Quick BI 对MOOC网站日志分析

    • 相关电子书

    更多
  • 2019大数据技术公开课第五季—日志数据如何同步到MaxCompute之实践讲解
  • CentOS Nginx PHP JAVA 多语言镜像使用手
  • CentOS Nginx PHP JAVA多语言镜像使用手册

    • 相关实验场景

    更多
  • 如何快速创建插件agent
  • 使用CloudLens管理SLS日志采集
  • 使用CloudLens采集PolarDB日志并进行审计分析
  • 使用CloudLens采集RDS日志并进行审计分析
  • 使用日志服务SLS进行OSS可观测分析
  • Arthas中help、history、keymap、options基础命令应用
    • 下一篇
    DataWorks