AI 助理
备案控制台
开发者社区 开发与运维 文章 正文

3天玩转shell--12.实战编写nginx日志统计脚本【终结篇】

2023-02-16 490
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
本文涉及的产品
日志服务 SLS，月写入数据量 50GB 1个月
简介： 本节课是shell教程系列的最后一篇文章，希望这12篇文章对shell的学习者有帮助。往后有空会发表一些关于golang编程的实战系列文章、或者是k8s相关的文章。

一、功能介绍：

【1】统计各种状态码的数量
【2】统计访问最多的referer
【3】统计访问最高的uri
【4】统计访问最多的 ip和ua
【5】统计每分钟的请求数、流量、请求时间、状态码等

二、通过代码进行讲解

本节是通过编写一段统计访问日志，统计日志的状态码、浏览器类型、URL、流量、状态码分布等信息，这对于网站的维护人员，能很好的掌握网站的运行情况。将附件保存12.example.txt 
#!/bin/bash
#12.sh v1
PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"
export LANG=zh_CN.UTF-8
export PATH

#统计状态码并排序
GroupByStatusCode(){
  file=$1
  cat $file |awk -F "|" '{Arry[$5]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计IP并排序
GroupByRemoteAddress(){
  file=$1
  cat $file |awk -F "|" '{Arry[$1]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计referer并排序
GroupByReferer(){
  file=$1
  cat $file |awk -F "|" '{Arry[$7]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计浏览器并排序
GroupByUserAgent(){
  file=$1
  cat $file |awk -F "|" '{Arry[$8]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计URi不含参数，并排序
GroupByUri(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," |?");Arry[aa[3]]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计URL并排序
GroupByUrl(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," ");Arry[aa[2]]+=1;total++}END{for(s in Arry){printf "%d\t%.4f\t%s\n",Arry[s],Arry[s]/total,s}}'|sort -nr -k 1
}

#统计请求方法并排序
GroupByMethod(){
  file=$1
  cat $file |awk -F "|" '{split($4,aa," |?");Arry[aa[2]]+=1;total++}END{for(s in Arry){printf "%s\t%d\t%.4f\n",s,Arry[s],Arry[s]/total}}'|sort -nr -k 2
}

#统计请求时间并排序
GroupByRequestTime(){
  file=$1
  #cat $file |awk -F "|" '{if($14<1){reqs["<1"]++}else if($14>1 && $14<5){reqs["1-5"]++}else if($14>5 && $14<10){reqs["5-10"]++}else{reqs[">10"]++};total++}END{printf "<1\t%.4f\t%d\n1-5\t%.4f\t%d\n5-10\t%.4f\t%d\n>10\t%.4f\t%d\n",reqs["<
1"]/total,reqs["<1"],reqs["1-5"]/total,reqs["1-5"],reqs["5-10"]/total,reqs["5-10"],reqs[">10"]/total,reqs[">10"]}'
  cat $file |awk -F "|" '{
  if($14<1){reqs["<1"]++}
  else if($14>1 && $14<5) {reqs["1-5"]++}
  else if($14>5 && $14<10){reqs["5-10"]++}
  else {reqs[">10"]++};
  total++
  }END{
   printf "<1\t%.4f\t%d\n1-5\t%.4f\t%d\n5-10\t%.4f\t%d\n>10\t%.4f\t%d\n",reqs["<1"]/total,reqs["<1"],reqs["1-5"]/total,reqs["1-5"],reqs["5-10"]/total,reqs["5-10"],reqs[">10"]/total,reqs[">10"]
  }'
}

#统计每分钟请求的数量
TrendTimesCount(){
 file=$1
 cat $file|awk -F "|" '{tms[substr($2,15,5)]+=1}END{for(t in tms)print t" "tms[t]}'
}

#统计每分钟的流量
TrendTimesNetFlow(){
 file=$1
 cat $file|awk -F "|" '{tms[substr($2,15,5)]+=$6}END{for(t in tms)printf "%s\t%.4fMB\n",t,tms[t]/1024/1024}'
}

#统计每分钟的数量、流量、状态码分布、平均请求时间
TrendTimesAll(){
 file=$1
 #cat $file|awk -F "|" 'BEGIN{printf "时间\t数量\t流量[MB]\t请求时间\t20x\t30x\t40x\t50x\t60x\n"}{tm=substr($2,15,5);tms[tm]+=$6;cnt[tm]+=1;reqt[tm]+=$14;if($5 ~ /20/){sc20x[tm]++}else if($5 ~ /30/){sc30x[tm]++}else if($5 ~ /40/){sc40x[
tm]++}else if($5 ~ /50/){sc50x[tm]++}else{sc60x[tm]++};}END{for(t in tms)printf "%s\t%d\t%.4f\t\t%.4f\t\t%d\t%d\t%d\t%d\t%d\n",t,cnt[t],tms[t]/1024/1024,reqt[t]/cnt[t],sc20x[t],sc30x[t],sc40x[t],sc50x[t],sc60x[t]}'
 cat $file|awk -F "|" 'BEGIN{
      printf "时间\t数量\t流量[MB]\t请求时间\t20x\t30x\t40x\t50x\t60x\n"
  }{
      tm=substr($2,15,5);
      tms[tm]+=$6;
      cnt[tm]+=1;
      reqt[tm]+=$14;
      if($5 ~ /20/) {sc20x[tm]++}
      else if($5 ~ /30/) {sc30x[tm]++}
      else if($5 ~ /40/) {sc40x[tm]++}
      else if($5 ~ /50/) {sc50x[tm]++}
      else {sc60x[tm]++};
   }END{
   for(t in tms)
        printf "%s\t%d\t%.4f\t\t%.4f\t\t%d\t%d\t%d\t%d\t%d\n",t,cnt[t],tms[t]/1024/1024,reqt[t]/cnt[t],sc20x[t],sc30x[t],sc40x[t],sc50x[t],sc60x[t]
   }'
}

#帮助函数
Help(){
 echo "Help:"
 echo "\$1 args \$2 logfile"
 echo "s :groupby statuscode"
 echo "h :groupby remoteaddress"
 echo "e :groupby referer"
 echo "a :groupby useragent"
 echo "i :groupby uri"
 echo "r :groupby url"
 echo "t :groupby requestTime"
 echo "m :groupby method"
 echo "tc :trend count request by minute"
 echo "tn :trend count netflow by minute"
 echo "ta :trend count request,netflow,requesTime,statuscode by minute"
 echo "UseAge Example:"
 echo "$0 s access.log"
 echo "$0 h access.log"
 echo "$0 i access.log"
 echo "$0 tn access.log"
 echo "$0 ta access.log"
}

if [ $# -ne 2 ];then
   Help
   exit 1
else
   if [ ! -s $2 ];then
      echo "$2 file does not exist"
      exit 1
   fi
fi

case $1 in
    s|statuscode)
      GroupByStatusCode "$2";;
    h|remoteaddress)
      GroupByRemoteAddress $2;;
    e|referer)
      GroupByReferer $2;;
    a|useragent)
      GroupByUserAgent $2;;
    i|uri)
      GroupByUri $2;;
    r|url)
      GroupByUrl $2;;
    t|requesttime)
      GroupByRequestTime $2;;
    m|method)
      GroupByMethod $2;;
    tc|trendocunt)
      TrendTimesCount $2;;
    tn|trendnet)
      TrendTimesNetFlow $2;;
    ta|trendall)
      TrendTimesAll $2;;
    *)
      Help;;
esac

三.脚本执行结果：

1.打印帮助信息：
help.png
2.统计状态码和IP并排序：
code.png
3.统计请求方法、统计响应时间分布、统计uri
m.png
4.统计每分钟的数量、每分钟的流量、每分钟的状态码走势
t.png

四.附件

113.96.140.246 | [28/May/2020:15:32:55 +0800] | www.shell.com.cn | "GET /pcedu/ios/1106/2448633_1.html HTTP/1.1" | 200 | 8289 | "-" | "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" | "-" | - | - | - | 172.16.239.5 | 0.234 | https  | 47585
172.16.239.156 | [28/May/2020:15:32:55 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 1.232 | http  | 35949
172.16.239.158 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.982 | http  | 58399
172.16.246.51 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.545 | http  | 52384
113.96.140.246 | [28/May/2020:15:32:56 +0800] | office.shell.com.cn | "GET /1282/12825949_3.html HTTP/1.1" | 200 | 31173 | "https://office.shell.com.cn/1282/12825949_2.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" | "-" | - | - | - | 172.16.239.5 | 0.645 | https  | 47791
58.55.123.88 | [28/May/2020:15:32:56 +0800] | www1.shell.com.cn | "GET /2015/dl/client/images/btn_client.png HTTP/1.1" | 304 | 0 | "https://js.shell.com/bashTest/2015/dl/css/client.css" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.358 | https  | 26877
172.16.239.174 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.435 | http  | 40954
172.16.238.72 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /pcclub/itbbsindex/1702/intf17885.html HTTP/1.1" | 200 | 1766 | "-" | "NodeJs" | "-" | - | - | - | 172.16.239.5 | 0.519 | http  | 55833
172.16.240.137 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /product/fz/2015/1511/intf11701.html HTTP/1.1" | 200 | 0 | "-" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.905 | http  | 46830
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/409990.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.715 | https  | 47574
229.137.244.63 | [28/May/2020:15:32:56 +0800] | www1.shell.com.cn | "GET /zt/gz20140905/404/css/404.css HTTP/1.0" | 200 | 4429 | "-" | "-" | "-" | - | - | - | 172.16.239.5 | 0.926 | http  | 43571
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/385364.html" | "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36 2345Explorer/6.1.0.7934" | "-" | - | - | - | 172.16.239.5 | 0.445 | https  | 47574
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/385364.html" | "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36 2345Explorer/6.1.0.7934" | "-" | - | - | - | 172.16.239.5 | 0.506 | https  | 48098
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/63718.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.68 Safari/537.36 Core/1.53.4033.400 QQBrowser/9.6.12624.400" | "-" | - | - | - | 172.16.239.5 | 0.371 | https  | 48096
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/63718.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.68 Safari/537.36 Core/1.53.4033.400 QQBrowser/9.6.12624.400" | "-" | - | - | - | 172.16.239.5 | 0.341 | https  | 48098
113.96.140.245 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/html_2/1/121/id=1051&pn=0.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.805 | https  | 48098
114.203.12.194 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /xlab/miniapp/2005/intf23253_1.js?t=1590651164841 HTTP/1.1" | 200 | 1243 | "https://servicewechat.com/wx25cb37caf7f44a41/0/page-frame.html" | "Mozilla/5.0 (Linux; Android 8.0.0; Mi Note 2 Build/OPR1.170623.032; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/78.0.3904.62 XWEB/2353 MMWEBSDK/200401 Mobile Safari/537.36 MMWEBID/747 MicroMessenger/7.0.14.1660(0x27000E37) Process/appbrand0 NetType/WIFI Language/zh_CN ABI/arm64 WeChat/arm64" | "-" | - | - | - | 172.16.239.5 | 0.998 | https  | 44620
172.16.238.228 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.486 | http  | 50761
110.80.141.159 | [28/May/2020:15:32:56 +0800] | www.shell.com.cn | "GET /pcedu/soft/wl/brower/0311/242394.html HTTP/1.1" | 302 | 270 | "-" | "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" | "-" | - | - | - | 172.16.239.5 | 0.015 | https  | 38277
113.96.140.246 | [28/May/2020:15:32:56 +0800] | pcedu.shell.com.cn | "GET /1346/13462969.html HTTP/1.1" | 200 | 11872 | "https://www.baidu.com/link?url=4cyuuadApwd6GD33cnmpoToHdfj9zL1lQ4mJSsE43ROVRFUJ43OyJziAxJK8KPBMkYefrsIZr3CObtpXvHRG2q&wd=&eqid=c14c6a850005b8910.6600065ecf6924" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 2.265 | https  | 49769
113.96.140.246 | [28/May/2020:15:32:56 +0800] | mobile.shell.com.cn | "GET /380/3806282.html HTTP/1.1" | 200 | 28100 | "-" | "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" | "-" | - | - | - | 172.16.239.5 | 0.181 | https  | 31225
172.16.238.146 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.705 | http  | 53098
113.96.140.246 | [28/May/2020:15:32:57 +0800] | gps.shell.com.cn | "GET /553/5535909.html HTTP/1.1" | 200 | 13072 | "-" | "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)" | "-" | - | - | - | 172.16.239.5 | 0.255 | https  | 49801
113.96.140.245 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/1655767.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363" | "-" | - | - | - | 172.16.239.5 | 0.304 | https  | 49684
172.16.239.59 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.905 | http  | 35067
110.80.141.158 | [28/May/2020:15:32:57 +0800] | pcedu.shell.com.cn | "GET /1239/12390087.html HTTP/1.1" | 200 | 14525 | "https://www.so.com/link?m=a92lOp6pOJvedmxUx9TBeQMp4n8OlvOlrcFo88bNTlcmQHQGUIPkwTIJIGjfOGAQByXA1tYZ5GHFd7DO0SYwzXg%2Bv5NwTD1QSYf2AYR4GfD9H3jXpaj9nv5Az8Yf%2BKkCohL2N9RZGto6%2B5coh%2BJvERvajHydHfQ4YTH59HnxjqG6lwQIelamKmPceG3YQCP0LvFYjgwLSCLuER8G%2FmtJwk1RLZ6rpaNq5NlFtXw%3D%3D" | "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.941 | https  | 36322
150.138.109.205 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /2014/0121/zt4201891.html HTTP/1.1" | 301 | 290 | "-" | "Mozilla/5.0 (Linux; Android 7.0; FRD-AL00 Build/HUAWEIFRD-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/53.0.2785.49 Mobile MQQBrowser/6.2 TBS/043602 Safari/537.36 MicroMessenger/6.5.16.1120 NetType/WIFI Language/zh_CN" | "-" | - | - | - | 172.16.239.5 | 0.387 | http  | 59501
172.16.239.174 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.515 | http  | 41052
172.16.246.51 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/wap/1712/intf19897.html HTTP/1.1" | 200 | 2054 | "http://r.shell.com.cnhttp://dl.shell.com.cn/vr/" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.968 | http  | 52389
172.16.239.175 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.311 | http  | 60251
113.96.140.245 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/359460-1.html" | "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.781 | https  | 49684
112.90.135.106 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/2352841.html" | "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" | "-" | - | - | - | 172.16.239.5 | 0.878 | https  | 38873
172.16.239.22 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/jd/ac/1712/10519871.html HTTP/1.0" | 200 | 27162 | "-" | "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" | "116.179.32.71, 172.16.237.58, 172.16.238.216, 172.16.238.216" | - | - | - | 172.16.239.5 | 0.214 | http  | 10246
172.16.237.200 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/software/excel/1403/4509076.html HTTP/1.0" | 200 | 8078 | "https://www.so.com/link?m=as5SxBFhuuzOANq9bvfy5AcMPf8wMcuiS45Td%2FDwcSHQ744TpvL%2BM3iHG7CDeRYSptnK6efNNN3zWfp3mg4hDRhiUD%2BY0ss8pxOZ9tP04Xo2NGclBG9JVuB1PCXzHD1wiVOf5OUT%2F6tSjGvszO4fhPn3CHTXY%2FFRVXMHlyWGjzwsiWkkNChoxzJwhN64cseN5XBAUP9aMrrfVqFj4GtMqmToYKYc1RFwY8bkGncb1oFj4qedamsvLKGfEy6r0cdRS8y%2BZZDOjngsAZKhYw1rkUztOwVCFrdA1BtgsRC%2FuMdwjnLD6IhB1qepAnk4qZrbQ" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "113.65.30.180" | - | - | - | 172.16.239.5 | 0.199 | http  | 43433
27.128.146.74 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/2380882.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.048 | https  | 47253
27.148.141.118 | [28/May/2020:15:32:57 +0800] | hk.shell.com.cn | "GET //network.shell.com.cn/1344/13441764.html HTTP/1.1" | 302 | 270 | "-" | "Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; Media Center PC 4.0; SLCC1; .NET CLR 3.0.04320)" | "-" | - | - | - | 172.16.239.5 | 0.590 | http  | 40505
172.16.237.200 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /pdbk/itbk/software/dnyw/1707/9686422.html HTTP/1.0" | 200 | 8420 | "-" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362" | "171.113.81.245, 222.79.64.76" | - | - | - | 172.16.239.5 | 0.634 | http  | 43442
27.148.141.119 | [28/May/2020:15:32:57 +0800] | mobile.shell.com.cn | "GET /453/4537978.html HTTP/1.1" | 200 | 13638 | "-" | "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)" | "-" | - | - | - | 172.16.239.5 | 0.538 | https  | 62475
172.16.239.54 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.670 | http  | 55058
27.128.146.76 | [28/May/2020:15:32:57 +0800] | pcedu.shell.com.cn | "GET /1055/10554305.html HTTP/1.1" | 200 | 17286 | "https://www.baidu.com/link?url=QkXeaFixVvfQKX6LyyE29rGja5mQwx_L8_ehdHNnl_NO5bJ_TAlul2OpKpen2IJQOp-bXieCVidwkjP2o2HGjK&wd=&eqid=f67b455a0003bf310.8340045ecf6926" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 2.265 | https  | 47929
27.128.146.76 | [28/May/2020:15:32:57 +0800] | www.shell.com.cn | "GET /factory_pic/883/8830500_pic.html?imgsrc=//img0.shell.com.cn/bashTest/1702/14/8830500_1.jpg&channel=309 HTTP/1.1" | 200 | 9638 | "-" | "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)" | "-" | - | - | - | 172.16.239.5 | 0.206 | https  | 47997
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 304 | 0 | "https://dl.shell.com.cn/download/1825853.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.6788.400 QQBrowser/10.3.2776.400" | "-" | - | - | - | 172.16.239.5 | 0.220 | https  | 38873
27.148.141.117 | [28/May/2020:15:32:58 +0800] | tv.shell.com.cn | "GET /1283/12830092.html HTTP/1.1" | 200 | 32856 | "https://product.shell.com.cn/lcd_tv/sony/1156127.html" | "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.635 | https  | 37993
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/1825853.html" | "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.26 Safari/537.36 Core/1.63.6788.400 QQBrowser/10.3.2776.400" | "-" | - | - | - | 172.16.239.5 | 0.058 | https  | 39865
124.236.26.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /projector/437/4375785.html HTTP/1.1" | 200 | 14556 | "-" | "Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://aspiegel.com/petalbot)" | "-" | - | - | - | 172.16.239.5 | 0.119 | https  | 62999
112.90.135.106 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/activitys/2016/1611/intf17385.js HTTP/1.1" | 200 | 0 | "https://dl.shell.com.cn/download/2684489.html" | "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3947.100 Safari/537.36" | "-" | - | - | - | 172.16.239.5 | 0.286 | https  | 39865
172.16.238.61 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /global/navibar_html/index.html HTTP/1.1" | 200 | 13418 | "-" | "NodeJs" | "-" | - | - | - | 172.16.239.5 | 0.182 | http  | 50513
172.16.236.29 | [28/May/2020:15:32:58 +0800] | picture.shell.com.cn | "GET /diy/1112/2619852_6.html HTTP/1.0" | 200 | 4446 | "-" | "Mozilla/5.0 (compatible; SemrushBot/6~bl; +http://www.semrush.com/bot.html)" | "46.229.168.143, 172.16.237.171, 172.16.237.171, 172.16.239.12" | - | - | - | 172.16.239.5 | 0.321 | http  | 33494
172.16.239.59 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /product/11fz/1106/intf1224.html HTTP/1.1" | 200 | 1874 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.683 | http  | 35075
172.16.239.158 | [28/May/2020:15:32:58 +0800] | www.shell.com.cn | "GET /jrzcz/jk/pc/1608/intf15725.html HTTP/1.1" | 200 | 1549 | "http://pdlib.shell.com.cnproduct.shell.com.cn" | "cn.bashTest.JavaHttpClient" | "-" | - | - | - | 172.16.239.5 | 0.898 | http  | 58592
文章标签：
日志服务
Go
Shell
容器
应用服务中间件
Kubernetes
nginx
关键词：
日志服务统计
实战日志服务
Shell脚本
Nginx实战
日志服务实战
相关实践学习
日志服务之使用Nginx模式采集日志
本文介绍如何通过日志服务控制台创建Nginx模式的Logtail配置快速采集Nginx日志并进行多维度分析。
运维包工头
目录
相关文章
瓴羊Dataphin
|
11天前
|
存储 运维 监控
API明细日志及运维统计日志全面提升API可运维性
在数字化转型的大潮中，数据已成为企业最宝贵的资产之一。而数据服务API可快速为数据应用提供数据接口。面对越来越多的API以及越来越多的应用调用，如何快速查看API的服务情况、异常情况及影响范围，以及查看API的调用详情，进行API的性能优化、错误排查变得越来越重要，本文将介绍如何配置和开通API运维统计及明细日志，以及如何查看日志进行介绍。
瓴羊Dataphin
51 0
LKIDTI数据
|
1月前
|
Shell
一个用于添加/删除定时任务的shell脚本
一个用于添加/删除定时任务的shell脚本
LKIDTI数据
86 1
冷冻工厂
|
25天前
|
Shell Linux 测试技术
6种方法打造出色的Shell脚本
6种方法打造出色的Shell脚本
冷冻工厂
48 2
6种方法打造出色的Shell脚本
蓝易云
|
11天前
|
XML JSON 监控
Shell脚本要点和难点以及具体应用和优缺点介绍
Shell脚本在系统管理和自动化任务中扮演着重要角色。尽管存在调试困难、可读性差等问题，但其简洁高效、易于学习和强大的功能使其在许多场景中不可或缺。通过掌握Shell脚本的基本语法、常用命令和函数，并了解其优缺点，开发者可以编写出高效的脚本来完成各种任务，提高工作效率。希望本文能为您在Shell脚本编写和应用中提供有价值的参考和指导。
蓝易云
33 1
eisc
|
16天前
|
Ubuntu Shell 开发工具
ubuntu/debian shell 脚本自动配置 gitea git 仓库
这是一个自动配置 Gitea Git 仓库的 Shell 脚本，支持 Ubuntu 20+ 和 Debian 12+ 系统。脚本会创建必要的目录、下载并安装 Gitea，创建 Gitea 用户和服务，确保 Gitea 在系统启动时自动运行。用户可以选择从官方或小绿叶技术博客下载安装包。
eisc
37 2
eisc
|
30天前
|
监控 网络协议 Shell
ip和ip网段攻击拦截系统-绿叶结界防火墙系统shell脚本
这是一个名为“小绿叶技术博客扫段攻击拦截系统”的Bash脚本，用于监控和拦截TCP攻击。通过抓取网络数据包监控可疑IP，并利用iptables和firewalld防火墙规则对这些IP进行拦截。同时，该系统能够查询数据库中的白名单，确保合法IP不受影响。此外，它还具备日志记录功能，以便于后续分析和审计。
eisc
49 6
我不是游客20240119
|
26天前
|
运维 监控 Shell
深入理解Linux系统下的Shell脚本编程
【10月更文挑战第24天】本文将深入浅出地介绍Linux系统中Shell脚本的基础知识和实用技巧，帮助读者从零开始学习编写Shell脚本。通过本文的学习，你将能够掌握Shell脚本的基本语法、变量使用、流程控制以及函数定义等核心概念，并学会如何将这些知识应用于实际问题解决中。文章还将展示几个实用的Shell脚本例子，以加深对知识点的理解和应用。无论你是运维人员还是软件开发者，这篇文章都将为你提供强大的Linux自动化工具。
我不是游客20240119
45 1
蓝染-惣右介
|
19天前
|
XML 安全 Java
【日志框架整合】Slf4j、Log4j、Log4j2、Logback配置模板
本文介绍了Java日志框架的基本概念和使用方法，重点讨论了SLF4J、Log4j、Logback和Log4j2之间的关系及其性能对比。SLF4J作为一个日志抽象层，允许开发者使用统一的日志接口，而Log4j、Logback和Log4j2则是具体的日志实现框架。Log4j2在性能上优于Logback，推荐在新项目中使用。文章还详细说明了如何在Spring Boot项目中配置Log4j2和Logback，以及如何使用Lombok简化日志记录。最后，提供了一些日志配置的最佳实践，包括滚动日志、统一日志格式和提高日志性能的方法。
蓝染-惣右介
156 30
【日志框架整合】Slf4j、Log4j、Log4j2、Logback配置模板
最好zzz
|
1月前
|
XML JSON Java
Logback 与 log4j2 性能对比：谁才是日志框架的性能王者？
【10月更文挑战第5天】在Java开发中，日志框架是不可或缺的工具，它们帮助我们记录系统运行时的信息、警告和错误，对于开发人员来说至关重要。在众多日志框架中，Logback和log4j2以其卓越的性能和丰富的功能脱颖而出，成为开发者们的首选。本文将深入探讨Logback与log4j2在性能方面的对比，通过详细的分析和实例，帮助大家理解两者之间的性能差异，以便在实际项目中做出更明智的选择。
最好zzz
245 3
绿水长流*z
|
1月前
|
存储 缓存 关系型数据库
MySQL事务日志-Redo Log工作原理分析
事务的隔离性和原子性分别通过锁和事务日志实现，而持久性则依赖于事务日志中的`Redo Log`。在MySQL中，`Redo Log`确保已提交事务的数据能持久保存，即使系统崩溃也能通过重做日志恢复数据。其工作原理是记录数据在内存中的更改，待事务提交时写入磁盘。此外，`Redo Log`采用简单的物理日志格式和高效的顺序IO，确保快速提交。通过不同的落盘策略，可在性能和安全性之间做出权衡。
绿水长流*z
1645 14

热门文章

最新文章

  • 1
    如何优雅打印nginx header和body
  • 2
    nagios监控nginx status
  • 3
    nginx跳转
  • 4
    [译]Nginx和Lua
  • 5
    nginx禁止ip访问web
  • 6
    Caddy Web服务器深度解析与对比：Caddy vs. Nginx vs. Apache
  • 7
    nginx HTTP Upstream模块
  • 8
    Nginx使用指南(一)
  • 9
    CentOS 7 - 安装Nginx
  • 10
    Nginx防盗链、Nginx访问控制、Nginx解析php相关配置、Nginx代理
  • 1
    java使用AOP切面获取请求日志并记录
    160
  • 2
    实时计算 Flink版产品使用合集之 Flink on YARN 中使用滚动日志时配置不生效如何解决
    82
  • 3
    orhanobut/logger - 强大的Android日志打印库
    307
  • 4
    Java一分钟之-Log4j与日志记录的重要性
    175
  • 5
    webserver--多缓冲区实现日志系统
    62
  • 6
    实现Python日志点击跳转到代码位置的方法
    105
  • 7
    【后台开发】TinyWebser学习笔记（4）定时器、日志系统（单例模式）、封装互斥锁信号量
    39
  • 8
    日志服务的典型应用场景
    113
  • 9
    rman 多通道拷贝归档日志文件
    34
  • 10
    mongoDB查看数据的插入日志
    451

    • 相关课程

    更多
  • Shell 脚本进阶
  • Shell 脚本入门
  • Nginx企业级Web服务实战
  • Linux Web服务器Nginx搭建与配置
  • Linux Shell 编程入门与实战
  • Shell 编程入门到精通

    • 相关电子书

    更多
  • Shell 脚本速查手册
  • CentOS Nginx PHP JAVA 多语言镜像使用手
  • CentOS Nginx PHP JAVA多语言镜像使用手册

    • 相关实验场景

    更多
  • 使用CloudLens管理SLS日志采集
  • 使用CloudLens采集PolarDB日志并进行审计分析
  • 使用CloudLens采集RDS日志并进行审计分析
  • 使用日志服务SLS进行OSS可观测分析
  • Arthas中session、tee、version、wc基础命令应用
  • Arthas中cat 、cls、echo、grep基础命令应用
    • 下一篇
    无影云桌面