Python 的 logging 模块提供了通用的日志系统,可以方便第三方模块或者是应用使用。这个模块提供不同的日志级别,并可以采用不同的方式记录日志,比如:文件、HTTP GET/POST、SMTP、Socket等,甚至可以自己实现具体的日志记录方式。logging 模块与 log4j 的机制是一样的,只是具体的实现细节不同。模块提供 logger、handler、filter、formatter。
Python日志格式
[backcolor=transparent]日志格式
日志的格式在formatter中指定日志记录输出的具体格式。formatter的构造方法需要两个参数:消息的格式字符串和日期字符串,这两个参数都是可选的。
Python日志格式:
- [backcolor=transparent]import[backcolor=transparent] logging
- [backcolor=transparent]import[backcolor=transparent] logging[backcolor=transparent].[backcolor=transparent]handlers
- [backcolor=transparent]LOG_FILE [backcolor=transparent]=[backcolor=transparent] [backcolor=transparent]'tst.log'[backcolor=transparent]
- [backcolor=transparent]handler [backcolor=transparent]=[backcolor=transparent] logging[backcolor=transparent].[backcolor=transparent]handlers[backcolor=transparent].[backcolor=transparent]RotatingFileHandler[backcolor=transparent]([backcolor=transparent]LOG_FILE[backcolor=transparent],[backcolor=transparent] maxBytes [backcolor=transparent]=[backcolor=transparent] [backcolor=transparent]1024[backcolor=transparent]*[backcolor=transparent]1024[backcolor=transparent],[backcolor=transparent] backupCount [backcolor=transparent]=[backcolor=transparent] [backcolor=transparent]5[backcolor=transparent])[backcolor=transparent] [backcolor=transparent]# 实例化 handler
- [backcolor=transparent]fmt [backcolor=transparent]=[backcolor=transparent] [backcolor=transparent]'%(asctime)s - %(filename)s:%(lineno)s - %(name)s - %(message)s'[backcolor=transparent]
- [backcolor=transparent]formatter [backcolor=transparent]=[backcolor=transparent] logging[backcolor=transparent].[backcolor=transparent]Formatter[backcolor=transparent]([backcolor=transparent]fmt[backcolor=transparent])[backcolor=transparent] [backcolor=transparent]# 实例化 formatter
- [backcolor=transparent]handler[backcolor=transparent].[backcolor=transparent]setFormatter[backcolor=transparent]([backcolor=transparent]formatter[backcolor=transparent])[backcolor=transparent] [backcolor=transparent]# 为 handler 添加 formatter
- [backcolor=transparent]logger [backcolor=transparent]=[backcolor=transparent] logging[backcolor=transparent].[backcolor=transparent]getLogger[backcolor=transparent]([backcolor=transparent]'tst'[backcolor=transparent])[backcolor=transparent] [backcolor=transparent]# 获取名为 tst 的 logger
- [backcolor=transparent]logger[backcolor=transparent].[backcolor=transparent]addHandler[backcolor=transparent]([backcolor=transparent]handler[backcolor=transparent])[backcolor=transparent] [backcolor=transparent]# 为 logger 添加 handler
- [backcolor=transparent]logger[backcolor=transparent].[backcolor=transparent]setLevel[backcolor=transparent]([backcolor=transparent]logging[backcolor=transparent].[backcolor=transparent]DEBUG[backcolor=transparent])[backcolor=transparent]
- [backcolor=transparent]logger[backcolor=transparent].[backcolor=transparent]info[backcolor=transparent]([backcolor=transparent]'first info message'[backcolor=transparent])[backcolor=transparent]
- [backcolor=transparent]logger[backcolor=transparent].[backcolor=transparent]debug[backcolor=transparent]([backcolor=transparent]'first debug message'[backcolor=transparent])
[backcolor=transparent]字段含义
关于 formatter 的配置,采用的是 %(key)s 的形式,就是字典的关键字替换。提供的关键字包括:
[backcolor=transparent]日志样例
- [backcolor=transparent]2015[backcolor=transparent]-[backcolor=transparent]03[backcolor=transparent]-[backcolor=transparent]04[backcolor=transparent] [backcolor=transparent]23[backcolor=transparent]:[backcolor=transparent]21[backcolor=transparent]:[backcolor=transparent]59[backcolor=transparent],[backcolor=transparent]682[backcolor=transparent] [backcolor=transparent]-[backcolor=transparent] log_test[backcolor=transparent].[backcolor=transparent]py[backcolor=transparent]:[backcolor=transparent]16[backcolor=transparent] [backcolor=transparent]-[backcolor=transparent] tst [backcolor=transparent]-[backcolor=transparent] first info message
- [backcolor=transparent]2015[backcolor=transparent]-[backcolor=transparent]03[backcolor=transparent]-[backcolor=transparent]04[backcolor=transparent] [backcolor=transparent]23[backcolor=transparent]:[backcolor=transparent]21[backcolor=transparent]:[backcolor=transparent]59[backcolor=transparent],[backcolor=transparent]682[backcolor=transparent] [backcolor=transparent]-[backcolor=transparent] log_test[backcolor=transparent].[backcolor=transparent]py[backcolor=transparent]:[backcolor=transparent]17[backcolor=transparent] [backcolor=transparent]-[backcolor=transparent] tst [backcolor=transparent]-[backcolor=transparent] first debug message
配置Logtail收集Python日志
配置Logtail收集Python日志的详细操作步骤请参考
快速入门和
apache日志,根据您的网络部署和实际情况选择对应配置。
在生成正则式的部分,由于自动生成的正则式只参考了日志样例,无法覆盖所有的日志情况,所以需要用户在自动生成之后做一些微调。
常见的Python日志及其正则表达式:
- 日志样例:
-
[backcolor=transparent]
2016[backcolor=transparent]
-[backcolor=transparent]
02[backcolor=transparent]
-[backcolor=transparent]
19[backcolor=transparent]
[backcolor=transparent]
11[backcolor=transparent]
:[backcolor=transparent]
03[backcolor=transparent]
:[backcolor=transparent]
13[backcolor=transparent]
,[backcolor=transparent]
410[backcolor=transparent]
[backcolor=transparent]
-[backcolor=transparent]
test[backcolor=transparent]
.[backcolor=transparent]
py[backcolor=transparent]
:[backcolor=transparent]
19[backcolor=transparent]
[backcolor=transparent]
-[backcolor=transparent]
tst [backcolor=transparent]
-[backcolor=transparent]
first debug message
正则表达式:
- [backcolor=transparent]([backcolor=transparent]\d[backcolor=transparent]+-[backcolor=transparent]\d[backcolor=transparent]+-[backcolor=transparent]\d[backcolor=transparent]+[backcolor=transparent]\s\S[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+-[backcolor=transparent]\s[backcolor=transparent]+([^:]+):([backcolor=transparent]\d[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+-[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\w[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+-[backcolor=transparent]\s[backcolor=transparent]+(.*)
日志格式:
-
[backcolor=transparent]
%([backcolor=transparent]
asctime[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
-[backcolor=transparent]
[backcolor=transparent]
%([backcolor=transparent]
filename[backcolor=transparent]
)[backcolor=transparent]
s[backcolor=transparent]
:%([backcolor=transparent]
lineno[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
-[backcolor=transparent]
[backcolor=transparent]
%([backcolor=transparent]
levelno[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
%([backcolor=transparent]
levelname[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
%([backcolor=transparent]
pathname[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
%([backcolor=transparent]
module[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
%([backcolor=transparent]
funcName[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
%([backcolor=transparent]
created[backcolor=transparent]
)[backcolor=transparent]
f [backcolor=transparent]
%([backcolor=transparent]
thread[backcolor=transparent]
)[backcolor=transparent]
d [backcolor=transparent]
%([backcolor=transparent]
threadName[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
%([backcolor=transparent]
process[backcolor=transparent]
)[backcolor=transparent]
d [backcolor=transparent]
%([backcolor=transparent]
name[backcolor=transparent]
)[backcolor=transparent]
s [backcolor=transparent]
-[backcolor=transparent]
[backcolor=transparent]
%([backcolor=transparent]
message[backcolor=transparent]
)[backcolor=transparent]
s
日志样例:
- [backcolor=transparent]2016[backcolor=transparent]-[backcolor=transparent]02[backcolor=transparent]-[backcolor=transparent]19[backcolor=transparent] [backcolor=transparent]11[backcolor=transparent]:[backcolor=transparent]06[backcolor=transparent]:[backcolor=transparent]52[backcolor=transparent],[backcolor=transparent]514[backcolor=transparent] [backcolor=transparent]-[backcolor=transparent] test[backcolor=transparent].[backcolor=transparent]py[backcolor=transparent]:[backcolor=transparent]19[backcolor=transparent] [backcolor=transparent]-[backcolor=transparent] [backcolor=transparent]10[backcolor=transparent] DEBUG test[backcolor=transparent].[backcolor=transparent]py test [backcolor=transparent]<module>[backcolor=transparent] [backcolor=transparent]1455851212.514271[backcolor=transparent] [backcolor=transparent]139865996687072[backcolor=transparent] [backcolor=transparent]MainThread[backcolor=transparent] [backcolor=transparent]20193[backcolor=transparent] tst [backcolor=transparent]-[backcolor=transparent] first debug message
正则表达式:
- [backcolor=transparent]([backcolor=transparent]\d[backcolor=transparent]+-[backcolor=transparent]\d[backcolor=transparent]+-[backcolor=transparent]\d[backcolor=transparent]+[backcolor=transparent]\s\S[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]-[backcolor=transparent]\s[backcolor=transparent]([^:]+):([backcolor=transparent]\d[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+-[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\d[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\w[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\S[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\w[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\S[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\S[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\d[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\w[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\d[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+([backcolor=transparent]\w[backcolor=transparent]+)[backcolor=transparent]\s[backcolor=transparent]+-[backcolor=transparent]\s[backcolor=transparent]+(.*)