入参出参日志
我们日常开发中日志是不可缺少的一部分，
如mini-cloud架构图所示，大型系统一般可用elk 等进行日志收集
中小型系统也可以用spring-boot-admin 等进行收集，但我们业务场景经常
会有一种需求，就是一些重要入参出参接口希望按照url 进行收集并便于以后排查分析
比较典型的就是金融产品或者银行产品扣款，出账，转账，扣款等
期望效果
我们可能会希望通过一个url 或者关联参数定位查询某接口入参出参，比如转账例子，我们希望单独看转账接口得入出参日志
url: /transfer
args: {"transfer":1,"amount":200,"to":2}
response: {"status":200,"msg":"转账成功"}
keyword:{"转账"}
description:"转账接口记录"
架构图
源码
共通部分aop+自定义注解
我们在共同中添加common-log模块，并添加spring.factories 自动注入，目录结构如下：
pom.xml
1. <dependencies> <!--web 模块--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-aop</artifactId> </dependency> <dependency> <groupId>cn.hutool</groupId> <artifactId>hutool-all</artifactId> </dependency> <!--rocketmq依赖--> <dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-spring-boot-starter</artifactId> <version>2.2.2</version> </dependency> </dependencies>
IOLogRecordDTO.java
@Builder @Getter @Setter public class IOLogRecordDTO implements Serializable { private Long timestamp ; private String method; private String url ; private String contentType; private String args ; private Object response ; private String dateTime; private String keyword ; private String description; }
IOLogRecorder.java
@Retention(RetentionPolicy.RUNTIME) @Target(ElementType.METHOD) public @interface IOLogRecorder { String keyword() default ""; String descrition() default ""; }
IOLogAspect.java
@Aspect public class IOLogAspect { @Autowired RocketMQTemplate rocketMQTemplate; @Pointcut("@annotation(com.minicloud.common.log.annotation.IOLogRecorder)") public void pointCut() { } @Around("pointCut()") public Object record(ProceedingJoinPoint joinPoint) throws Throwable { ServletRequestAttributes servletRequestAttributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes(); HttpServletRequest request = servletRequestAttributes.getRequest(); String url = request.getRequestURI(); String contentType= request.getHeader("content-type"); String method = request.getMethod(); Signature signature = joinPoint.getSignature(); MethodSignature methodSignature = (MethodSignature) signature; IOLogRecorder ioLogRecorder = methodSignature.getMethod().getAnnotation(IOLogRecorder.class); System.out.println(methodSignature.getMethod().getName()); Object[] args = joinPoint.getArgs(); String inArgs = JSONUtil.toJsonStr(args); Object response = joinPoint.proceed(); long timestamp = System.nanoTime(); IOLogRecordDTO ioLogRecordDTO = IOLogRecordDTO.builder().keyword(ioLogRecorder.keyword()).description(ioLogRecorder.descrition()).url(url).contentType(contentType).method(method).args(inArgs).response(response).dateTime(LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS"))).timestamp(timestamp).build(); rocketMQTemplate.send("iolog", MessageBuilder.withPayload(ioLogRecordDTO).build()); return response; } }
IOLogConfigration.java
@Configuration public class IOLogConfigration { @Bean public IOLogAspect ioLogAspect(){ return new IOLogAspect(); } }
spring.factories
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\ com.minicloud.common.log.config.IOLogConfigration
rockemq搭建
本篇暂时不提及rocketmq搭建，因为涉及内容太多，会有单独篇幅介绍
日志数据库创建
日志存储可以是数据库，文件，缓存等，本篇使用的是mysql数据库
CREATE TABLE `iolog` ( `id` int(11) NOT NULL AUTO_INCREMENT COMMENT '住建', `timestamp` bigint(20) DEFAULT NULL COMMENT '时间戳', `method` varchar(10) DEFAULT '' COMMENT '请求方式', `url` varchar(100) DEFAULT NULL COMMENT '请求url', `content_type` varchar(30) DEFAULT '' COMMENT '数据类型', `args` varchar(500) DEFAULT '' COMMENT '请求参数', `response` varchar(1000) DEFAULT '' COMMENT '响应', `data_time` varchar(30) DEFAULT '' COMMENT '日志时间', `keyword` varchar(50) DEFAULT '' COMMENT '关键字', `description` varchar(100) DEFAULT NULL COMMENT '描述', PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=18 DEFAULT CHARSET=utf8mb4;
mq 消费端搭建
mq消费端是一个独立的mq服务，因为以后还需要集成别的消费业务，所以独立消费端便于扩展，不与具体某业务服务耦合，具体代码如下：
pom.xml
<dependencies> <!--注册中心客户端--> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId> </dependency> <!--配置中心客户端--> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId> </dependency> <!--日志拦截依赖--> <dependency> <groupId>org.mini-cloud</groupId> <artifactId>mini-cloud-common-log</artifactId> <version>1.0-SNAPSHOT</version> </dependency> <dependency> <groupId>org.mini-cloud</groupId> <artifactId>mini-cloud-common-fegin</artifactId> <version>1.0-SNAPSHOT</version> <scope>compile</scope> </dependency> <!--common data 依赖 --> <dependency> <groupId>org.mini-cloud</groupId> <artifactId>mini-cloud-common-data</artifactId> <version>1.0-SNAPSHOT</version> </dependency> <dependency> <groupId>org.mini-cloud</groupId> <artifactId>mini-cloud-common-auth</artifactId> <version>1.0-SNAPSHOT</version> <scope>compile</scope> </dependency> </dependencies>
IOLogConsumer.java
@Component @RocketMQMessageListener(consumerGroup = "group-a", topic = "iolog",consumeMode = ConsumeMode.ORDERLY) public class IOLogConsumer implements RocketMQListener<String> { @Autowired private IOLogRecordDao ioLogRecordDao; @Override public void onMessage(String message) { IOLogRecordDTO ioLogRecordDTO = JSONUtil.toBean(message,IOLogRecordDTO.class); ioLogRecordDao.insert(ioLogRecordDTO); } }
MiniCloudLogConsumerApplication.java
@SpringCloudApplication @EnableCaching @EnableMiniCloudFeignClients @EnableMiniCloudResourceServer public class MiniCloudLogConsumerApplication { public static void main(String[] args) { SpringApplication.run(MiniCloudLogConsumerApplication.class, args); } }
nacos 中 mini-cloud-log-consumer-dev.yml
server: port: 6600 tomcat: uri-encoding: UTF-8 max-threads: 500 max-connections: 10000 accept-count: 500 spring: shardingsphere: props: sql: show: true datasource: names: master,slave0,slave1 master: type: com.alibaba.druid.pool.DruidDataSource driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://${MYSQL_HOST:192.168.1.2}:${MYSQL_PORT:3306}/${MYSQL_DB:mini_cloud_log}?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true&allowPublicKeyRetrieval=true username: root password: root slave0: type: com.alibaba.druid.pool.DruidDataSource driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://${MYSQL_HOST:192.168.1.2}:${MYSQL_PORT:3306}/${MYSQL_DB:mini_cloud_log}?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true&allowPublicKeyRetrieval=true username: root password: root slave1: type: com.alibaba.druid.pool.DruidDataSource driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://${MYSQL_HOST:192.168.1.2}:${MYSQL_PORT:3306}/${MYSQL_DB:mini_cloud_log}?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true&allowPublicKeyRetrieval=true username: root password: root sharding: master-slave-rules: master: master-data-source-name: master slave-data-source-names: slave0,slave1
服务日志发送方集成
日志发送方其实就是实际的各个业务端服务，本文在upms 服务的一个角色分页接口加上日志注解，只要添加 @IOLogRecorder 变可自动收集入参出参发送到mq,如下：
重启服务以及查看结果
我们在业务端请求一个角色分页列表接口，来看看效
业务消费端接收到消息
来看看response
{ "headers": { }, "statusCodeValue": 200, "body": { "data": [ { "roleId": 33, "roleDesc": "最大权限", "roleCode": "SUPER_ADMIN", "roleName": "超级管理员", "tenantId": 1, "upmsPermDTOS": [ ] }, { "roleId": 34, "roleDesc": "普通用户1", "roleCode": "USER1", "roleName": "普通用户1", "tenantId": 1, "upmsPermDTOS": [ ] }, { "roleId": 35, "roleDesc": "普通用户2", "roleCode": "USER2", "roleName": "普通用户2", "tenantId": 1, "upmsPermDTOS": [ ] } ], "total": 3, "size": 10, "page": 1 }, "statusCode": "OK" }
看结果已经通过mq消费端保存到数据库里了，以后可以通过各个字段进行查询操作
