问题背景:
一般程序系统发生异常,可能是由于多种原因引起的,例如:由于消息队列处理超时或文件系统上传超时等导致的问题;恰好日志log又没有做对应的打印输出,ELK
又未收集到对应的异常信息,无法清楚的知道程序具体的一个业务流程,对应的开发、测试环境又暂时无法使用或没有可供测试的数据。
解决方案:
可以通过线上监控工具,在不影响程序运行和业务使用的情况下,对程序功能代码进行方法级别的监控分析。通过监控观察程序的:类名、方法名、入参、出参等,来提供解决问题的排查思路。
watch命令的使用
watch [-b] [-e] [--exclude-class-pattern <value>] [-x <value>] [-f] [-h] [-n <value>] [--listenerId <value>] [-E] [-M <value>] [-s] [-v] class-pattern met hod-pattern [express] [condition-express] 复制代码
参数说明:
比如查看某个类的某个方法,同时查看请求参数与返回结果,示例如下:
watch com.offlineRepayment.server.controller.InvitationCodeCtrlImpl generateInvitationCode '{params[0],returnObj,throwExp}' -x 2 复制代码
null, [arthas@1]$ watch com..offlineRepayment.server.controller.InvitationC Press Q or Ctrl+C to abort. Affect(classcount:2method count:2) cost in 322 ms,listenemIdr
页面请求接口之后,可以看到如下内容:
[arthas@1]$ watch com offlineRepayment.server.controller.InvitationCodeCtrlImpl qenerateInvitationCode Press Q or Ctrl+C to abort. Affect(class count:2,method count: 2) cost in 322 ms, listenerId: 4 nethod=com. .offlineRepayment.server.controller.InvitationCodeCtrlImpl.qenerateInvitationCode location=At ts=2021-05-14 11:42:36;[cost=15.590555ms] result=@ArrayList [ @BaseRequest[ sign=@String[string], isonStr=@String[string],入参 ], @Bse64JsonResponse[ sign=@String[7cdcaa310f3f609fe40e322524d885a1],出参 jsonStr=@String[eyJyZXR1cm5Db2R1IjoiMDAiLCJtc2ci0iLnJ/miJDpgoDor7fnoIHlh7rplJnvvJpjb20uZmFzdGVyeG1sLmp 90T2sn0iB3YXMgZXhwZWN0aW5nICgndHJ1ZScsICdmYWxzZScgb3IgJ251bGwnKVxuIGF0IFtTb3VyY2U6IHNpZ25Jc05vdE9r0yBsaw5l0iAxI null,异常打印如果有
如果还行更进一步监控所调方法的参数,可继续直接监控方法里面的调用方法,比如:
watch com.offlineRepayment.platform.utils.JSONUtil readValue '{params,returnObj,throwExp}' -x 2 复制代码
Press Q or Ctrl+C to abort. Affect(classcount:1,method count: 1) cost in 115 ms, listenerId: 8 method=com.ppmoney.offlineRepayment.platform.utils.JSONUtil.readValue location=AtExceptionExit ts=2021-05-14 18:01:22;[cost=0.443645ms] result=@ArrayList [ @Object [] [ @String[siqnIsNotOk], @Class[classcom.ppmoney.offlineRepayment.platform.vo.GenerateInvitationCodeRequestVo],], null, "false' java.lang.IlleqalArqumentException:com.fasterxml.jackson.core.JsonParseException:Unrecoqnized token 'signIsNotok": was expecting ('true', at [Source:siqnIsNot0k;line:1,column:23] atcomppmoney.offlineRepayment.platform.utils.JSONUtil.readValue(JSONUtil.java:32) at com.ppmoney.offlineRepayment.server.controller.InvitationCodeCtrlImpl.generateInvitationCode(InvitationCodectrlImpl.java:47) at com.ppmoney.offlineRepayment.server.controller.InvitationCodeCtrlImpl$$FastClassBySpringCGLIB$$4cd118bb.invoke(generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) atorg.springframework.aop.framework.cglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) atorg.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85) atcom.ppmoney.offlineRepayment.platform.aop.channelsInterFaceLoqLoqAop.doMethod(ChannelsInterFaceLoqLoqAop.java:39) at sun.reflect.GeneratedMethodAccessor173.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodwithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:52) atorg.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) atorg.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:62) at ora.sprinaframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.iava:168) atorg.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at ora.sprinaframework.aop.framework.CalibAoDProxvSDvnamicAdvisedInterceptor.interceptlCalibAoDProxv.iava:673) at com.ppmoney.offlineRepayment.server.controller.Invitationcodectr1Imp1$$EnhancerByspringcGLIB$$6e20e72e.generateInvitationc@d希(g国金技t本dt:区5) at sun.reflect.GeneratedMethodAccessor172.invoke(Unknown Source)
参考
arthas其他应用:
- 查看cpu使用率;
- 热部署(进行编译录入,不需要重启应用);
- 反编译class文件;
- 动态跟踪Java代码;
- 实时监控JVM状态等如果有兴趣可在官网上查询