一、问题回顾
线上的代码之前运行的都很平稳,突然就出现了一个很奇怪的问题,看错误信息是第三方框架Druid报出来了,连接池回收连接时出现的问题。
2018-05-14 20:01:32.810 ERROR [hystrix-UpgradeResultReportController-49][DruidDataSource.java:1297] - recyle error java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_131] at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_131] at com.alibaba.druid.pool.DruidDataSource.recycle(DruidDataSource.java:1279) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.pool.DruidPooledConnection.recycle(DruidPooledConnection.java:292) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.filter.FilterChainImpl.dataSource_recycle(FilterChainImpl.java:4534) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.filter.FilterAdapter.dataSource_releaseConnection(FilterAdapter.java:2717) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.filter.FilterChainImpl.dataSource_recycle(FilterChainImpl.java:4530) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.filter.stat.StatFilter.dataSource_releaseConnection(StatFilter.java:646) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.filter.FilterChainImpl.dataSource_recycle(FilterChainImpl.java:4530) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.pool.DruidPooledConnection.syncClose(DruidPooledConnection.java:269) [druid-1.0.11.jar!/:1.0.11] at com.alibaba.druid.pool.DruidPooledConnection.close(DruidPooledConnection.java:228) [druid-1.0.11.jar!/:1.0.11] at org.springframework.jdbc.datasource.DataSourceUtils.doCloseConnection(DataSourceUtils.java:341) [spring-jdbc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE] at org.springframework.jdbc.datasource.DataSourceUtils.doReleaseConnection(DataSourceUtils.java:328) [spring-jdbc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE] at org.springframework.jdbc.datasource.DataSourceUtils.releaseConnection(DataSourceUtils.java:294) [spring-jdbc-4.3.8.RELEASE.jar!/:4.3.8.RELEASE] at org.mybatis.spring.transaction.SpringManagedTransaction.close(SpringManagedTransaction.java:127) [mybatis-spring-1.3.1.jar!/:1.3.1] at org.apache.ibatis.executor.BaseExecutor.close(BaseExecutor.java:90) [mybatis-3.4.2.jar!/:3.4.2] at org.apache.ibatis.executor.CachingExecutor.close(CachingExecutor.java:64) [mybatis-3.4.2.jar!/:3.4.2] at org.apache.ibatis.session.defaults.DefaultSqlSession.close(DefaultSqlSession.java:264) [mybatis-3.4.2.jar!/:3.4.2] at org.mybatis.spring.SqlSessionUtils.closeSqlSession(SqlSessionUtils.java:193) [mybatis-spring-1.3.1.jar!/:1.3.1] at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:454) [mybatis-spring-1.3.1.jar!/:1.3.1] at com.sun.proxy.$Proxy133.update(Unknown Source) [?:?] at org.mybatis.spring.SqlSessionTemplate.update(SqlSessionTemplate.java:294) [mybatis-spring-1.3.1.jar!/:1.3.1] at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:62) [mybatis-3.4.2.jar!/:3.4.2] at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:59) [mybatis-3.4.2.jar!/:3.4.2] at com.sun.proxy.$Proxy148.updDeviceUpgradeStatus(Unknown Source) [?:?] at com.phicomm.smarthome.ota.service.impl.UpgradeHistoryDaoServiceImpl.updUpgradeSuccessStatus(UpgradeHistoryDaoServiceImpl.java:50R [classes!/:0.0.1-SNAPSHOT] at com.phicomm.smarthome.ota.service.impl.UpgradeHistoryDaoServiceImpl$$FastClassBySpringCGLIB$$9b468fba.invoke(<generated>) [classes!/:0.0.1-SNAPSHOT] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-4.3.8.RELEASE.jar!/:4.3.8.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) [spring-aop-4.3.8.RELEASE.jar!/:4.3.8.RELEASE]
二、排查问题
查看一下他的源码
看了一下也没有发现所以然,然后去GitHub上的Druid官方开源处,看了一下历史问题修复,发现这个是旧版本已知的一个Bug。
https://github.com/alibaba/druid/issues/785
看了一下当前使用的Druid的版本【1.0.11】,然后又确认了一下官方最新的版本是【1.1.9】,已经差了很多个版本了。
果断将版本更新至最新
<dependency> <groupId>com.alibaba</groupId> <artifactId>druid</artifactId> <version>1.1.9</version> </dependency>
翻看之前出错的那里的代码发现
更新到最新版本后,上述问题得到解决。
数据库连接被中断的原因有很多,有一种是初始化配置时设置的数据库连接回收时长,通过datasource.getConnontion() 取得的连接必须在removeAbandonedTimeout这么多秒内调用close(),否则就主动将其杀死,就是conn不能超过指定的租期。
详细配置可以参考这篇文章: