优化一
故障现象
radius认证存在大量的重复请求，radius server在timeout之前无法及时响应。
分析过程
1、使用java提供的线程dump工具dump出当前的线程状态，进行分析
Thread: default_authen_thread7(16408500110), State: RUNNABLE
at java.net.SocketInputStream.socketRead0(SocketInputStream.java:-2)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at net.sourceforge.jtds.jdbc.SharedSocket.readPacket(SharedSocket.java:846)
at net.sourceforge.jtds.jdbc.SharedSocket.getNetPacket(SharedSocket.java:727)
at net.sourceforge.jtds.jdbc.ResponseStream.getPacket(ResponseStream.java:466)
at net.sourceforge.jtds.jdbc.ResponseStream.read(ResponseStream.java:103)
at net.sourceforge.jtds.jdbc.ResponseStream.peek(ResponseStream.java:88)
at net.sourceforge.jtds.jdbc.TdsCore.wait(TdsCore.java:3932)
at net.sourceforge.jtds.jdbc.TdsCore.executeSQL(TdsCore.java:1046)
at net.sourceforge.jtds.jdbc.TdsCore.submitSQL(TdsCore.java:899)
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.commit(ConnectionJDBC2.java:2094)
at com.alibaba.druid.filter.FilterChainImpl.connection_commit(FilterChainImpl.java:180)
at com.alibaba.druid.filter.FilterAdapter.connection_commit(FilterAdapter.java:780)
at com.alibaba.druid.filter.FilterChainImpl.connection_commit(FilterChainImpl.java:176)
at com.alibaba.druid.proxy.jdbc.ConnectionProxyImpl.commit(ConnectionProxyImpl.java:123)
at com.alibaba.druid.pool.DruidPooledConnection.commit(DruidPooledConnection.java:731)
at org.hibernate.transaction.JDBCTransaction.commitAndResetAutoCommit(JDBCTransaction.java:170)
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:146)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:655)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
at com.ruijie.spl.common.cache2.TransactionManagerWrapper.commit(TransactionManagerWrapper.java:42)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy412.addMacBindInfoTCache(null:-1)
at com.ruijie.spl.usermanage.service.UserMabCheckServiceImpl.autoStudyTerminalInfo(UserMabCheckServiceImpl.java:249)
at com.ruijie.spl.radius.author.face.AuthorInterface.verifyForPackageMode(AuthorInterface.java:2860)
at com.ruijie.spl.radius.author.face.AuthorInterface.verify(AuthorInterface.java:257)
at com.ruijie.spl.radius.authen.support.eap.AuthenRuleImpl_EAP.doEapMD5Authen(AuthenRuleImpl_EAP.java:374)
at com.ruijie.spl.radius.authen.support.eap.AuthenRuleImpl_EAP.doAuthen(AuthenRuleImpl_EAP.java:308)
at com.ruijie.spl.radius.authen.support.RadiusAuthenRule.doAuthen(RadiusAuthenRule.java:56)
at com.ruijie.spl.radius.authen.process.RadiusProcessor.process(RadiusProcessor.java:490)
at com.ruijie.spl.common.net.radius.PacketProcessThread.runOnce(PacketProcessThread.java:68)
at com.ruijie.spl.common.power.AbstractDaemonThread.run(AbstractDaemonThread.java:157)分析发现，当前的性能瓶颈在数据库，认证过程中发生了数据库的IO操作。
分析代码发现，实质上当前卡住的位置使用的是异步数据库插入、更新策略，不存在数据库IO。
public void update(MacBindInfo info, boolean async) {
UpdateMacBindInfoTask updateTask = new UpdateMacBindInfoTask(this, info, getTransactionTemplate().getTransactionManager());
AsyncExecutorManager asyncManager = (AsyncExecutorManager)SpringBeanFinder.findSingleton("asyncExecutorManager");
if (async && asyncManager.isEnabled()) {
asyncManager.getExecutor().submit(updateTask);//程序中实质上使用的异步队列
} else {
updateTask.run();
}
if (Debuger.isDebugEnabled()) {
Debuger.debug("用户: "+info.getUserId()+" 已经被刷新到数据库");
}
}仔细分析发现，spring的申明式事务管理，对接口进行了动态代理，自动加上了开启事务、提交事务的操作，涉及到数据库IO操作。
<bean id="transactionProxy" abstract="true"
class="org.springframework.transaction.interceptor.TransactionProxyFactoryBean">
<property name="transactionManager" ref="TransactionManager" />
<property name="transactionAttributes">
<props>
<prop key="move*">PROPAGATION_REQUIRED,-Exception</prop>
<prop key="insert*">
PROPAGATION_REQUIRED,-Exception
</prop>
<prop key="add*">PROPAGATION_REQUIRED,-Exception</prop>
<prop key="delete*">
PROPAGATION_REQUIRED,-Exception
</prop>
<prop key="update*">
PROPAGATION_REQUIRED,-Exception
</prop>
<prop key="save*">PROPAGATION_REQUIRED,-Exception</prop>
<prop key="find*">PROPAGATION_SUPPORTS,-readOnly</prop>
<prop key="get*">PROPAGATION_SUPPORTS,-readOnly</prop>
<prop key="count*">PROPAGATION_SUPPORTS,-readOnly</prop>
<prop key="check*">PROPAGATION_SUPPORTS,-readOnly</prop>
</props>
</property>
</bean>业务的解法：分析业务发现，该业务设计的初衷是必须使用异步提交策略，不应该开启事务，结合spring的申明式事务配置机制，对接口重新定义（换个名字）。
2、进行如上优化后，故障并没有明显的缓解，继续进行线程dump分析。
Thread: default_authen_thread31(17414000170), State: WAITING
at sun.misc.Unsafe.park(Unsafe.java:-2)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:731)
at com.ruijie.spl.proxy.pool.StrategySamMapPool.findAll(StrategySamMapPool.java:104)
at com.ruijie.spl.proxy.service.StrategySamServiceImpl.getStrategyAllByOperatorNmae(StrategySamServiceImpl.java:61)
at sun.reflect.GeneratedMethodAccessor1891.invoke(null:-1)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy467.getStrategyAllByOperatorNmae(null:-1)operater.strategy.autoregist
at com.ruijie.spl.proxy.util.StrategyCheckUtil.checkLimitemode(StrategyCheckUtil.java:37)
at com.ruijie.spl.proxy.mapping.AcceptIntercepter.processAccept(AcceptIntercepter.java:225)
at com.ruijie.spl.proxy.util.ProxyUtil.acceptIntercept(ProxyUtil.java:1868)
at com.ruijie.spl.proxy.util.ProxyUtil.processLocalResponse(ProxyUtil.java:1670)
at com.ruijie.spl.radius.authen.process.RadiusProcessor.authProxyResponseProcess(RadiusProcessor.java:1529)
at com.ruijie.spl.radius.authen.process.RadiusProcessor.process(RadiusProcessor.java:404)
at com.ruijie.spl.common.net.radius.PacketProcessThread.runOnce(PacketProcessThread.java:68)
at com.ruijie.spl.common.power.AbstractDaemonThread.run(AbstractDaemonThread.java:157)
分析发现，认证线程都卡在这个锁的地方，分析代码，代码有个查找全部的方法，加了一个读写锁的读锁。
public List<StrategySam> findAll() {
lock.readLock().lock();
try {
LinkedList<StrategySam> result = new LinkedList<StrategySam>();
if(null!=strategySamMap.values()&& strategySamMap.values().size()>0){
result.addAll(strategySamMap.values());
}
return result;
} finally {
lock.readLock().unlock();
}
}查找整个线程dump，写锁并没有锁住，对于读写锁，在没有写锁的情况，读锁多线程可以共享，不应该存在性能瓶颈问题。
findall接口，查找全部并遍历，因为问题在这个地方，看出数据库中的数据，表中存在大量数据。
解法：算法优化。
故障二
故障现象
rsa加解密算法性能瓶颈
Thread: PeapEapAuthProcessorThread-19, State: BLOCKED
at sun.security.rsa.RSACore$BlindingParameters.getBlindingRandomPair(RSACore.java:404)
at sun.security.rsa.RSACore.getBlindingRandomPair(RSACore.java:445)
at sun.security.rsa.RSACore.crtCrypt(RSACore.java:178)
at sun.security.rsa.RSACore.rsa(RSACore.java:122)
at sun.security.rsa.RSASignature.engineSign(RSASignature.java:177)
at java.security.Signature$Delegate.engineSign(Signature.java:1205)
at java.security.Signature.sign(Signature.java:578)
at sun.security.ssl.HandshakeMessage$ECDH_ServerKeyExchange.<init>(HandshakeMessage.java:1012)
at sun.security.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:779)
at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:167)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:913)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:853)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:851)
at java.security.AccessController.doPrivileged(AccessController.java:-2)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1285)
at com.ruijie.spl.radius.authen.support.tls.TlsEngine.runSslTask(TlsEngine.java:278)
at com.ruijie.spl.radius.authen.support.tls.TlsEngine.doSSLHandshake(TlsEngine.java:217)
at com.ruijie.spl.radius.authen.support.tls.TlsEngine.process(TlsEngine.java:139)
at com.ruijie.spl.radius.authen.support.peap.PeapAuthEngine.doPeap(PeapAuthEngine.java:199)
at com.ruijie.spl.radius.authen.support.peap.PeapAuthEngine.process(PeapAuthEngine.java:81)
at com.ruijie.spl.radius.authen.support.peap.PeapAuthEngine.process(PeapAuthEngine.java:70)
at com.ruijie.spl.radius.authen.support.eap.EapAuthEngine.doInnerAuth(EapAuthEngine.java:176)
at com.ruijie.spl.radius.authen.support.eap.EapAuthEngine.doAuth(EapAuthEngine.java:136)
at com.ruijie.spl.radius.authen.support.eap.EapAuthEngine.doAuth(EapAuthEngine.java:59)
at com.ruijie.spl.radius.authen.support.eap.EapAuthEngine.doAuthen(EapAuthEngine.java:64)
at com.ruijie.spl.radius.authen.support.eap.AuthenRuleImpl_EAP_PEAP.doAuthen(AuthenRuleImpl_EAP_PEAP.java:136)
at com.ruijie.spl.radius.authen.support.RadiusAuthenRule.doAuthen(RadiusAuthenRule.java:58)
at com.ruijie.spl.radius.authen.support.peap.PeapEapAuthProcessorThread.runOnce(PeapEapAuthProcessorThread.java:166)
at com.ruijie.spl.common.power.AbstractDaemonThread.run(AbstractDaemonThread.java:156)分析发现，RSA算法加解密需要进行大量计算，JDK底层针对这个问题进行的优化策略是引入缓存。引入缓存后，为解决多线程并发计算的现场安全问题，在计算时进行加锁处理。导致了本次的性能瓶颈
优化方案
由于在业务场景下（无线1x认证，TLS通道加密），必须是RSA算法。同时在网上搜索许久也没有搜到源码。想是JDK真的这部分没有开放源代码。最终想到的是使用反射进行缓存替换，使用线程本地变量，将缓存进行扩充，解决多线程锁的性能消耗。
final Wireless1XWeakHashMap Wireless1XWeakHashMap = new Wireless1XWeakHashMap();
Field nameField1 = RSACore.class.getDeclaredField("blindingCache");
Field modifiersField1 = Field.class.getDeclaredField("modifiers");
modifiersField1.setAccessible(true);
modifiersField1.setInt(nameField1, nameField1.getModifiers() & ~Modifier.FINAL); //解决final无法重新赋值的问题
nameField1.setAccessible(true);
nameField1.set(RSACore.class, Wireless1XWeakHashMap);
System.out.println(nameField1.get(RSACore.class)); public class Wireless1XWeakHashMap extends WeakHashMap<BigInteger, Object>{
/***存储 RSACore。BlindingParameters 类型***/
ThreadLocal<WeakHashMap<BigInteger, Object>> longLocal = new ThreadLocal<WeakHashMap<BigInteger, Object>>();
public Object get(BigInteger key) {
WeakHashMap<BigInteger, Object> weakHashMap = longLocal.get();
if(weakHashMap == null) {
return null;
}
return weakHashMap.get(key);
}
public Object put(BigInteger key, Object value) {
WeakHashMap<BigInteger, Object> weakHashMap = longLocal.get();
if(weakHashMap == null) {
longLocal.set(new WeakHashMap<BigInteger, Object>());
}
weakHashMap = longLocal.get();
return weakHashMap.put(key, value);
}
}参考：http://blog.51cto.com/nxlhero/1832127
jtds + sqlserver2008r2的一次数据库连接被持续关闭的故障定位
故障现象
系统在运行时，整个系统的业务不定时不正常，包括web页面的访问，在不正常时，后台对应的错误日志为数据库连接被关闭
2018-10-09 20:49:50,029 ERROR [default_authen_thread10] com.ruijie.spl.radius.authen.process.RadiusProcessor -
org.springframework.transaction.TransactionSystemException: Could not roll back Hibernate transaction; nested exception is org.hibernate.TransactionException: JDBC rollback failed
at org.springframework.orm.hibernate3.HibernateTransactionManager.doRollback(HibernateTransactionManager.java:677)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:823)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:800)
at com.ruijie.spl.common.cache2.TransactionManagerWrapper.rollback(TransactionManagerWrapper.java:84)
at org.springframework.transaction.support.TransactionTemplate.rollbackOnException(TransactionTemplate.java:154)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:132)
at com.ruijie.spl.onlineuser.service.OnlineUserServiceImpl.directDeleteAndAuthLater(OnlineUserServiceImpl.java:2664)
at sun.reflect.GeneratedMethodAccessor2565.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
at com.sun.proxy.$Proxy196.directDeleteAndAuthLater(Unknown Source)
at com.ruijie.spl.radius.authen.support.DirectDeleteFalseOnlineUserAction.execution(DirectDeleteFalseOnlineUserAction.java:13)
at com.ruijie.spl.radius.authen.process.RadiusProcessor.process(RadiusProcessor.java:689)
at com.ruijie.spl.common.net.radius.PacketProcessThread.runOnce(PacketProcessThread.java:62)
at com.ruijie.spl.common.power.AbstractDaemonThread.run(AbstractDaemonThread.java:156)
Caused by: org.hibernate.TransactionException: JDBC rollback failed
at org.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction.java:204)
at org.springframework.orm.hibernate3.HibernateTransactionManager.doRollback(HibernateTransactionManager.java:674)
... 16 more
Caused by: java.sql.SQLException: Invalid state, the Connection object is closed.
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.checkOpen(ConnectionJDBC2.java:1699)
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.rollback(ConnectionJDBC2.java:2100)
at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.jdbcRollback(BaseWrapperManagedConnection.java:697)
at org.jboss.resource.adapter.jdbc.WrappedConnection.rollback(WrappedConnection.java:500)
at com.ruijie.spl.common.datasource.SplConnection.rollback(SplConnection.java:196)
at org.hibernate.transaction.JDBCTransaction.rollbackAndResetAutoCommit(JDBCTransaction.java:217)
at org.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction.java:196)
... 17 more故障分析
1、查看源码发现，jtds的数据库连接的状态通过boolean 成员变量closed进行控制，在网上查找资料，说1.2.5版本存在一些问题，于是借着这个机会将jtds的版本进行了一次升级。
2、下载jtds的1.3.1的源码进行分析，发现源码变化很大，原本的net.sourceforge.jtds.jdbc.ConnectionJDBC2已经变更为了net.sourceforge.jtds.jdbc.JtdsConnection。分析下源码，将关闭数据库连接的地方都加上调试日志
void setClosed()
synchronized public void close() throws SQLException
3、升级jtds到1.3.1，故障现象依旧，问题依然没有得到解决，但是本地升级机制的将日志加上了，开启log4j的日志，日志报错信息如下
2018-10-15 20:02:02,131 ERROR [default_authen_thread7] net.sourceforge.jtds.jdbc.JtdsConnection - ------------数据库被setClosed关闭-----------
java.lang.Exception
at net.sourceforge.jtds.jdbc.JtdsConnection.setClosed(JtdsConnection.java:1855)
at net.sourceforge.jtds.jdbc.TdsCore.tdsDoneToken(TdsCore.java:3698)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2472)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:677)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:613)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:572)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeUpdate(JtdsPreparedStatement.java:727)
at org.jboss.resource.adapter.jdbc.CachedPreparedStatement.executeUpdate(CachedPreparedStatement.java:96)
at org.jboss.resource.adapter.jdbc.WrappedPreparedStatement.executeUpdate(WrappedPreparedStatement.java:365)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:798)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:591)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:792)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:850)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:858)
at com.ruijie.spl.onlineuser.dao.OnlineUserDAOImpl.delete(OnlineUserDAOImpl.java:813)
at com.ruijie.spl.onlineuser.service.OnlineUserServiceImpl.deleteOnlineUser(OnlineUserServiceImpl.java:396)
at com.ruijie.spl.onlineuser.service.OnlineUserServiceImpl.dealUserLogout(OnlineUserServiceImpl.java:2481)
at com.ruijie.spl.onlineuser.service.OnlineUserServiceImpl.billAndLogout(OnlineUserServiceImpl.java:2230)
at sun.reflect.GeneratedMethodAccessor915.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy196.billAndLogout(Unknown Source)
at com.ruijie.spl.onlineuser.interf.OnlineUserForRadiusService.billAndLogout(OnlineUserForRadiusService.java:79)
at com.ruijie.spl.radius.author.face.InterfFromOnlineuserService$1.doInTransaction(InterfFromOnlineuserService.java:113)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:128)
at com.ruijie.spl.radius.author.face.InterfFromOnlineuserService.billAndLogout(InterfFromOnlineuserService.java:111)
at com.ruijie.spl.onlineuser.service.OnlineUserServiceImpl$4.doInTransaction(OnlineUserServiceImpl.java:2669)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:128)
at com.ruijie.spl.onlineuser.service.OnlineUserServiceImpl.directDeleteAndAuthLater(OnlineUserServiceImpl.java:2664)
at sun.reflect.GeneratedMethodAccessor984.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
at com.sun.proxy.$Proxy196.directDeleteAndAuthLater(Unknown Source)
at com.ruijie.spl.radius.authen.support.DirectDeleteFalseOnlineUserAction.execution(DirectDeleteFalseOnlineUserAction.java:13)
at com.ruijie.spl.radius.authen.process.RadiusProcessor.process(RadiusProcessor.java:689)
at com.ruijie.spl.common.net.radius.PacketProcessThread.runOnce(PacketProcessThread.java:62)
at com.ruijie.spl.common.power.AbstractDaemonThread.run(AbstractDaemonThread.java:156)根据日志信息，结合系统业务，当前执行的sql为
"DELETE FROM ONLINE_USER WHERE ONLINE_USER_UUID=?", new Object[]{onlineUser.getOnlineUserUuid()}分析sqlserver的日志,服务器上会相对应的记录如下错误信息
日期 2018/10/16 11:35:46
日志 SQL Server (存档编号1 - 2018/10/16 13:14:00)
源 spid57
消息
Database ID 5, Page (1:1729) is marked RestorePending, which may indicate disk corruption. To recover from this state, perform a restore.说明数据库完整性存在问题。使用如下脚本进行完整性修复
exec sp_dboption DB_, 'single_user', true
DBCC CHECKDB (DB_NAME, repair_allow_data_loss) with NO_INFOMSGS
exec sp_dboption DB_NAME, 'single_user', false修复完后
修复: 已为数据库 'DB_NAME' 中的对象 'dbo.ONLINE_USER, idx_online_user_nas' 成功地重新生成了 Nonclustered 索引。
修复: 页 (1:1729) 已从对象 ID 410484541，索引 ID 3，分区 ID 72057596299902976，分配单元 ID 72057596354363392 (类型为 In-row data)释放。
消息 8945，级别 16，状态 1，第 2 行
表错误: 将重新生成对象 ID 410484541，索引 ID 3。
该错误已修复。
消息 8928，级别 16，状态 1，第 2 行
对象 ID 410484541，索引 ID 3，分区 ID 72057596299902976，分配单元 ID 72057596354363392 (类型为 In-row data): 无法处理页 (1:1729)。有关详细信息，请参阅其他错误消息。
该错误已修复。
消息 8939，级别 16，状态 98，第 2 行
表错误: 对象 ID 410484541，索引 ID 3，分区 ID 72057596299902976，分配单元 ID 72057596354363392 (类型为 In-row data)，页 (1:1729)。测试(IS_OFF (BUF_IOERR, pBUF->bstat))失败。值为 12584969 和 -6。
该错误已修复。
消息 8976，级别 16，状态 1，第 2 行
表错误: 对象 ID 410484541，索引 ID 3，分区 ID 72057596299902976，分配单元 ID 72057596354363392 (类型为 In-row data)。在扫描过程中未发现页 (1:1729)，但该页的父级 (1:13283) 和上一页 (1:53014) 都引用了它。请检查以前的错误消息。
该错误已修复。
消息 8978，级别 16，状态 1，第 2 行
表错误: 对象 ID 410484541，索引 ID 3，分区 ID 72057596299902976，分配单元 ID 72057596354363392 (类型为 In-row data)。页 (1:53008) 缺少上一页 (1:1729) 对它的引用。可能是因为链链接有问题。
该错误已修复。至此，问题完美解决。
