发现跑了半天的game server登录不了，简单jstack了下，发现登录线程 game-bus_init-X 被堵塞了，一直堵在抛出登录事件上，原因是
Affects AAffects Version:3.3.2 Fix version:3.3.6
JDK 1.8
Cause: Nope this is not resolved by dropping the events. Ring Buffer Works on Producer and Consumer Thread Model.This issue actually happened due to signal missed by consumer. As per LMAX disruptor signal miss should not happen.
这不是通过删除事件来解决。 环形缓冲区在生产者和消费者线程模型上工作。这个问题实际上是由于消费者错过信号而发生的。 根据LMAX中断器信号缺失不应该发生。
Solution:by mikeb01
I added support for re-signalling the listeners if the producer encounters a full ring buffer, so if the signal happened to be lost somehow (not convinced that it can be) then it will be on the next attempted publication.
如果生产者遇到一个已塞满的环形缓冲区，我添加了对重新发信号通知监听器的支持，所以如果信号偶然丢失（不确信是不是这个原因），那么它将在下一次重新发布。
If it detects that the ring buffer is full it will re-signal the wait strategy, this should unstick the consumer if it happened to miss a signal.
如果它检测到环形缓冲区已满，它将重发信号的等待策略，如果它碰巧错过一个信号，这应该解锁消费者。
也可以改用Add LiteTimeoutBlockingWaitStrategy
"game-bus_init-3" #69 daemon prio=5 os_prio=0 tid=0x00007f9ba1cf1800 nid=0x1113 waiting on condition [0x00007f9af25e8000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000d5054d20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45) at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:123) at java.lang.Thread.run(Thread.java:745) "game-bus_init-2" #68 daemon prio=5 os_prio=0 tid=0x00007f9ba1cef800 nid=0x1112 runnable [0x00007f9af26e8000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105) at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246) at com.lingyu.common.message.BalanceBusinessExecutor$ExecutorPoolGroup$Executor.execute(BalanceBusinessExecutor.java:156) at com.lingyu.game.service.event.EventManager.asyncCall(EventManager.java:84) at com.lingyu.game.service.event.AbEvent.advancedAsyncCall(AbEvent.java:114) at com.lingyu.game.service.event.AbEvent.dispatch(AbEvent.java:73) at com.lingyu.game.service.event.LoginGameEvent.publish(LoginGameEvent.java:130) at com.xianling.public_.nodecontrol.service.impl.NodeControlServiceImpl.login(NodeControlServiceImpl.java:55) at com.lingyu.game.action.bus.LoginAction.loginGame(LoginAction.java:254) at com.lingyu.game.action.bus.LoginActionMethodAccess.invoke(Unknown Source) at com.lingyu.common.message.MessageResolver.execute(MessageResolver.java:46) at com.lingyu.common.message.MessageMediator.execute(MessageMediator.java:75) at com.lingyu.common.message.ExecutorRunnablePool$RunnableImpl.run(ExecutorRunnablePool.java:87) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:20) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:11) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.lang.Thread.run(Thread.java:745) "game-bus_init-1" #67 daemon prio=5 os_prio=0 tid=0x00007f9ba1ced800 nid=0x1111 runnable [0x00007f9af27e9000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105) at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246) at com.lingyu.common.message.BalanceBusinessExecutor$ExecutorPoolGroup$Executor.execute(BalanceBusinessExecutor.java:156) at com.lingyu.game.service.event.EventManager.asyncCall(EventManager.java:84) at com.lingyu.game.service.event.AbEvent.advancedAsyncCall(AbEvent.java:114) at com.lingyu.game.service.event.AbEvent.dispatch(AbEvent.java:73) at com.lingyu.game.service.event.LoginGameEvent.publish(LoginGameEvent.java:130) at com.xianling.public_.nodecontrol.service.impl.NodeControlServiceImpl.login(NodeControlServiceImpl.java:55) at com.lingyu.game.action.bus.LoginAction.loginGame(LoginAction.java:254) at com.lingyu.game.action.bus.LoginActionMethodAccess.invoke(Unknown Source) at com.lingyu.common.message.MessageResolver.execute(MessageResolver.java:46) at com.lingyu.common.message.MessageMediator.execute(MessageMediator.java:75) at com.lingyu.common.message.ExecutorRunnablePool$RunnableImpl.run(ExecutorRunnablePool.java:87) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:20) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:11) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.lang.Thread.run(Thread.java:745) "game-bus_init-0" #66 daemon prio=5 os_prio=0 tid=0x00007f9ba1ceb800 nid=0x1110 runnable [0x00007f9af28eb000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105) at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246) at com.lingyu.common.message.BalanceBusinessExecutor$ExecutorPoolGroup$Executor.execute(BalanceBusinessExecutor.java:156) at com.lingyu.game.service.event.EventManager.asyncCall(EventManager.java:84) at com.lingyu.game.service.event.AbEvent.advancedAsyncCall(AbEvent.java:114) at com.lingyu.game.service.event.AbEvent.dispatch(AbEvent.java:73) at com.lingyu.game.service.event.LoginGameEvent.publish(LoginGameEvent.java:130) at com.xianling.public_.nodecontrol.service.impl.NodeControlServiceImpl.login(NodeControlServiceImpl.java:55) at com.lingyu.game.action.bus.LoginAction.loginGame(LoginAction.java:254) at com.lingyu.game.action.bus.LoginActionMethodAccess.invoke(Unknown Source) at com.lingyu.common.message.MessageResolver.execute(MessageResolver.java:46) at com.lingyu.common.message.MessageMediator.execute(MessageMediator.java:75) at com.lingyu.common.message.ExecutorRunnablePool$RunnableImpl.run(ExecutorRunnablePool.java:87) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:20) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:11) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.lang.Thread.run(Thread.java:745) "game-bus_cache-3" #65 daemon prio=5 os_prio=0 tid=0x00007f9ba1cea000 nid=0x110f waiting on condition [0x00007f9af29ec000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000d5054d80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45) at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:123) at java.lang.Thread.run(Thread.java:745) "game-bus_cache-2" #64 daemon prio=5 os_prio=0 tid=0x00007f9ba1ce8000 nid=0x110e waiting on condition [0x00007f9af2aed000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000d5054db0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45) at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:123) at java.lang.Thread.run(Thread.java:745) "game-bus_cache-1" #63 daemon prio=5 os_prio=0 tid=0x00007f9ba1ce6000 nid=0x110d waiting on condition [0x00007f9af2bee000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000d50e2650> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45) at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:55) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:123) at java.lang.Thread.run(Thread.java:745) "game-bus_cache-0" #62 daemon prio=5 os_prio=0 tid=0x00007f9ba1ce4000 nid=0x110c runnable [0x00007f9af2cee000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136) at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105) at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246) at com.lingyu.common.message.BalanceBusinessExecutor$ExecutorPoolGroup$Executor.execute(BalanceBusinessExecutor.java:156) at com.lingyu.game.service.event.EventManager.asyncCall(EventManager.java:84) at com.lingyu.game.service.event.AbEvent.advancedAsyncCall(AbEvent.java:114) at com.lingyu.game.service.event.AbEvent.dispatch(AbEvent.java:73) at com.lingyu.game.service.event.MoneyIncrEvent.publish(MoneyIncrEvent.java:65) at com.lingyu.game.service.role.RoleManager.incrCoin(RoleManager.java:2369) at com.lingyu.game.service.role.CoinMoneyOperation.incr(CoinMoneyOperation.java:19) at com.lingyu.game.service.money.MoneyManager.incr(MoneyManager.java:203) - locked <0x00000000ebf8eaa8> (a [Ljava.lang.Byte;) at com.lingyu.game.service.money.MoneyManager.incr(MoneyManager.java:136) at com.lingyu.game.service.money.MoneyManager.incr(MoneyManager.java:171) at com.lingyu.game.action.bus.RoleAction.addMoney(RoleAction.java:906) at com.lingyu.game.action.bus.RoleActionMethodAccess.invoke(Unknown Source) at com.lingyu.common.message.MessageResolver.execute(MessageResolver.java:35) at com.lingyu.common.message.MessageMediator.execute(MessageMediator.java:69) at com.lingyu.common.message.ExecutorRunnablePool$RunnableImpl.run(ExecutorRunnablePool.java:85) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:20) at com.lingyu.common.message.GameEventHandler.onEvent(GameEventHandler.java:11) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.lang.Thread.run(Thread.java:745)
参考文章:
Dead lock observed in BlockingWaitStrategy
Dead lock observed in BlockingWaitStrategy in Log4J
Add LiteTimeoutBlockingWaitStrategy
后发现又重现这个问题了，原来是因为待处理的事件超过了ringbuff size的上限，导致事件又被覆盖导致了，无法处理。
