lettuce客户端底层bug(-READONLY You can‘t write against a read only replica.)

本文涉及的产品
云数据库 Redis 版,社区版 2GB
推荐场景:
搭建游戏排行榜
简介: lettuce客户端底层、READONLY You can't write against a read only replica.

出现的现象:

这个问题只在 AWS elastic cache版本会重现

org.springframework.data.redis.RedisSystemException:
Error in execution; nested exception is io.lettuce.core.RedisCommandExecutionException:
ERR Error running script (call to f_e9f69f2beb755be68b5e456ee2ce9aadfbc4ebf4):
@user_script:1: @user_script: 1: -READONLY You can't write against a read only replica.  
        at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:54)
        at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:52)
        at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:41)
        at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:44)
        at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:42)
        at org.springframework.data.redis.connection.lettuce.LettuceConnection.convertLettuceAccessException(LettuceConnection.java:275)
        at org.springframework.data.redis.connection.lettuce.LettuceScriptingCommands.convertLettuceAccessException(LettuceScriptingCommands.java:236)
        at org.springframework.data.redis.connection.lettuce.LettuceScriptingCommands.evalSha(LettuceScriptingCommands.java:195)
        at org.springframework.data.redis.connection.DefaultedRedisConnection.evalSha(DefaultedRedisConnection.java:1502)
        at jdk.internal.reflect.GeneratedMethodAccessor225.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.data.redis.core.CloseSuppressingInvocationHandler.invoke(CloseSuppressingInvocationHandler.java:61)
        at com.sun.proxy.$Proxy248.evalSha(Unknown Source)
        at org.springframework.data.redis.core.script.DefaultScriptExecutor.eval(DefaultScriptExecutor.java:77)
        at org.springframework.data.redis.core.script.DefaultScriptExecutor.lambda$execute$0(DefaultScriptExecutor.java:68)
        at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:228)
        at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188)
        at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:175)
        at org.springframework.data.redis.core.script.DefaultScriptExecutor.execute(DefaultScriptExecutor.java:58)
        at org.springframework.data.redis.core.script.DefaultScriptExecutor.execute(DefaultScriptExecutor.java:52)
        at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:350)


官方关于这个bug的描述:

image.png

image.png

environment

  • Lettuce version(s): [5.0.3.RELEASE, 5.0.4.RELEASE,5.1.0.M1]
  • Redis version: AWS 6.0.5

官方case:

https://stackoverflow.com/questions/62992129/lettuce-readonly-you-cant-write-against-a-read-only-slave

https://github.com/lettuce-io/lettuce-core/issues/804


问题原因:

ReadFrom类详情配置:https://lettuce.io/core/6.1.4.RELEASE/api/index.html

Modifier and Type   Field and Description
static ReadFrom ANY                     Setting to read from any node.
static ReadFrom ANY_REPLICA             Setting to read from any replica node.
static ReadFrom MASTER                  Setting to read from the upstream only.
static ReadFrom MASTER_PREFERRED        Setting to read preferred from the upstream and fall back to a replica if the master is not available.
static ReadFrom NEAREST                 Setting to read from the nearest node.
static ReadFrom REPLICA                 Setting to read from the replica only.
static ReadFrom REPLICA_PREFERRED       Setting to read preferred from replica and fall back to upstream if no replica is not available.
static ReadFrom SLAV                    EDeprecated. renamed to REPLICA.
static ReadFrom SLAVE_PREFERRED         Deprecated. Renamed to REPLICA_PREFERRED.
static ReadFrom UPSTREAM                Setting to read from the upstream only.
static ReadFrom UPSTREAM_PREFERRED      Setting to read preferred from the upstream and fall back to a replica if the upstream is not available.

查看读写分离的读取节点

io.lettuce.core.masterslave.MasterSlaveConnectionProvider
 public CompletableFuture<StatefulRedisConnection<K, V>> getConnectionAsync(MasterSlaveConnectionProvider.Intent intent) {
        if (this.debugEnabled) {
            logger.debug("getConnectionAsync(" + intent + ")");
        }
        if (this.readFrom != null && intent == MasterSlaveConnectionProvider.Intent.READ) {
            List<RedisNodeDescription> selection = this.readFrom.select(new Nodes() {
                public List<RedisNodeDescription> getNodes() {
                    return MasterSlaveConnectionProvider.this.knownNodes;
                }
                public Iterator<RedisNodeDescription> iterator() {
                    return MasterSlaveConnectionProvider.this.knownNodes.iterator();
                }
            });
            if (selection.isEmpty()) {
                throw new RedisException(String.format("Cannot determine a node to read (Known nodes: %s) with setting %s", this.knownNodes, this.readFrom));
            } else {
                try {
                    Flux<StatefulRedisConnection<K, V>> connections = Flux.empty();
                    RedisNodeDescription node;
                    for(Iterator var4 = selection.iterator(); var4.hasNext(); connections = connections.concatWith(Mono.fromFuture(this.getConnection(node)))) {
                        node = (RedisNodeDescription)var4.next();
                    }
                    return !OrderingReadFromAccessor.isOrderSensitive(this.readFrom) && selection.size() != 1 ? connections.filter(StatefulConnection::isOpen).collectList().map((it) -> {
                        int index = ThreadLocalRandom.current().nextInt(it.size());
                        return (StatefulRedisConnection)it.get(index);
                    }).switchIfEmpty(connections.next()).toFuture() : connections.filter(StatefulConnection::isOpen).next().switchIfEmpty(connections.next()).toFuture();
                } catch (RuntimeException var6) {
                    throw Exceptions.bubble(var6);
                }
            }
        } else {
            return this.getConnection(this.getMaster());
        }
    }


rredisTemplate.execute方法在执行lua脚本时,使用的是redis的evalsha方法,默认算读操作(io.lettuce.core.masterslave.ReadOnlyCommands.isReadOnlyCommand方法中有体现),

所以被分配到了slave节点,而lua脚本一般读写并存操作,比如以上代码释放redis锁,而redis的slave节点不能进行write操作,所以报以下错误


// eval 和 evalsha是redis执行脚本的方法,

EVAL,EVALSHA



解决方案:


1、重写lettuce底层源码ReadOnlyCommands中CommandName枚举类,将 EVAL, EVALSHA从此类删除

  • 找到你所要重写类,查看其中的路径;io.lettuce.core.masterslave.ReadOnlyCommands
  • 在我们的 src 目录下新建一个同包名同类名的类;
  • 将 jar 包中的重写方法所在类的所有代码复制到我们新建的同包名同类名的类中;
  • 在CommandName枚举中删除EVAL, EVALSHA属性
  • 程序会优先使用我们 src 下面的类,这样就覆盖了 jar 包的方法 。

2、升级lettuce-core到最新的版本,新版本已经解决。


相关实践学习
基于Redis实现在线游戏积分排行榜
本场景将介绍如何基于Redis数据库实现在线游戏中的游戏玩家积分排行榜功能。
云数据库 Redis 版使用教程
云数据库Redis版是兼容Redis协议标准的、提供持久化的内存数据库服务,基于高可靠双机热备架构及可无缝扩展的集群架构,满足高读写性能场景及容量需弹性变配的业务需求。 产品详情:https://www.aliyun.com/product/kvstore &nbsp; &nbsp; ------------------------------------------------------------------------- 阿里云数据库体验:数据库上云实战 开发者云会免费提供一台带自建MySQL的源数据库&nbsp;ECS 实例和一台目标数据库&nbsp;RDS实例。跟着指引,您可以一步步实现将ECS自建数据库迁移到目标数据库RDS。 点击下方链接,领取免费ECS&amp;RDS资源,30分钟完成数据库上云实战!https://developer.aliyun.com/adc/scenario/51eefbd1894e42f6bb9acacadd3f9121?spm=a2c6h.13788135.J_3257954370.9.4ba85f24utseFl
相关文章
|
1月前
|
存储 C++
webserver--Buffer类实现内存缓冲区读写(全代码)
webserver--Buffer类实现内存缓冲区读写(全代码)
|
8月前
|
存储 缓存
【什么是Read Write Through机制】
【什么是Read Write Through机制】
|
数据挖掘 Python
4.【.netcore Configuration】理解Configuration中Section弱方式读取和Binder模式的强类型读取代码
4.【.netcore Configuration】理解Configuration中Section弱方式读取和Binder模式的强类型读取代码
89 0
|
缓存 Linux 数据安全/隐私保护
实战分享|Write Cache设置效果为何有差异?
sdparm和hdparm去修改HDD的write cache,发现在系统下write cache设置的效果有差异。
mode:类型非常多 r:只读 从头部开始读 io.UnsupportedOperation: not writable w:写入 每次都是从头部开始写 原有的内容
mode:类型非常多 r:只读 从头部开始读 io.UnsupportedOperation: not writable w:写入 每次都是从头部开始写 原有的内容
如何处理UI5一般性错误Cannot read property md of undefined
如何处理UI5一般性错误Cannot read property md of undefined
174 0
如何处理UI5一般性错误Cannot read property md of undefined
|
弹性计算 NoSQL 网络协议
短连接访问Redis报错Cannot assign requested address解决方案
“短连接访问Redis报错Cannot assign requested address”,出现这种错误的应用程序使用的架构基本都是php-fpm+phpredis。并发较大的情况下,处于TIME-WAIT状态下的TCP连接较多,客户端无法分配出新的端口,报错Cannot assign requested address。下文针对这种情况给出两种解决方案。
2481 0
|
弹性计算 安全 关系型数据库
PostgreSQL 12 preview - 可靠性提升 - data_sync_retry 消除os层write back failed status不可靠的问题
标签 PostgreSQL , data_sync_retry , write back , retry , failed status 背景 有些OS系统,对fsync的二次调用不敏感,因为OS层可能有自己的CACHE,如果使用了buffer write,并且出现write back failed的情况,有些OS可能在下次fsync时并不能正确的反馈fsync的可靠性与否。(因为这个B
534 0
Hyperledger Fabric Read-Write set semantics——读写集
Read-Write set semantics(读写集) 本文讨论了关于读写集当前实现的细节。   Transaction simulation and read-write set(事务模拟和读写集) 客户端提交事务到peer,peer会执行背书验证并模拟该事务的请求结果,为该事务的请求准备一个读写集。
1599 0
|
Oracle 关系型数据库 数据库