redis.conf翻译与配置(四)【redis6.0.6】

本文涉及的产品
云数据库 Tair(兼容Redis),内存型 2GB
Redis 开源版,标准版 2GB
推荐场景:
搭建游戏排行榜
日志服务 SLS,月写入数据量 50GB 1个月
简介:
学习redis的途中,碰上了redis.conf,突发奇想,想着来进行一波翻译输出。
源码之前,了无秘密。
在这里插入图片描述

@[toc]

慢删除

原文

############################# LAZY FREEING ####################################

# Redis has two primitives to delete keys. One is called DEL and is a blocking
# deletion of the object. It means that the server stops processing new commands
# in order to reclaim all the memory associated with an object in a synchronous
# way. If the key deleted is associated with a small object, the time needed
# in order to execute the DEL command is very small and comparable to most other
# O(1) or O(log_N) commands in Redis. However if the key is associated with an
# aggregated value containing millions of elements, the server can block for
# a long time (even seconds) in order to complete the operation.
#
# For the above reasons Redis also offers non blocking deletion primitives
# such as UNLINK (non blocking DEL) and the ASYNC option of FLUSHALL and
# FLUSHDB commands, in order to reclaim memory in background. Those commands
# are executed in constant time. Another thread will incrementally free the
# object in the background as fast as possible.
#
# DEL, UNLINK and ASYNC option of FLUSHALL and FLUSHDB are user-controlled.
# It's up to the design of the application to understand when it is a good
# idea to use one or the other. However the Redis server sometimes has to
# delete keys or flush the whole database as a side effect of other operations.
# Specifically Redis deletes objects independently of a user call in the
# following scenarios:
#
# 1) On eviction, because of the maxmemory and maxmemory policy configurations,
#    in order to make room for new data, without going over the specified
#    memory limit.
# 2) Because of expire: when a key with an associated time to live (see the
#    EXPIRE command) must be deleted from memory.
# 3) Because of a side effect of a command that stores data on a key that may
#    already exist. For example the RENAME command may delete the old key
#    content when it is replaced with another one. Similarly SUNIONSTORE
#    or SORT with STORE option may delete existing keys. The SET command
#    itself removes any old content of the specified key in order to replace
#    it with the specified string.
# 4) During replication, when a replica performs a full resynchronization with
#    its master, the content of the whole database is removed in order to
#    load the RDB file just transferred.
#
# In all the above cases the default is to delete objects in a blocking way,
# like if DEL was called. However you can configure each case specifically
# in order to instead release memory in a non-blocking way like if UNLINK
# was called, using the following configuration directives.

lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no

# It is also possible, for the case when to replace the user code DEL calls
# with UNLINK calls is not easy, to modify the default behavior of the DEL
# command to act exactly like UNLINK, using the following configuration
# directive:

lazyfree-lazy-user-del no

译文

Redis有两个原语来删除键。

一个是DEL,它是对象的阻塞删除。这意味着服务器停止处理新命令,以同步方式收回与一个对象关联的所有内存。
如果删除的键与一个小对象相关联,那么执行DEL命令所需的时间非常小,可以与Redis中的大多数O(1)或O(log_N)命令相媲美。
但是,如果键与包含数百万个元素的聚合值相关联,服务器可能会阻塞很长时间(甚至几秒钟)以完成操作。

基于上述原因,Redis还提供了非阻塞删除原语,如UNLINK(非阻塞DEL)和异步选项的FLUSHALL和FLUSHDB命令,以便在后台回收内存。这些命令在固定的时间内执行。另一个线程将在后台以尽可能快的速度递增地释放对象。

FLUSHALL 和 FLUSHDB 的删除选项(DEL, UNLINK 和 ASYNC)是有使用者选定的。

何时使用哪一种会比较好,这取决于应用程序的设计。
然而,有时由于其他操作的副作用,redis不得不删除或刷新整个数据库。

具体来说,在以下场景,redis将自行删除数据对象:

1)在回收时,由于maxmemory和maxmemory策略配置,为了给新数据腾出空间,而不超过指定的内存限制。
2)因为过期:当必须从内存中删除一个具有关联生存时间的键(参见expire命令)时。
3)由于一个命令的副作用,该命令将数据存储在一个可能已经存在的键上。例如,RENAME命令可能会在用另一个键替换旧键内容时删除它。类似的,SUNIONSTORE或SORT with STORE选项可以删除现有的键。SET命令本身删除指定键的任何旧内容,以便将其替换为指定的字符串。
4)在复制过程中,当一个副本与它的主服务器执行完全的重新同步时,整个数据库的内容被删除,以便加载刚刚传输的RDB文件。

在上述所有情况下,默认情况是以阻塞的方式删除对象,比如调用DEL。但是,您可以具体配置每种情况,以便以非阻塞的方式释放内存,比如在UNLINK被调用时,使用以下配置指令:

lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no

也有可能,当用UNLINK调用替换用户代码DEL调用不那么顺利的情况下,可以修改DEL命令的默认行为,使其完全像UNLINK一样,使用以下配置指令:

lazyfree-lazy-user-del no

线程I/O

原文

################################ THREADED I/O #################################

# Redis is mostly single threaded, however there are certain threaded
# operations such as UNLINK, slow I/O accesses and other things that are
# performed on side threads.
#
# Now it is also possible to handle Redis clients socket reads and writes
# in different I/O threads. Since especially writing is so slow, normally
# Redis users use pipelining in order to speedup the Redis performances per
# core, and spawn multiple instances in order to scale more. Using I/O
# threads it is possible to easily speedup two times Redis without resorting
# to pipelining nor sharding of the instance.
#
# By default threading is disabled, we suggest enabling it only in machines
# that have at least 4 or more cores, leaving at least one spare core.
# Using more than 8 threads is unlikely to help much. We also recommend using
# threaded I/O only if you actually have performance problems, with Redis
# instances being able to use a quite big percentage of CPU time, otherwise
# there is no point in using this feature.
#
# So for instance if you have a four cores boxes, try to use 2 or 3 I/O
# threads, if you have a 8 cores, try to use 6 threads. In order to
# enable I/O threads use the following configuration directive:
#
# io-threads 4
#
# Setting io-threads to 1 will just use the main thread as usually.
# When I/O threads are enabled, we only use threads for writes, that is
# to thread the write(2) syscall and transfer the client buffers to the
# socket. However it is also possible to enable threading of reads and
# protocol parsing using the following configuration directive, by setting
# it to yes:
#
# io-threads-do-reads no
#
# Usually threading reads doesn't help much.
#
# NOTE 1: This configuration directive cannot be changed at runtime via
# CONFIG SET. Aso this feature currently does not work when SSL is
# enabled.
#
# NOTE 2: If you want to test the Redis speedup using redis-benchmark, make
# sure you also run the benchmark itself in threaded mode, using the
# --threads option to match the number of Redis theads, otherwise you'll not
# be able to notice the improvements.

译文

Redis大部分是单线程的,但是也有一些线程操作,如解除链接,缓慢的I/O访问和一些其他事情是在侧线程上执行的。
现在它也可以在不同的I/O线程下去处理Redis客户端套接字读写。特别是“写”过程是如此的慢吞吞,通常Redis用户使用管道通信来提高每个核的Redis性能,并衍生多个实例来扩大规模。
使用I/O线程,它可以很容易地加速两倍的Redis没有诉诸于管道或切分的实例。

默认情况下,线程是禁用的,我们建议仅在拥有至少4个或更多核的机器上启用它,至少留下一个备用核。
使用超过8个线程多半没什么用。我们还建议只有在你确实有性能问题的时候才使用线程I/O,因为Redis实例会占用相当大百分比的CPU,所以没事儿开些线程出来玩没必要。
例如,如果你有4核,可以尝试使用2或3个I/O线程,如果你有8核,可以尝试使用6个线程。

以下配置指令开启线程之旅:# io-threads 4

将io-threads设置为1只会像往常一样使用主线程。当启用I/O线程时,我们只使用线程进行写操作,因为对线程来说,write(2)系统调用和传输客户端缓冲区到套接字,然而,也有可能启用线程读取和协议解析使用以下配置指令,通过将它设置为yes:# io-threads-do-reads no

通常情况下线程拿去读没什么意义。

注意1:这个配置指令不能在运行时通过配置设置来改变。当SSL被启用时,这个特性不能用。
注2:如果你想用Redis -benchmark测试Redis加速,请确保你也在线程模式下运行基准测试,使用——threads选项来匹配Redis头的数量,否则你将无法注意到改进。


追加持久化(只附加模式)

原文

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes

译文

默认情况下,Redis异步转储数据集到磁盘上。这种模式在许多应用程序中已经足够好了,但Redis进程的问题或断电可能会导致几分钟的写丢失(取决于配置的保存点)。

Append Only文件是另一种持久性模式,它提供了更好的持久性。例如使用默认数据fsync策略配置文件中(见后),Redis可以在一个意外事件中失去一秒的写操作,比如服务器断电,或者如果Redis进程本身发生了什么问题,一个写操作,但操作系统仍然正常运行。

AOF和RDB持久性可以同时启用而没有问题。如果在启动时启用了AOF, Redis将加载AOF,这是一个更好的持久性保障。
详情见:http://redis.io/topics/persistence

append only 文件名:"appendonly.aof"


fsync()调用告诉操作系统在磁盘上实际写入数据,而不是等待输出缓冲区中的更多数据。一些操作系统会真正刷新磁盘上的数据,而另一些操作系统会尝试尽快这么做。

Redis支持以下三种不同的模式:

no: 不fsync,只要让操作系统在需要的时候刷新数据即可。尽快。
always: fsync在每次写入append only日志后执行。慢点,安全第一。
everysec:一秒执行一次fsync,折中。

默认值是“每秒钟”,因为这通常是速度和数据安全之间的折中。如果你能处理好这个“不”字,让操作系统刷新输出缓冲区时,为了更好的表现(但是如果你可以忍受一些数据丢失的),那你可以选“不”。或相反,使用“总是”非常缓慢但比everysec更安全一点。

更多细节参见:http://antirez.com/post/redis-persistence-demystified.html

当AOF fsync策略设置为always或everysec,并且后台保存进程(后台保存或AOF日志后台重写)对磁盘执行大量I/O操作时,在一些Linux配置中,Redis可能会在fsync()调用上阻塞太长时间。注意,这个问题目前还没有解决办法,因为即使在不同的线程中执行fsync,也会阻塞我们的同步write(2)调用。

为了减轻这个问题,可以使用以下选项来防止在执行BGSAVE或BGREWRITEAOF时在主进程中调用fsync()。
这意味着,当另一个子对象正在保存,Redis的持久性与“appendfsync none”相同。实际上,这意味着在最坏的情况下(使用默认的Linux设置)可能会丢失最多30秒的日志。

如果有延迟问题,请将此选项转换为“yes”。否则,从耐久性的角度来看,这是最安全的选择。


自动重写仅追加的文件:
当AOF日志大小增长指定的百分比时,Redis会隐式调用BGREWRITEAOF,自动重写日志文件。

它的工作模式是:Redis会记住最近一次重写后的AOF文件的大小(如果重启后没有重写,则使用启动时的AOF文件的大小)。

如果当前大小大于指定的百分比,就会触发重写。您还需要指定要重写的AOF文件的最小大小,这有助于避免重写AOF文件,即使达到百分比增长,但它仍然很小。
指定一个百分比为零,以禁用自动AOF重写功能。


在redis启动期间,可能会发现aof文件在结尾处被截断 ,这可能会发生在Redis运行的系统崩溃时,特别是ext4文件系统在没有data=ordered选项的情况下挂载(然而,这不会发生在Redis本身崩溃或中止但操作系统仍然正常工作时)。

发生这种情况时,redis可以退出并返回错误,或者加载尽可能多的数据。
以下选项控制此行为:
如果AOF -load-truncated被设置为yes,出现AOF文件加载被截断时Redis会发送一个日志来提醒用户。否则,如果将该选项设置为no,服务器将以错误终止并拒绝启动。当选项设置为no时,用户需要在重启服务器之前使用“redis-check-aof”程序修复AOF文件。

注意,如果发现AOF文件在中间损坏,服务器仍然会退出并出现错误。这个选项只适用于当Redis将尝试读取更多的数据从AOF文件,但没有足够的字节时生效。


当重写AOF文件,Redis能够使用一个RDB序言将AOF文件更快的重写和恢复。当这个选项被打开时,重写的AOF文件由两个不同的节组成:[RDB file][AOF tail]

加载Redis时,会识别出AOF文件以“Redis”字符串开头并加载带前缀的RDB文件,然后继续加载AOF尾部。


LUA脚本

原文

################################ LUA SCRIPTING  ###############################

# Max execution time of a Lua script in milliseconds.
#
# If the maximum execution time is reached Redis will log that a script is
# still in execution after the maximum allowed time and will start to
# reply to queries with an error.
#
# When a long running script exceeds the maximum execution time only the
# SCRIPT KILL and SHUTDOWN NOSAVE commands are available. The first can be
# used to stop a script that did not yet called write commands. The second
# is the only way to shut down the server in the case a write command was
# already issued by the script but the user doesn't want to wait for the natural
# termination of the script.
#
# Set it to 0 or a negative value for unlimited execution without warnings.
lua-time-limit 5000

译文

Lua脚本的最长执行时间(毫秒)。

如果达到最大执行时间,Redis将记录在允许的最长时间之后脚本仍在执行中,并将开始以错误的方式答复查询。

当长时间运行的脚本超过最大执行时间时,只有脚本KILL和SHUTDOWN NOSAVE命令可用。第一个命令可用于停止尚未调用write命令的脚本。第二种方法是在脚本已经发出write命令但用户不想等待脚本自然终止的情况下关闭服务器的唯一方法。

将其设置为0或负值,以无限制地执行而不发出警告。

相关实践学习
基于Redis实现在线游戏积分排行榜
本场景将介绍如何基于Redis数据库实现在线游戏中的游戏玩家积分排行榜功能。
云数据库 Redis 版使用教程
云数据库Redis版是兼容Redis协议标准的、提供持久化的内存数据库服务,基于高可靠双机热备架构及可无缝扩展的集群架构,满足高读写性能场景及容量需弹性变配的业务需求。 产品详情:https://www.aliyun.com/product/kvstore     ------------------------------------------------------------------------- 阿里云数据库体验:数据库上云实战 开发者云会免费提供一台带自建MySQL的源数据库 ECS 实例和一台目标数据库 RDS实例。跟着指引,您可以一步步实现将ECS自建数据库迁移到目标数据库RDS。 点击下方链接,领取免费ECS&RDS资源,30分钟完成数据库上云实战!https://developer.aliyun.com/adc/scenario/51eefbd1894e42f6bb9acacadd3f9121?spm=a2c6h.13788135.J_3257954370.9.4ba85f24utseFl
相关文章
|
16天前
|
存储 SQL 关系型数据库
2024Mysql And Redis基础与进阶操作系列(1)作者——LJS[含MySQL的下载、安装、配置详解步骤及报错对应解决方法]
Mysql And Redis基础与进阶操作系列(1)之[MySQL的下载、安装、配置详解步骤及报错对应解决方法]
|
28天前
|
存储 NoSQL Redis
Redis 配置
10月更文挑战第14天
24 1
|
1月前
|
存储 缓存 NoSQL
大数据-46 Redis 持久化 RDB AOF 配置参数 混合模式 具体原理 触发方式 优点与缺点
大数据-46 Redis 持久化 RDB AOF 配置参数 混合模式 具体原理 触发方式 优点与缺点
56 1
|
1月前
|
消息中间件 NoSQL Kafka
大数据-116 - Flink DataStream Sink 原理、概念、常见Sink类型 配置与使用 附带案例1:消费Kafka写到Redis
大数据-116 - Flink DataStream Sink 原理、概念、常见Sink类型 配置与使用 附带案例1:消费Kafka写到Redis
129 0
|
1月前
|
NoSQL Ubuntu Linux
redis的基本安装配置启动使用
redis的基本安装配置启动使用
34 0
|
1月前
|
缓存 NoSQL 数据处理
原生php实现redis缓存配置和使用方法
通过上述步骤,你可以在PHP项目中配置并使用Redis作为高性能的缓存解决方案。合理利用Redis的各种数据结构和特性,可以有效提升应用的响应速度和数据处理效率。记得在实际应用中根据具体需求选择合适的缓存策略,如设置合理的过期时间,以避免内存过度消耗。
49 0
|
3月前
|
NoSQL Redis 容器
【Azure Cache for Redis】Redis的导出页面无法配置Storage SAS时通过az cli来完成
【Azure Cache for Redis】Redis的导出页面无法配置Storage SAS时通过az cli来完成
|
1月前
|
消息中间件 缓存 NoSQL
Redis 是一个高性能的键值对存储系统,常用于缓存、消息队列和会话管理等场景。
【10月更文挑战第4天】Redis 是一个高性能的键值对存储系统,常用于缓存、消息队列和会话管理等场景。随着数据增长,有时需要将 Redis 数据导出以进行分析、备份或迁移。本文详细介绍几种导出方法:1)使用 Redis 命令与重定向;2)利用 Redis 的 RDB 和 AOF 持久化功能;3)借助第三方工具如 `redis-dump`。每种方法均附有示例代码,帮助你轻松完成数据导出任务。无论数据量大小,总有一款适合你。
74 6
|
6天前
|
缓存 NoSQL 关系型数据库
大厂面试高频:如何解决Redis缓存雪崩、缓存穿透、缓存并发等5大难题
本文详解缓存雪崩、缓存穿透、缓存并发及缓存预热等问题,提供高可用解决方案,帮助你在大厂面试和实际工作中应对这些常见并发场景。关注【mikechen的互联网架构】,10年+BAT架构经验倾囊相授。
大厂面试高频:如何解决Redis缓存雪崩、缓存穿透、缓存并发等5大难题
|
7天前
|
存储 缓存 NoSQL
【赵渝强老师】基于Redis的旁路缓存架构
本文介绍了引入缓存后的系统架构,通过缓存可以提升访问性能、降低网络拥堵、减轻服务负载和增强可扩展性。文中提供了相关图片和视频讲解,并讨论了数据库读写分离、分库分表等方法来减轻数据库压力。同时,文章也指出了缓存可能带来的复杂度增加、成本提高和数据一致性问题。
【赵渝强老师】基于Redis的旁路缓存架构