buffer & cache

简介: In Linux, what is the difference between “buffers” and “cache” reported by the free command? This is an old question that I've seen from time to time. My understanding of it is rather

奋斗

In Linux, what is the difference between “buffers” and “cache” reported by the free command?

This is an old question that I've seen from time to time. My understanding of it is rather limited (having read about the differences a long time ago, but the factoid(s) involved never really stuck).

As I understand it,

  • Buffers

    Are used by programs with active I/O operations, i.e. data waiting to be written to disk

  • Cache

    Is the result of completed I/O operations, i.e. buffers that have been flushed or data read from disk to satisfy a request.


The "cached" total will also include some other memory allocations, such as any tmpfs filesytems. To see this in effect try:

mount -ttmpfs none t
dd if=dev/zero of=t/zero.file bs=10240 count=10240
sync; echo 3 > /proc/sys/vm/drop_caches; free -m
umount t
sync; echo 3 > /proc/sys/vm/drop_caches; free -m

and you will see the "cache" value drop by the 100Mb that you copied to the ram-based filesystem (assuming there was enough free RAM, you might find some of it ended up in swap if the machine is already over-committed in terms of memory use). The "sync; echo 3 > /proc/sys/vm/drop_caches" before each call to free should write anything pending in all write buffers (the sync) and clear all cached/buffered disk blocks from memory so free will only be reading other allocations in the "cached" value.

The RAM used by virtual machines (such as those running under VMWare) will also be counted in free's "cached" value, as will RAM used by currently open memory-mapped files.

So it isn't as simple as "buffers counts pending file/network writes and cached counts recently read/written blocks held in RAM to save future physical reads", though for most purposes this simpler description will do.


Freeing buffer/cache

Warning This explain a strong method not recommended on production server! So you're warned, don't blame me if something goes wrong.

For understanding, the thing, you could force your system to delegate as many memory as possible tocache than drop the cached file:

Preamble

Before of doing the test, you could open another window an hit:

$ vmstat -n 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  1  39132  59740  39892 1038820    0    0     1     0    3    3  5 13 81  1
 1  0  39132  59140  40076 1038812    0    0   184     0 10566 2157 27 15 48 11
...

for following evolution of swap in real time.

Nota: You must dispose of as many disk free on current directory, you have mem+swap

The demo
$ free
         total       used       free     shared    buffers     cached
Mem:       2064396    2004320      60076          0      90740     945964
-/+ buffers/cache:     967616    1096780
Swap:      3145720      38812    3106908

$ tot=0
$ while read -a line;do
      [[ "${line%:}" =~ ^(Swap|Mem)Total$ ]] && ((tot+=2*${line[1]}))
    done </proc/meminfo
$ echo $tot
10420232

$ dd if=/dev/zero of=veryBigFile count=$tot
10420232+0 records in
10420232+0 records out
5335158784 bytes (5.3 GB) copied, 109.526 s, 48.7 MB/s

$ cat >/dev/null veryBigFile

$ free
             total       used       free     shared    buffers     cached
Mem:       2064396    2010160      54236          0      41568    1039636
-/+ buffers/cache:     928956    1135440
Swap:      3145720      39132    3106588

$ rm veryBigFile 

$ free
         total       used       free     shared    buffers     cached
Mem:       2064396    1005104    1059292          0      41840      48124
-/+ buffers/cache:     915140    1149256
Swap:      3145720      39132    3106588

Nota, the host on wich I've done this is strongly used. This will be more significant on a really quiet machine.




buffer & cache


  A buffer is something that has yet to be "written" to disk. A cache is something that has been "read" from the disk and stored for later use.
  解释参考:Difference Between Buffer and Cache, orHere
  对于共享内存(Shared memory),主要用于在UNIX 环境下不同进程之间共享数据,是进程间通信的一种方法,一般的应用程序不会申请使用共享内存,笔者也没有去验证共享内存对上面等式的影响。如果你有兴趣,请参考:What is Shared Memory?

  cache 和 buffer的区别:
  Cache:高速缓存,是位于CPU与主内存间的一种容量较小但速度很高的存储器。由于CPU的速度远高于主内存,CPU直接从内存中存取数据要等待一定时间周期, Cache中保存着CPU刚用过或循环使用的一部分数据,当CPU再次使用该部分数据时可从Cache中直接调用,这样就减少了CPU的等待时间,提高了系统的效率。Cache又分为一级Cache(L1 Cache)和二级Cache(L2 Cache),L1 Cache集成在CPU内部,L2 Cache早期一般是焊在主板上,现在也都集成在CPU内部,常见的容量有256KB或512KB L2 Cache。

  Buffer:缓冲区,一个用于存储速度不同步的设备或优先级不同的设备之间传输数据的区域。通过缓冲区,可以使进程之间的相互等待变少,从而使从速度慢的设备读入数据时,速度快的设备的操作进程不发生间断。

  Free中的buffer和cache:(它们都是占用内存):
  buffer : 作为buffer cache的内存,是块设备的读写缓冲区
  cache: 作为page cache的内存, 文件系统的cache

  如果 cache 的值很大,说明cache住的文件数很多。如果频繁访问到的文件都能被cache住,那么磁盘的读IO bi会非常小。

一句话区分:readcache,writebuffer. cache常用于表示读缓存:memcache; buffer常用于写缓存: Intel CPU store buffer; 当然, 更常见的是二者混用: Oracle/MySQL buffer cache!(@here)


参考1 

page cache和buffer cache最大的差别在于:page cache是对文件数据的缓存;buffer cache是对设备数据的缓存!

参考2


目录
相关文章
|
5天前
|
存储 弹性计算 人工智能
【2025云栖精华内容】 打造持续领先,全球覆盖的澎湃算力底座——通用计算产品发布与行业实践专场回顾
2025年9月24日,阿里云弹性计算团队多位产品、技术专家及服务器团队技术专家共同在【2025云栖大会】现场带来了《通用计算产品发布与行业实践》的专场论坛,本论坛聚焦弹性计算多款通用算力产品发布。同时,ECS云服务器安全能力、资源售卖模式、计算AI助手等用户体验关键环节也宣布升级,让用云更简单、更智能。海尔三翼鸟云服务负责人刘建锋先生作为特邀嘉宾,莅临现场分享了关于阿里云ECS g9i推动AIoT平台的场景落地实践。
【2025云栖精华内容】 打造持续领先,全球覆盖的澎湃算力底座——通用计算产品发布与行业实践专场回顾
|
4天前
|
云安全 人工智能 自然语言处理
阿里云x硅基流动:AI安全护栏助力构建可信模型生态
阿里云AI安全护栏:大模型的“智能过滤系统”。
|
4天前
|
人工智能 自然语言处理 自动驾驶
关于举办首届全国大学生“启真问智”人工智能模型&智能体大赛决赛的通知
关于举办首届全国大学生“启真问智”人工智能模型&智能体大赛决赛的通知
|
Linux 虚拟化 iOS开发
VMware Workstation Pro 25H2 for Windows & Linux - 领先的免费桌面虚拟化软件
VMware Workstation Pro 25H2 for Windows & Linux - 领先的免费桌面虚拟化软件
1006 0
|
7天前
|
存储 机器学习/深度学习 人工智能
大模型微调技术:LoRA原理与实践
本文深入解析大语言模型微调中的关键技术——低秩自适应(LoRA)。通过分析全参数微调的计算瓶颈,详细阐述LoRA的数学原理、实现机制和优势特点。文章包含完整的PyTorch实现代码、性能对比实验以及实际应用场景,为开发者提供高效微调大模型的实践指南。
640 2
|
6天前
|
JavaScript API 开发工具
如何在原生App中调用Uniapp的原生功能?
如何在原生App中调用Uniapp的原生功能?
317 139
|
5天前
|
编解码 自然语言处理 文字识别
Qwen3-VL再添丁!4B/8B Dense模型开源,更轻量,仍强大
凌晨,Qwen3-VL系列再添新成员——Dense架构的Qwen3-VL-8B、Qwen3-VL-4B 模型,本地部署友好,并完整保留了Qwen3-VL的全部表现,评测指标表现优秀。
423 7
Qwen3-VL再添丁!4B/8B Dense模型开源,更轻量,仍强大