一、问题
由于通过pmdk库使用PM进行编程时,需要创建文件,然后将其映射到内存,这个文件大小是固定的。那么当这块PM内存块使用完后,需要重新映射。但是当前用户进程扩展后的大小对其他用户进程不可见,其他用户进程需要重新映射后才能使用扩展后的内存。对于写密集的应用,需要频繁进行扩展,从而频繁解除映射、重新映射,对性能带来较大影响。
有没有方法创建文件并映射内存后,使其动态在线进行扩展,不需要重新解除映射、重新映射。
二、解决方法
想到libpmemobj的pool set,这个库初步理解可创建一个内存池,然后从这个内存池申请内存。多个文件可以组成一个pool。那么有没有一个方法使用pool的时候进行在线动态扩展?
正好pmdk有这样的开放性讨论问题,并且是已经关闭的状态,也就是说这个问题可以解决:
进去查看讨论内容:
QUESTION:
How to dynamically increase pool size in runtime?
Details
This is quite extended question to issue #4217
I'm interested in dynamic management of pool size during runtime.
I read about the descriptions and explanations of Directories section in POOLSET and manpage, but I don't get it clear.
I looked for some scripts or source codes doing the directory allocation, but couldn't find any.
Can you provide some simple examples?
第一个人的给出的方法:
All you have to do is to create a poolset with a directory:
PMEMPOOLSET
OPTION SINGLEHDR
500G /mnt/pmem/my_pool/
Create it using pmempool create:
pmempool create obj my_pool.set --layout foobar
And then use the pool in your application:
...
PMEMobjpool *pop = pmemobj_open("my_pool.set", "foobar");
...
And that's it. Your pool will now create new files in the poolset directory whenever new space is needed.
但是按照这个方法创建的时候,一直不成功,总是报pool文件是一个目录。接着看其他人方法:
This is controlled by heap.size.granularity ctl:
https://pmem.io/pmdk/manpages/linux/master/libpmemobj/pmemobj_ctl_get.3
https://pmem.io/pmdk/manpages/linux/master/pmem_ctl/pmem_ctl.5
Take a look at the test for this functionality:
https://github.com/pmem/pmdk/tree/master/src/test/obj_ctl_heap_size
It shows you both how to create a directory poolset and how to control granularity.
这个例子的前提也是先创建一个pool set的目录。仔细阅读这个例子的TEST0后,发现需要首先创建一个目录,然后创建pool set,其pool文件为该目录,然后再通过pmemobj_open打开pool后,设置pmemobj_alloc(pop, NULL, OBJ_SIZE, 0, NULL, NULL) 将这个池子消耗完,然后将heap.size.granularity设置成非0值:
ssize_t new_granularity = CUSTOM_GRANULARITY;
ret = pmemobj_ctl_set(pop, "heap.size.granularity",&new_granularity);
或者将heap.size.extend设置成非0值:
ssize_t extend_size = CUSTOM_GRANULARITY;
ret = pmemobj_ctl_exec(pop, "heap.size.extend", &extend_size);
他会自动通过创建一个新文件.pmem扩展pool set大小,后续再进行pmemobj_alloc分配内存的时候分配成功。
三、总结
1)创建一个目录
2)创建一个pool set,该pool set指定的文件为该目录
3)设置heap_size.granularity为非0值或者设置heap.size.extend设置非0值
会通过自动创建一个.pmem文件扩展pool set。