我按照如下链接,进行了百川2-7B-预训练模型(Baichuan2-7B-Base)的加载运行尝试,
https://modelscope.cn/models/baichuan-inc/Baichuan2-7B-Base/summary
我的代码如下:
from modelscope.utils.constant import Tasks
from modelscope.pipelines import pipeline
text_generation_zh = pipeline(task=Tasks.text_generation,
model='baichuan-inc/Baichuan2-7B-Base',
offload_folder='/usr/local/baichuan/offload_folder/',
device='cpu',
model_revision='v1.0.4')
text_generation_zh._model_prepare = True
result_zh = text_generation_zh('登鹳雀楼->王之涣\n夜雨寄北->',
min_length=10, max_length=64, num_beams=3, temperature=0.3, do_sample=False, early_stopping=True, top_k=50, top_p=0.85, repetition_penalty=1.1, length_penalty=1.2, no_repeat_ngram_size=6)
print(result_zh)
但是却得到了如下错误提示:
2024-03-25 16:23:35,858 - modelscope - INFO - Use user-specified model revision: v1.0.4
2024-03-25 16:23:36,210 - modelscope - WARNING - ('PIPELINES', 'text-generation', 'Baichuan2-7B-text-generation-pipe') not found in ast index file
ValueError: Baichuan7BTextGenerationPipeline: The current `device_map` had weights offloaded to the disk. Please provide an `offload_folder` for them. Alternatively, make sure you have `safetensors` installed if the model you are using offers the weights in this format.
请问,这其中的offload_folder该如何配置呢?
我的环境是使用了魔塔平台的PAI-DSW实例(CPU)