百川2-7B-预训练模型(Baichuan2-7B-Base)加载运行失败

我按照如下链接,进行了百川2-7B-预训练模型(Baichuan2-7B-Base)的加载运行尝试,
https://modelscope.cn/models/baichuan-inc/Baichuan2-7B-Base/summary

我的代码如下:

from modelscope.utils.constant import Tasks
from modelscope.pipelines import pipeline
text_generation_zh  = pipeline(task=Tasks.text_generation, 
                               model='baichuan-inc/Baichuan2-7B-Base',
                               offload_folder='/usr/local/baichuan/offload_folder/',
                               device='cpu',
                               model_revision='v1.0.4')
text_generation_zh._model_prepare = True
result_zh = text_generation_zh('登鹳雀楼->王之涣\n夜雨寄北->', 
                               min_length=10, max_length=64, num_beams=3, temperature=0.3, do_sample=False, early_stopping=True, top_k=50, top_p=0.85, repetition_penalty=1.1, length_penalty=1.2, no_repeat_ngram_size=6)
print(result_zh)

但是却得到了如下错误提示:


2024-03-25 16:23:35,858 - modelscope - INFO - Use user-specified model revision: v1.0.4
2024-03-25 16:23:36,210 - modelscope - WARNING - ('PIPELINES', 'text-generation', 'Baichuan2-7B-text-generation-pipe') not found in ast index file
ValueError: Baichuan7BTextGenerationPipeline: The current `device_map` had weights offloaded to the disk. Please provide an `offload_folder` for them. Alternatively, make sure you have `safetensors` installed if the model you are using offers the weights in this format.

请问,这其中的offload_folder该如何配置呢?

我的环境是使用了魔塔平台的PAI-DSW实例(CPU)

展开
收起
游客j6shnwof6yj3i 2024-03-25 17:02:23 288 分享 版权
0 条回答
写回答
取消 提交回答

包含命名实体识别、文本分类、分词、关系抽取、问答、推理、文本摘要、情感分析、机器翻译等多个领域

热门讨论

热门文章

还有其他疑问?
咨询AI助理