执行:
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'
from swift.llm import (
get_model_tokenizer, get_template, inference, ModelType, get_default_template_type,
)
from swift.utils import seed_everything
from swift.tuners import Swift
seed_everything(42)
ckpt_dir = 'v0-20240411-151422/checkpoint-48/'
model_type = ModelType.qwen1half_7b_chat
template_type = get_default_template_type(model_type)
model, tokenizer = get_model_tokenizer(model_type, model_kwargs={'device_map': 'auto'})
model.generation_config.max_new_tokens = 512
model = Swift.from_pretrained(model, ckpt_dir, inference_mode=True)
template = get_template(template_type, tokenizer)
query = '你是qwen吗?'
response, history = inference(model, template, query)
print(f'response: {response}')
print(f'history: {history}')
错误:ValueError: Target modules ['attn.c_attn', 'mlp.w1', 'attn.c_proj', 'mlp.c_proj', 'mlp.w2'] not found in the base model. Please check the target modules and try again.
以上modelscope错误怎么处理?"
ModelScope旨在打造下一代开源的模型即服务共享平台,为泛AI开发者提供灵活、易用、低成本的一站式模型服务产品,让模型应用更简单!欢迎加入技术交流群:微信公众号:魔搭ModelScope社区,钉钉群号:44837352