通过langchain如何基于modelscope运行百川13B对话模型?
langchian是一个应用程序的框架,比如通过langchian调用Hugging Face 模型的一个示例如下:import torchfrom langchain import PromptTemplate, LLMChainfrom langchain.llms import HuggingFacePipelinefrom transformers import AutoTokenizer, AutoModelForCausalLM, pipeline, AutoModelForSeq2SeqLMfrom transformers.generation.utils import GenerationConfig
model_id = 'baichuan-inc/Baichuan-13B-Chat'tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained(model_id, device_map='auto', torch_dtype=torch.float16, trust_remote_code=True)
pipe = pipeline( 'text-generation', model=model, tokenizer=tokenizer, max_length=10000)
local_llm = HuggingFacePipeline(pipeline=pipe)print(local_llm('What is the capital of France? '))
template = '''Question: {question} Answer: Let's think step by step.'''prompt = PromptTemplate(template=template, input_variables=['question'])
llm_chain = LLMChain(prompt=prompt, llm=local_llm)question = 'What is the capital of England?'print(llm_chain.run(question))
我想请教的是可以通过类似的方式运行model scope中的模型吗?
赞0
踩0