DashVector&千问7B模型,打造极简RAG 之 基于专属知识的问答服务实践

本文涉及的产品
NLP自然语言处理_高级版,每接口累计50万次
NLP自然语言处理_基础版,每接口每天50万次
NLP 自学习平台,3个模型定制额度 1个月
简介: 参考官方教程和代码,以一个0基础学习者的角色,用DashVector&千问7B模型,打造极简RAG-基于专属知识的问答服务实践,也带入了自己的一些理解。

整体介绍

本教程演示如何使用向量检索服务(DashVector),结合LLM大模型等能力,来打造基于垂直领域专属知识等问答服务。其中LLM大模型能力,以及文本向量生成等能力,这里基于灵积模型服务上的通义千问 API以及Embedding API来接入。

考虑到有官方教程,亲测过,很多代码跑不起来,还有不少坑。我以一个0基础学习者的角色,参考官方代码重新写一个实践的guide,也带入了自己的一些理解。

大语言模型(LLM)作为自然语言处理领域的核心技术,具有丰富的自然语言处理能力。但其训练语料库具有一定的局限性,一般由普适知识、常识性知识,如维基百科、新闻、小说,和各种领域的专业知识组成。导致LLM在处理特定领域的知识表示和应用时存在一定的局限性,特别对于垂直领域内,或者企业内部等私域专属知识。特别是金融、医疗等比较封闭的行业。

image.png

一、创建资源

魔搭账号申请并领用算力

https://modelscope.cn/home

点到我的Notebook页面

绑定阿里云账号

免费申请并启动GPU,能申请免费36小时的GPU环境,不用就停掉,尽量省着点

模型服务灵积Dashscope免费领用

dashscope(灵积)是围绕AI各领域模型,通过标准化的API提供包括模型推理、模型微调训练在内的多种模型服务。

https://dashscope.console.aliyun.com/overview

免费领取灵积服务,并在api-key管理页面创建一个dashscope的api-key

免费开工通向量检索服务DashVector

DashVector是一款向量数据库,通过他的服务能有效提升向量检索效率,实现针对非结构化数据的高性能向量检索服务,可广泛应用于大模型搜索、多模态搜索、AI搜索、分子结构分析等几乎所有的 AI 搜索场景

https://dashvector.console.aliyun.com/cn-hangzhou/overview

到api-key管理页面创建一个key

到Cluster列表免费申领1个向量库实例(1个月免费)

这里看到的collection是在后续代码里把数据集加载进去的

创建好实例后,保存endpoiint

进入编码环境界面

来到我们的丘比特环境

二、代码是这样的

安装 灵积模型服务、向量数据库服务和transformer模型的依赖资源包

# 安装所需要的资源包
!pip install dashvector dashscope
!pip install transformers_stream_generator python-dotenv
!pip install --upgrade transformers

会看到这样的日志

Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: dashvector in /opt/conda/lib/python3.10/site-packages (1.0.11)
Requirement already satisfied: dashscope in /opt/conda/lib/python3.10/site-packages (1.14.1)
Requirement already satisfied: aiohttp<4.0.0,>=3.1.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.9.3)
Requirement already satisfied: certifi<2024.0.0,>=2023.7.22 in /opt/conda/lib/python3.10/site-packages (from dashvector) (2023.11.17)
Requirement already satisfied: grpcio>=1.22.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.60.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.26.3)
Requirement already satisfied: protobuf<4.0.0,>=3.8.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.20.3)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from dashscope) (2.31.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (4.0.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (1.26.16)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers_stream_generator in /opt/conda/lib/python3.10/site-packages (0.0.4)
Requirement already satisfied: python-dotenv in /opt/conda/lib/python3.10/site-packages (1.0.1)
Requirement already satisfied: transformers>=4.26.1 in /opt/conda/lib/python3.10/site-packages (from transformers_stream_generator) (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip

下载我们的专属数据集

CEC-Corpus 是一些中文突发事件语料库

能看到里面有很多txt文档的预料信息

我们把他clone下来

# prepare news corpus as knowledge source
!git clone https://github.com/shijiebei2009/CEC-Corpus.git

执行会看到这样的日志

初始化向量数据库对象,并以向量维度1536创建一个数据集

import dashscope
import os
from dotenv import load_dotenv
from dashscope import TextEmbedding
from dashvector import Client, Doc
dashscope.api_key = '{你的魔搭apikey}'
# 初始化DashVector向量库
dashvector_client = Client(api_key='{填写你的向量库apikey}',endpoint='{填写你的向量库endpoint}')
# 定义一个数据集名称
collection_name = 'news_embeddings'
# 如果重复就删掉这个数据集
dashvector_client.delete(collection_name)
# 在向量数据库下创建一个数据集,向量维度设置为1536
rsp = dashvector_client.create(collection_name, 1536)
collection = dashvector_client.get(collection_name)

这里你会发现你的dashvector多了一个名为news_embeddings的collection

定义一个函数,以遍历文件夹下所有的文件语料,写入向量数据库中

def prepare_data_from_dir(path, size):
    #把准备好的语料以适合的大小更新到DashVector
    batch_docs = []
    for file in os.listdir(path):
        with open(path + '/' + file, 'r', encoding='utf-8') as f:
            batch_docs.append(f.read())
            if len(batch_docs) == size:
                yield batch_docs[:]
                batch_docs.clear()
    if batch_docs:
        yield batch_docs

定义一个函数,将单个文件语料,写入向量数据库中 --这个函数后面没有用到

def prepare_data_from_file(path, size):
    # 把准备好的预料文件以合理的文档大小插入到 DashVector 中。
    batch_docs = []
    chunk_size = 12
    with open(path, 'r', encoding='utf-8') as f:
        doc = ''
        count = 0
        for line in f:
            if count < chunk_size and line.strip() != '':
                doc += line
                count += 1
            if count == chunk_size:
                batch_docs.append(doc)
                if len(batch_docs) == size:
                    yield batch_docs[:]
                    batch_docs.clear()
                doc = ''
                count = 0
    if batch_docs:
        yield batch_docs
def generate_embeddings(docs):
    # 通过魔搭的文本向量化api,将语料向量化
    rsp = TextEmbedding.call(model=TextEmbedding.Models.text_embedding_v1,
                             input=docs)
    embeddings = [record['embedding'] for record in rsp.output['embeddings']]
    return embeddings if isinstance(docs, list) else embeddings[0]

上面都是定义函数,这一步开始把原始文件词向量化到Dashvector中

id = 0
dir_name = 'CEC-Corpus/raw corpus/allSourceText'
# 得到DashVector向量数据库对象
collection = dashvector_client.get(collection_name)
# 最大批次大小设置为4
batch_size = 4  
for news in list(prepare_data_from_dir(dir_name, batch_size)):
    ids = [id + i for i, _ in enumerate(news)]
    id += len(news)
    # 通过原始文件生成词向量
    
    vectors = generate_embeddings(news)
    # upsert到向量库的collection中
    ret = collection.upsert(
        [
            Doc(id=str(id), vector=vector, fields={"raw": doc})
            for id, doc, vector in zip(ids, news, vectors)
        ]
    )
    print(ret)

这里会看到这样的原始文件词向量化的信息打印

这里我们可以回去 https://dashvector.console.aliyun.com/cn-hangzhou/cluster/wondashvector/collectionList 看看我们的collection中,已经有了很多数据

定义函数从collection中查询召回相关的信息优先级最高的一条,并尝试查询召回

def search_relevant_context(question, topk=1, client=dashvector_client):
    # 从collection中查询召回相关的信息
    collection = client.get(collection_name)
    # 从向量数据库召回优先级最高的一条信息
    rsp = collection.query(generate_embeddings(question), output_fields=['raw'],
                           topk=topk)
    return "".join([item.fields['raw'] for item in rsp.output])

先试试看能不能成功召回

# query the top 1 results
question = '有没有中毒相关的新闻?'
context = search_relevant_context(question, topk=1)
print(context)

下载并加载通义千问7B模型

加载通义千问7b模型

!git clone https://www.modelscope.cn/qwen/Qwen-7B-Chat.git

会看到这样的成功日志

初始化千问7B模型

# initialize qwen 7B model
from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("/mnt/workspace/Qwen-7B-Chat", device_map="auto", trust_remote_code=True).eval()
model.generation_config = GenerationConfig.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True) 
# 可指定不同的生成长度、top_p等相关超参

成功初始化会看到这样的信息

设定一个简单的问题,并开始我们的测试

设定提示词模版原因是能把之前从dashvector中检索的信息加入提示词

# define a prompt template for the vectorDB-enhanced LLM generation
def answer_question(question, context):
    prompt = f'''请基于```内的内容回答问题。"
    ```
    {context}
    ```
    我的问题是:{question}。
    '''
    history = None
    print(prompt)
    response, history = model.chat(tokenizer, prompt, history=None)
    return response

我们先试试,只查询千问7B模型的效果

# test the case on plain LLM without vectorDB enhancement
question = '有没有中毒相关的新闻?'
answer = answer_question(question, '')
print(f'question: {question}\n' f'answer: {answer}')

再看看加入了专属知识后的效果

# test the case with knowledge
context = search_relevant_context(question, topk=1)
answer = answer_question(question, context)
print(f'question: {question}\n' f'answer: {answer}')

相关文章
|
4月前
|
机器学习/深度学习 人工智能 算法
通义千问Qwen-72B-Chat大模型在PAI平台的微调实践
本文将以Qwen-72B-Chat为例,介绍如何在PAI平台的快速开始PAI-QuickStart和交互式建模工具PAI-DSW中高效微调千问大模型。
|
4月前
|
并行计算 PyTorch 算法框架/工具
社区供稿 | 本地部署通义千问大模型做RAG验证
这篇文章中,我们通过将模搭社区开源的大模型部署到本地,并实现简单的对话和RAG。
|
4月前
|
机器学习/深度学习 自然语言处理 机器人
【RAG实践】基于LlamaIndex和Qwen1.5搭建基于本地知识库的问答机器人
LLM会产生误导性的 “幻觉”,依赖的信息可能过时,处理特定知识时效率不高,缺乏专业领域的深度洞察,同时在推理能力上也有所欠缺。
|
2月前
|
人工智能 前端开发 API
RAG+AI工作流+Agent:LLM框架该如何选择,全面对比MaxKB、Dify、FastGPT、RagFlow、Anything-LLM,以及更多推荐
【7月更文挑战第9天】RAG+AI工作流+Agent:LLM框架该如何选择,全面对比MaxKB、Dify、FastGPT、RagFlow、Anything-LLM,以及更多推荐
RAG+AI工作流+Agent:LLM框架该如何选择,全面对比MaxKB、Dify、FastGPT、RagFlow、Anything-LLM,以及更多推荐
|
28天前
|
人工智能 内存技术
通义语音AI技术问题之预训练模型的推理与微调如何解决
通义语音AI技术问题之预训练模型的推理与微调如何解决
25 4
|
2月前
|
自然语言处理 并行计算 Python
千问大模型对话
该段Python代码演示了如何使用`modelscope`库加载预训练的因果语言模型`Qwen2-1___5B-Instruct`到CUDA设备,并进行文本生成。首先,它定义了设备为`cuda`,然后加载模型和分词器。接着,设置了一个聊天情境并准备输入文本。通过调用`tokenizer.apply_chat_template`处理输入,然后将处理后的文本转化为模型输入张量并传入设备。使用`model.generate`方法生成新的文本,截取生成的输出并解码为人类可读的字符串。最后,打印出生成的响应。
90 6
|
3月前
|
机器学习/深度学习 算法 开发工具
通义千问2(Qwen2)大语言模型在PAI-QuickStart的微调、评测与部署实践
阿里云的人工智能平台PAI,作为一站式的机器学习和深度学习平台,对Qwen2模型系列提供了全面的技术支持。无论是开发者还是企业客户,都可以通过PAI-QuickStart轻松实现Qwen2系列模型的微调、评测和快速部署。
|
2月前
|
前端开发 Java API
阿里云百炼模型入门篇-大语言模型
本文主要介绍如何快速的通过阿里云百炼,带你如何快速入门通义千问系列大语言模型。
|
2月前
|
数据采集 监控 算法
阿里云百炼模型训练评测
【7月更文挑战第1天】阿里云百炼提供一站式的模型开发服务,包括大模型训练、调用与部署。用户可查看剩余调用次数,点击开通服务以使用模型。计费基于调用量,涵盖推理、训练和部署。开通服务需同意协议,成功后将收到短信通知。评测显示,平台功能丰富,易用性强,能显著提升模型效果,且模型部署简便。建议优化数据预处理工具并增加实例教程。
|
4月前
|
自然语言处理 算法 OLAP
阿里云PAI大模型RAG对话系统最佳实践
本文为大模型RAG对话系统最佳实践,旨在指引AI开发人员如何有效地结合LLM大语言模型的推理能力和外部知识库检索增强技术,从而显著提升对话系统的性能,使其能更加灵活地返回用户查询的内容。适用于问答、摘要生成和其他依赖外部知识的自然语言处理任务。通过该实践,您可以掌握构建一个大模型RAG对话系统的完整开发链路。

热门文章

最新文章