DashVector&千问7B模型,打造极简RAG 之 基于专属知识的问答服务实践

简介: 参考官方教程和代码,以一个0基础学习者的角色,用DashVector&千问7B模型,打造极简RAG-基于专属知识的问答服务实践,也带入了自己的一些理解。

整体介绍

本教程演示如何使用向量检索服务(DashVector),结合LLM大模型等能力,来打造基于垂直领域专属知识等问答服务。其中LLM大模型能力,以及文本向量生成等能力,这里基于灵积模型服务上的通义千问 API以及Embedding API来接入。

考虑到有官方教程,亲测过,很多代码跑不起来,还有不少坑。我以一个0基础学习者的角色,参考官方代码重新写一个实践的guide,也带入了自己的一些理解。

大语言模型(LLM)作为自然语言处理领域的核心技术,具有丰富的自然语言处理能力。但其训练语料库具有一定的局限性,一般由普适知识、常识性知识,如维基百科、新闻、小说,和各种领域的专业知识组成。导致LLM在处理特定领域的知识表示和应用时存在一定的局限性,特别对于垂直领域内,或者企业内部等私域专属知识。特别是金融、医疗等比较封闭的行业。

image.png

一、创建资源

魔搭账号申请并领用算力

https://modelscope.cn/home

点到我的Notebook页面

绑定阿里云账号

免费申请并启动GPU,能申请免费36小时的GPU环境,不用就停掉,尽量省着点

模型服务灵积Dashscope免费领用

dashscope(灵积)是围绕AI各领域模型,通过标准化的API提供包括模型推理、模型微调训练在内的多种模型服务。

https://dashscope.console.aliyun.com/overview

免费领取灵积服务,并在api-key管理页面创建一个dashscope的api-key

免费开工通向量检索服务DashVector

DashVector是一款向量数据库,通过他的服务能有效提升向量检索效率,实现针对非结构化数据的高性能向量检索服务,可广泛应用于大模型搜索、多模态搜索、AI搜索、分子结构分析等几乎所有的 AI 搜索场景

https://dashvector.console.aliyun.com/cn-hangzhou/overview

到api-key管理页面创建一个key

到Cluster列表免费申领1个向量库实例(1个月免费)

这里看到的collection是在后续代码里把数据集加载进去的

创建好实例后,保存endpoiint

进入编码环境界面

来到我们的丘比特环境

二、代码是这样的

安装 灵积模型服务、向量数据库服务和transformer模型的依赖资源包

# 安装所需要的资源包
!pip install dashvector dashscope
!pip install transformers_stream_generator python-dotenv
!pip install --upgrade transformers

会看到这样的日志

Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: dashvector in /opt/conda/lib/python3.10/site-packages (1.0.11)
Requirement already satisfied: dashscope in /opt/conda/lib/python3.10/site-packages (1.14.1)
Requirement already satisfied: aiohttp<4.0.0,>=3.1.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.9.3)
Requirement already satisfied: certifi<2024.0.0,>=2023.7.22 in /opt/conda/lib/python3.10/site-packages (from dashvector) (2023.11.17)
Requirement already satisfied: grpcio>=1.22.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.60.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.26.3)
Requirement already satisfied: protobuf<4.0.0,>=3.8.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.20.3)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from dashscope) (2.31.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (4.0.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (1.26.16)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers_stream_generator in /opt/conda/lib/python3.10/site-packages (0.0.4)
Requirement already satisfied: python-dotenv in /opt/conda/lib/python3.10/site-packages (1.0.1)
Requirement already satisfied: transformers>=4.26.1 in /opt/conda/lib/python3.10/site-packages (from transformers_stream_generator) (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip

下载我们的专属数据集

CEC-Corpus 是一些中文突发事件语料库

能看到里面有很多txt文档的预料信息

我们把他clone下来

# prepare news corpus as knowledge source
!git clone https://github.com/shijiebei2009/CEC-Corpus.git

执行会看到这样的日志

初始化向量数据库对象,并以向量维度1536创建一个数据集

import dashscope
import os
from dotenv import load_dotenv
from dashscope import TextEmbedding
from dashvector import Client, Doc
dashscope.api_key = '{你的魔搭apikey}'
# 初始化DashVector向量库
dashvector_client = Client(api_key='{填写你的向量库apikey}',endpoint='{填写你的向量库endpoint}')
# 定义一个数据集名称
collection_name = 'news_embeddings'
# 如果重复就删掉这个数据集
dashvector_client.delete(collection_name)
# 在向量数据库下创建一个数据集,向量维度设置为1536
rsp = dashvector_client.create(collection_name, 1536)
collection = dashvector_client.get(collection_name)

这里你会发现你的dashvector多了一个名为news_embeddings的collection

定义一个函数,以遍历文件夹下所有的文件语料,写入向量数据库中

def prepare_data_from_dir(path, size):
    #把准备好的语料以适合的大小更新到DashVector
    batch_docs = []
    for file in os.listdir(path):
        with open(path + '/' + file, 'r', encoding='utf-8') as f:
            batch_docs.append(f.read())
            if len(batch_docs) == size:
                yield batch_docs[:]
                batch_docs.clear()
    if batch_docs:
        yield batch_docs

定义一个函数,将单个文件语料,写入向量数据库中 --这个函数后面没有用到

def prepare_data_from_file(path, size):
    # 把准备好的预料文件以合理的文档大小插入到 DashVector 中。
    batch_docs = []
    chunk_size = 12
    with open(path, 'r', encoding='utf-8') as f:
        doc = ''
        count = 0
        for line in f:
            if count < chunk_size and line.strip() != '':
                doc += line
                count += 1
            if count == chunk_size:
                batch_docs.append(doc)
                if len(batch_docs) == size:
                    yield batch_docs[:]
                    batch_docs.clear()
                doc = ''
                count = 0
    if batch_docs:
        yield batch_docs
def generate_embeddings(docs):
    # 通过魔搭的文本向量化api,将语料向量化
    rsp = TextEmbedding.call(model=TextEmbedding.Models.text_embedding_v1,
                             input=docs)
    embeddings = [record['embedding'] for record in rsp.output['embeddings']]
    return embeddings if isinstance(docs, list) else embeddings[0]

上面都是定义函数,这一步开始把原始文件词向量化到Dashvector中

id = 0
dir_name = 'CEC-Corpus/raw corpus/allSourceText'
# 得到DashVector向量数据库对象
collection = dashvector_client.get(collection_name)
# 最大批次大小设置为4
batch_size = 4  
for news in list(prepare_data_from_dir(dir_name, batch_size)):
    ids = [id + i for i, _ in enumerate(news)]
    id += len(news)
    # 通过原始文件生成词向量
    
    vectors = generate_embeddings(news)
    # upsert到向量库的collection中
    ret = collection.upsert(
        [
            Doc(id=str(id), vector=vector, fields={"raw": doc})
            for id, doc, vector in zip(ids, news, vectors)
        ]
    )
    print(ret)

这里会看到这样的原始文件词向量化的信息打印

这里我们可以回去 https://dashvector.console.aliyun.com/cn-hangzhou/cluster/wondashvector/collectionList 看看我们的collection中,已经有了很多数据

定义函数从collection中查询召回相关的信息优先级最高的一条,并尝试查询召回

def search_relevant_context(question, topk=1, client=dashvector_client):
    # 从collection中查询召回相关的信息
    collection = client.get(collection_name)
    # 从向量数据库召回优先级最高的一条信息
    rsp = collection.query(generate_embeddings(question), output_fields=['raw'],
                           topk=topk)
    return "".join([item.fields['raw'] for item in rsp.output])

先试试看能不能成功召回

# query the top 1 results
question = '有没有中毒相关的新闻?'
context = search_relevant_context(question, topk=1)
print(context)

下载并加载通义千问7B模型

加载通义千问7b模型

!git clone https://www.modelscope.cn/qwen/Qwen-7B-Chat.git

会看到这样的成功日志

初始化千问7B模型

# initialize qwen 7B model
from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("/mnt/workspace/Qwen-7B-Chat", device_map="auto", trust_remote_code=True).eval()
model.generation_config = GenerationConfig.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True) 
# 可指定不同的生成长度、top_p等相关超参

成功初始化会看到这样的信息

设定一个简单的问题,并开始我们的测试

设定提示词模版原因是能把之前从dashvector中检索的信息加入提示词

# define a prompt template for the vectorDB-enhanced LLM generation
def answer_question(question, context):
    prompt = f'''请基于```内的内容回答问题。"
    ```
    {context}
    ```
    我的问题是:{question}。
    '''
    history = None
    print(prompt)
    response, history = model.chat(tokenizer, prompt, history=None)
    return response

我们先试试,只查询千问7B模型的效果

# test the case on plain LLM without vectorDB enhancement
question = '有没有中毒相关的新闻?'
answer = answer_question(question, '')
print(f'question: {question}\n' f'answer: {answer}')

再看看加入了专属知识后的效果

# test the case with knowledge
context = search_relevant_context(question, topk=1)
answer = answer_question(question, context)
print(f'question: {question}\n' f'answer: {answer}')

相关文章
|
9天前
|
并行计算 PyTorch 算法框架/工具
社区供稿 | 本地部署通义千问大模型做RAG验证
这篇文章中,我们通过将模搭社区开源的大模型部署到本地,并实现简单的对话和RAG。
|
9天前
|
存储 自然语言处理 算法
【学习大模型】RAG基础
RAG(Retrieval-Augmented Generation)技术是为了解决大模型中的幻觉问题、实时交互、数据安全和知识动态性挑战。它结合了搜索和大模型的提示功能,使模型能基于检索到的信息生成更准确的回答。RAG通过向量数据库和向量检索,将文本转化为向量表示,然后进行相似度计算和检索,以提供上下文相关的信息。
240 1
|
9天前
|
机器学习/深度学习 自然语言处理 机器人
【RAG实践】基于LlamaIndex和Qwen1.5搭建基于本地知识库的问答机器人
LLM会产生误导性的 “幻觉”,依赖的信息可能过时,处理特定知识时效率不高,缺乏专业领域的深度洞察,同时在推理能力上也有所欠缺。
|
9天前
|
机器学习/深度学习 人工智能 算法
通义千问Qwen-72B-Chat大模型在PAI平台的微调实践
本文将以Qwen-72B-Chat为例,介绍如何在PAI平台的快速开始PAI-QuickStart和交互式建模工具PAI-DSW中高效微调千问大模型。
|
9天前
|
弹性计算 自然语言处理 开发工具
基于阿里云向量检索 Milvus 版和 LangChain 快速构建 LLM 问答系统
本文介绍如何通过整合阿里云Milvus、阿里云DashScope Embedding模型与阿里云PAI(EAS)模型服务,构建一个由LLM(大型语言模型)驱动的问题解答应用,并着重演示了如何搭建基于这些技术的RAG对话系统。
|
9天前
|
自然语言处理 物联网 API
检索增强生成(RAG)实践:基于LlamaIndex和Qwen1.5搭建智能问答系统
检索增强生成(RAG)实践:基于LlamaIndex和Qwen1.5搭建智能问答系统
检索增强生成(RAG)实践:基于LlamaIndex和Qwen1.5搭建智能问答系统
|
9天前
|
自然语言处理 API 开发工具
基于LangChain-Chatchat实现的本地知识库的问答应用-快速上手(检索增强生成(RAG)大模型)
基于LangChain-Chatchat实现的本地知识库的问答应用-快速上手(检索增强生成(RAG)大模型)
基于LangChain-Chatchat实现的本地知识库的问答应用-快速上手(检索增强生成(RAG)大模型)
|
9天前
|
存储 SQL 关系型数据库
【LLM】基于pvVevtor和LangChain构建RAG(检索增强)服务
【5月更文挑战第4天】基于pgVector和LangChain构建RAG检索增强服务
44 4
|
9天前
|
自然语言处理 Swift
千亿大模型来了!通义千问110B模型开源,魔搭社区推理、微调最佳实践
近期开源社区陆续出现了千亿参数规模以上的大模型,这些模型都在各项评测中取得杰出的成绩。今天,通义千问团队开源1100亿参数的Qwen1.5系列首个千亿参数模型Qwen1.5-110B,该模型在基础能力评估中与Meta-Llama3-70B相媲美,在Chat评估中表现出色,包括MT-Bench和AlpacaEval 2.0。
|
9天前
|
人工智能 自然语言处理 监控
通过阿里云向量检索 Milvus 版和通义千问快速构建基于专属知识库的问答系统
本文展示了如何使用阿里云向量检索 Milvus 版和灵积(Dashscope)提供的通用千问大模型能力,快速构建一个基于专属知识库的问答系统。在示例中,我们通过接入灵积的通义千问 API 及文本嵌入(Embedding)API 来实现 LLM 大模型的相关功能。
通过阿里云向量检索 Milvus 版和通义千问快速构建基于专属知识库的问答系统