DashVector&千问7B模型,打造极简RAG 之 基于专属知识的问答服务实践

本文涉及的产品
NLP 自学习平台,3个模型定制额度 1个月
NLP自然语言处理_高级版,每接口累计50万次
NLP自然语言处理_基础版,每接口每天50万次
简介: 参考官方教程和代码,以一个0基础学习者的角色,用DashVector&千问7B模型,打造极简RAG-基于专属知识的问答服务实践,也带入了自己的一些理解。

整体介绍

本教程演示如何使用向量检索服务(DashVector),结合LLM大模型等能力,来打造基于垂直领域专属知识等问答服务。其中LLM大模型能力,以及文本向量生成等能力,这里基于灵积模型服务上的通义千问 API以及Embedding API来接入。

考虑到有官方教程,亲测过,很多代码跑不起来,还有不少坑。我以一个0基础学习者的角色,参考官方代码重新写一个实践的guide,也带入了自己的一些理解。

大语言模型(LLM)作为自然语言处理领域的核心技术,具有丰富的自然语言处理能力。但其训练语料库具有一定的局限性,一般由普适知识、常识性知识,如维基百科、新闻、小说,和各种领域的专业知识组成。导致LLM在处理特定领域的知识表示和应用时存在一定的局限性,特别对于垂直领域内,或者企业内部等私域专属知识。特别是金融、医疗等比较封闭的行业。

image.png

一、创建资源

魔搭账号申请并领用算力

https://modelscope.cn/home

点到我的Notebook页面

绑定阿里云账号

免费申请并启动GPU,能申请免费36小时的GPU环境,不用就停掉,尽量省着点

模型服务灵积Dashscope免费领用

dashscope(灵积)是围绕AI各领域模型,通过标准化的API提供包括模型推理、模型微调训练在内的多种模型服务。

https://dashscope.console.aliyun.com/overview

免费领取灵积服务,并在api-key管理页面创建一个dashscope的api-key

免费开工通向量检索服务DashVector

DashVector是一款向量数据库,通过他的服务能有效提升向量检索效率,实现针对非结构化数据的高性能向量检索服务,可广泛应用于大模型搜索、多模态搜索、AI搜索、分子结构分析等几乎所有的 AI 搜索场景

https://dashvector.console.aliyun.com/cn-hangzhou/overview

到api-key管理页面创建一个key

到Cluster列表免费申领1个向量库实例(1个月免费)

这里看到的collection是在后续代码里把数据集加载进去的

创建好实例后,保存endpoiint

进入编码环境界面

来到我们的丘比特环境

二、代码是这样的

安装 灵积模型服务、向量数据库服务和transformer模型的依赖资源包

# 安装所需要的资源包
!pip install dashvector dashscope
!pip install transformers_stream_generator python-dotenv
!pip install --upgrade transformers

会看到这样的日志

Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: dashvector in /opt/conda/lib/python3.10/site-packages (1.0.11)
Requirement already satisfied: dashscope in /opt/conda/lib/python3.10/site-packages (1.14.1)
Requirement already satisfied: aiohttp<4.0.0,>=3.1.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.9.3)
Requirement already satisfied: certifi<2024.0.0,>=2023.7.22 in /opt/conda/lib/python3.10/site-packages (from dashvector) (2023.11.17)
Requirement already satisfied: grpcio>=1.22.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.60.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.26.3)
Requirement already satisfied: protobuf<4.0.0,>=3.8.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.20.3)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from dashscope) (2.31.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (4.0.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (1.26.16)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers_stream_generator in /opt/conda/lib/python3.10/site-packages (0.0.4)
Requirement already satisfied: python-dotenv in /opt/conda/lib/python3.10/site-packages (1.0.1)
Requirement already satisfied: transformers>=4.26.1 in /opt/conda/lib/python3.10/site-packages (from transformers_stream_generator) (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip

下载我们的专属数据集

CEC-Corpus 是一些中文突发事件语料库

能看到里面有很多txt文档的预料信息

我们把他clone下来

# prepare news corpus as knowledge source
!git clone https://github.com/shijiebei2009/CEC-Corpus.git

执行会看到这样的日志

初始化向量数据库对象,并以向量维度1536创建一个数据集

import dashscope
import os
from dotenv import load_dotenv
from dashscope import TextEmbedding
from dashvector import Client, Doc
dashscope.api_key = '{你的魔搭apikey}'
# 初始化DashVector向量库
dashvector_client = Client(api_key='{填写你的向量库apikey}',endpoint='{填写你的向量库endpoint}')
# 定义一个数据集名称
collection_name = 'news_embeddings'
# 如果重复就删掉这个数据集
dashvector_client.delete(collection_name)
# 在向量数据库下创建一个数据集,向量维度设置为1536
rsp = dashvector_client.create(collection_name, 1536)
collection = dashvector_client.get(collection_name)

这里你会发现你的dashvector多了一个名为news_embeddings的collection

定义一个函数,以遍历文件夹下所有的文件语料,写入向量数据库中

def prepare_data_from_dir(path, size):
    #把准备好的语料以适合的大小更新到DashVector
    batch_docs = []
    for file in os.listdir(path):
        with open(path + '/' + file, 'r', encoding='utf-8') as f:
            batch_docs.append(f.read())
            if len(batch_docs) == size:
                yield batch_docs[:]
                batch_docs.clear()
    if batch_docs:
        yield batch_docs

定义一个函数,将单个文件语料,写入向量数据库中 --这个函数后面没有用到

def prepare_data_from_file(path, size):
    # 把准备好的预料文件以合理的文档大小插入到 DashVector 中。
    batch_docs = []
    chunk_size = 12
    with open(path, 'r', encoding='utf-8') as f:
        doc = ''
        count = 0
        for line in f:
            if count < chunk_size and line.strip() != '':
                doc += line
                count += 1
            if count == chunk_size:
                batch_docs.append(doc)
                if len(batch_docs) == size:
                    yield batch_docs[:]
                    batch_docs.clear()
                doc = ''
                count = 0
    if batch_docs:
        yield batch_docs
def generate_embeddings(docs):
    # 通过魔搭的文本向量化api,将语料向量化
    rsp = TextEmbedding.call(model=TextEmbedding.Models.text_embedding_v1,
                             input=docs)
    embeddings = [record['embedding'] for record in rsp.output['embeddings']]
    return embeddings if isinstance(docs, list) else embeddings[0]

上面都是定义函数,这一步开始把原始文件词向量化到Dashvector中

id = 0
dir_name = 'CEC-Corpus/raw corpus/allSourceText'
# 得到DashVector向量数据库对象
collection = dashvector_client.get(collection_name)
# 最大批次大小设置为4
batch_size = 4  
for news in list(prepare_data_from_dir(dir_name, batch_size)):
    ids = [id + i for i, _ in enumerate(news)]
    id += len(news)
    # 通过原始文件生成词向量
    
    vectors = generate_embeddings(news)
    # upsert到向量库的collection中
    ret = collection.upsert(
        [
            Doc(id=str(id), vector=vector, fields={"raw": doc})
            for id, doc, vector in zip(ids, news, vectors)
        ]
    )
    print(ret)

这里会看到这样的原始文件词向量化的信息打印

这里我们可以回去 https://dashvector.console.aliyun.com/cn-hangzhou/cluster/wondashvector/collectionList 看看我们的collection中,已经有了很多数据

定义函数从collection中查询召回相关的信息优先级最高的一条,并尝试查询召回

def search_relevant_context(question, topk=1, client=dashvector_client):
    # 从collection中查询召回相关的信息
    collection = client.get(collection_name)
    # 从向量数据库召回优先级最高的一条信息
    rsp = collection.query(generate_embeddings(question), output_fields=['raw'],
                           topk=topk)
    return "".join([item.fields['raw'] for item in rsp.output])

先试试看能不能成功召回

# query the top 1 results
question = '有没有中毒相关的新闻?'
context = search_relevant_context(question, topk=1)
print(context)

下载并加载通义千问7B模型

加载通义千问7b模型

!git clone https://www.modelscope.cn/qwen/Qwen-7B-Chat.git

会看到这样的成功日志

初始化千问7B模型

# initialize qwen 7B model
from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("/mnt/workspace/Qwen-7B-Chat", device_map="auto", trust_remote_code=True).eval()
model.generation_config = GenerationConfig.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True) 
# 可指定不同的生成长度、top_p等相关超参

成功初始化会看到这样的信息

设定一个简单的问题,并开始我们的测试

设定提示词模版原因是能把之前从dashvector中检索的信息加入提示词

# define a prompt template for the vectorDB-enhanced LLM generation
def answer_question(question, context):
    prompt = f'''请基于```内的内容回答问题。"
    ```
    {context}
    ```
    我的问题是:{question}。
    '''
    history = None
    print(prompt)
    response, history = model.chat(tokenizer, prompt, history=None)
    return response

我们先试试,只查询千问7B模型的效果

# test the case on plain LLM without vectorDB enhancement
question = '有没有中毒相关的新闻?'
answer = answer_question(question, '')
print(f'question: {question}\n' f'answer: {answer}')

再看看加入了专属知识后的效果

# test the case with knowledge
context = search_relevant_context(question, topk=1)
answer = answer_question(question, context)
print(f'question: {question}\n' f'answer: {answer}')

相关文章
|
19天前
|
机器学习/深度学习 自然语言处理 算法框架/工具
实战 | Qwen2.5-VL模型目标检测(Grounding)任务领域微调教程
在目标检测领域,众多神经网络模型早已凭借其卓越的性能,实现了精准的目标检测与目标分割效果。然而,随着多模态模型的崛起,其在图像分析方面展现出的非凡能力,为该领域带来了新的机遇。多模态模型不仅能够深入理解图像内容,还能将这种理解转化为文本形式输出,极大地拓展了其应用场景。
1355 74
|
30天前
|
人工智能 负载均衡 数据可视化
10分钟上手全球开源模型冠军 Qwen3
阿里通义千问Qwen3在最新全球AI基准测试中智能水平位列全球前五,开源第一,且成本优势显著,推理成本仅为DeepSeek-R1的1/3、Claude 3.7的1/20。Qwen3支持119种语言,具备强大的代码和数学能力,同时提供思考与非思考两种模式无缝切换,适合复杂与简单任务。通过阿里云百炼平台,用户可在10分钟内快速搭建Qwen3模型服务,结合Cherry Studio客户端实现便捷交互。本文详细介绍了Qwen3的部署、体验及工具调用能力,帮助用户轻松上手。
636 78
|
1月前
|
数据可视化 API Swift
全模态图像模型Nexus-Gen对齐GPT-4o!同时搞定,数据、训练框架、模型全面开源
OpenAI GPT-4o发布强大图片生成能力后,业界对大模型生图能力的探索向全模态方向倾斜,训练全模态模型成研发重点。
158 17
|
1月前
|
Kubernetes 调度 开发者
qwen模型 MindIE PD分离部署问题定位
使用MindIE提供的PD分离特性部署qwen2-7B模型,使用k8s拉起容器,参考这个文档进行部署:https://www.hiascend.com/document/detail/zh/mindie/100/mindieservice/servicedev/mindie_service0060.html,1个Prefill,1个Decode。 最后一步测试推理请求的时候,出现报错:model instance has been finalized or not initialized。
143 1
|
19天前
|
人工智能 数据挖掘 API
基于neo4j数据库和dify大模型框架的rag模型搭建——后续补充
基于neo4j数据库和dify大模型框架的rag模型搭建——后续补充
140 21
基于neo4j数据库和dify大模型框架的rag模型搭建——后续补充
|
20天前
|
Java 数据库 Docker
基于neo4j数据库和dify大模型框架的rag模型搭建
基于neo4j数据库和dify大模型框架的rag模型搭建
235 35
|
22天前
|
机器学习/深度学习 人工智能 算法
大型多模态推理模型技术演进综述:从模块化架构到原生推理能力的综合分析
该研究系统梳理了大型多模态推理模型(LMRMs)的技术发展,从早期模块化架构到统一的语言中心框架,提出原生LMRMs(N-LMRMs)的前沿概念。论文划分三个技术演进阶段及一个前瞻性范式,深入探讨关键挑战与评估基准,为构建复杂动态环境中的稳健AI系统提供理论框架。未来方向聚焦全模态泛化、深度推理与智能体行为,推动跨模态融合与自主交互能力的发展。
99 13
大型多模态推理模型技术演进综述:从模块化架构到原生推理能力的综合分析
|
8天前
|
机器学习/深度学习 编解码 缓存
通义万相首尾帧图模型一键生成特效视频!
本文介绍了阿里通义发布的Wan2.1系列模型及其首尾帧生视频功能。该模型采用先进的DiT架构,通过高效的VAE模型降低运算成本,同时利用Full Attention机制确保生成视频的时间与空间一致性。模型训练分为三个阶段,逐步优化首尾帧生成能力及细节复刻效果。此外,文章展示了具体案例,并详细说明了训练和推理优化方法。目前,该模型已开源。
|
23天前
|
人工智能 编解码 搜索推荐
通义万相新模型开源,首尾帧图一键生成特效视频!
通义万相首尾帧生视频14B模型正式开源,作为首个百亿级参数规模的开源模型,可依据用户提供的开始与结束图片生成720p高清衔接视频,满足延时摄影、变身等定制化需求。用户上传两张图片或输入提示词即可完成复杂视频生成任务,支持运镜控制和特效变化。该模型基于Wan2.1架构改进,训练数据专门构建,确保高分辨率和流畅性。

热门文章

最新文章