DashVector&千问7B模型,打造极简RAG 之 基于专属知识的问答服务实践

本文涉及的产品
NLP 自学习平台,3个模型定制额度 1个月
NLP自然语言处理_高级版,每接口累计50万次
NLP自然语言处理_基础版,每接口每天50万次
简介: 参考官方教程和代码,以一个0基础学习者的角色,用DashVector&千问7B模型,打造极简RAG-基于专属知识的问答服务实践,也带入了自己的一些理解。

整体介绍

本教程演示如何使用向量检索服务(DashVector),结合LLM大模型等能力,来打造基于垂直领域专属知识等问答服务。其中LLM大模型能力,以及文本向量生成等能力,这里基于灵积模型服务上的通义千问 API以及Embedding API来接入。

考虑到有官方教程,亲测过,很多代码跑不起来,还有不少坑。我以一个0基础学习者的角色,参考官方代码重新写一个实践的guide,也带入了自己的一些理解。

大语言模型(LLM)作为自然语言处理领域的核心技术,具有丰富的自然语言处理能力。但其训练语料库具有一定的局限性,一般由普适知识、常识性知识,如维基百科、新闻、小说,和各种领域的专业知识组成。导致LLM在处理特定领域的知识表示和应用时存在一定的局限性,特别对于垂直领域内,或者企业内部等私域专属知识。特别是金融、医疗等比较封闭的行业。

image.png

一、创建资源

魔搭账号申请并领用算力

https://modelscope.cn/home

点到我的Notebook页面

绑定阿里云账号

免费申请并启动GPU,能申请免费36小时的GPU环境,不用就停掉,尽量省着点

模型服务灵积Dashscope免费领用

dashscope(灵积)是围绕AI各领域模型,通过标准化的API提供包括模型推理、模型微调训练在内的多种模型服务。

https://dashscope.console.aliyun.com/overview

免费领取灵积服务,并在api-key管理页面创建一个dashscope的api-key

免费开工通向量检索服务DashVector

DashVector是一款向量数据库,通过他的服务能有效提升向量检索效率,实现针对非结构化数据的高性能向量检索服务,可广泛应用于大模型搜索、多模态搜索、AI搜索、分子结构分析等几乎所有的 AI 搜索场景

https://dashvector.console.aliyun.com/cn-hangzhou/overview

到api-key管理页面创建一个key

到Cluster列表免费申领1个向量库实例(1个月免费)

这里看到的collection是在后续代码里把数据集加载进去的

创建好实例后,保存endpoiint

进入编码环境界面

来到我们的丘比特环境

二、代码是这样的

安装 灵积模型服务、向量数据库服务和transformer模型的依赖资源包

# 安装所需要的资源包
!pip install dashvector dashscope
!pip install transformers_stream_generator python-dotenv
!pip install --upgrade transformers

会看到这样的日志

Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: dashvector in /opt/conda/lib/python3.10/site-packages (1.0.11)
Requirement already satisfied: dashscope in /opt/conda/lib/python3.10/site-packages (1.14.1)
Requirement already satisfied: aiohttp<4.0.0,>=3.1.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.9.3)
Requirement already satisfied: certifi<2024.0.0,>=2023.7.22 in /opt/conda/lib/python3.10/site-packages (from dashvector) (2023.11.17)
Requirement already satisfied: grpcio>=1.22.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.60.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.26.3)
Requirement already satisfied: protobuf<4.0.0,>=3.8.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.20.3)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from dashscope) (2.31.0)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (4.0.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (1.26.16)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers_stream_generator in /opt/conda/lib/python3.10/site-packages (0.0.4)
Requirement already satisfied: python-dotenv in /opt/conda/lib/python3.10/site-packages (1.0.1)
Requirement already satisfied: transformers>=4.26.1 in /opt/conda/lib/python3.10/site-packages (from transformers_stream_generator) (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (4.38.2)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.20.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (23.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (2023.12.25)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.15.1)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.4.1)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers) (4.65.0)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (2023.10.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2023.11.17)
DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: pip install --upgrade pip

下载我们的专属数据集

CEC-Corpus 是一些中文突发事件语料库

能看到里面有很多txt文档的预料信息

我们把他clone下来

# prepare news corpus as knowledge source
!git clone https://github.com/shijiebei2009/CEC-Corpus.git

执行会看到这样的日志

初始化向量数据库对象,并以向量维度1536创建一个数据集

import dashscope
import os
from dotenv import load_dotenv
from dashscope import TextEmbedding
from dashvector import Client, Doc
dashscope.api_key = '{你的魔搭apikey}'
# 初始化DashVector向量库
dashvector_client = Client(api_key='{填写你的向量库apikey}',endpoint='{填写你的向量库endpoint}')
# 定义一个数据集名称
collection_name = 'news_embeddings'
# 如果重复就删掉这个数据集
dashvector_client.delete(collection_name)
# 在向量数据库下创建一个数据集,向量维度设置为1536
rsp = dashvector_client.create(collection_name, 1536)
collection = dashvector_client.get(collection_name)

这里你会发现你的dashvector多了一个名为news_embeddings的collection

定义一个函数,以遍历文件夹下所有的文件语料,写入向量数据库中

def prepare_data_from_dir(path, size):
    #把准备好的语料以适合的大小更新到DashVector
    batch_docs = []
    for file in os.listdir(path):
        with open(path + '/' + file, 'r', encoding='utf-8') as f:
            batch_docs.append(f.read())
            if len(batch_docs) == size:
                yield batch_docs[:]
                batch_docs.clear()
    if batch_docs:
        yield batch_docs

定义一个函数,将单个文件语料,写入向量数据库中 --这个函数后面没有用到

def prepare_data_from_file(path, size):
    # 把准备好的预料文件以合理的文档大小插入到 DashVector 中。
    batch_docs = []
    chunk_size = 12
    with open(path, 'r', encoding='utf-8') as f:
        doc = ''
        count = 0
        for line in f:
            if count < chunk_size and line.strip() != '':
                doc += line
                count += 1
            if count == chunk_size:
                batch_docs.append(doc)
                if len(batch_docs) == size:
                    yield batch_docs[:]
                    batch_docs.clear()
                doc = ''
                count = 0
    if batch_docs:
        yield batch_docs
def generate_embeddings(docs):
    # 通过魔搭的文本向量化api,将语料向量化
    rsp = TextEmbedding.call(model=TextEmbedding.Models.text_embedding_v1,
                             input=docs)
    embeddings = [record['embedding'] for record in rsp.output['embeddings']]
    return embeddings if isinstance(docs, list) else embeddings[0]

上面都是定义函数,这一步开始把原始文件词向量化到Dashvector中

id = 0
dir_name = 'CEC-Corpus/raw corpus/allSourceText'
# 得到DashVector向量数据库对象
collection = dashvector_client.get(collection_name)
# 最大批次大小设置为4
batch_size = 4  
for news in list(prepare_data_from_dir(dir_name, batch_size)):
    ids = [id + i for i, _ in enumerate(news)]
    id += len(news)
    # 通过原始文件生成词向量
    
    vectors = generate_embeddings(news)
    # upsert到向量库的collection中
    ret = collection.upsert(
        [
            Doc(id=str(id), vector=vector, fields={"raw": doc})
            for id, doc, vector in zip(ids, news, vectors)
        ]
    )
    print(ret)

这里会看到这样的原始文件词向量化的信息打印

这里我们可以回去 https://dashvector.console.aliyun.com/cn-hangzhou/cluster/wondashvector/collectionList 看看我们的collection中,已经有了很多数据

定义函数从collection中查询召回相关的信息优先级最高的一条,并尝试查询召回

def search_relevant_context(question, topk=1, client=dashvector_client):
    # 从collection中查询召回相关的信息
    collection = client.get(collection_name)
    # 从向量数据库召回优先级最高的一条信息
    rsp = collection.query(generate_embeddings(question), output_fields=['raw'],
                           topk=topk)
    return "".join([item.fields['raw'] for item in rsp.output])

先试试看能不能成功召回

# query the top 1 results
question = '有没有中毒相关的新闻?'
context = search_relevant_context(question, topk=1)
print(context)

下载并加载通义千问7B模型

加载通义千问7b模型

!git clone https://www.modelscope.cn/qwen/Qwen-7B-Chat.git

会看到这样的成功日志

初始化千问7B模型

# initialize qwen 7B model
from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("/mnt/workspace/Qwen-7B-Chat", device_map="auto", trust_remote_code=True).eval()
model.generation_config = GenerationConfig.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True) 
# 可指定不同的生成长度、top_p等相关超参

成功初始化会看到这样的信息

设定一个简单的问题,并开始我们的测试

设定提示词模版原因是能把之前从dashvector中检索的信息加入提示词

# define a prompt template for the vectorDB-enhanced LLM generation
def answer_question(question, context):
    prompt = f'''请基于```内的内容回答问题。"
    ```
    {context}
    ```
    我的问题是:{question}。
    '''
    history = None
    print(prompt)
    response, history = model.chat(tokenizer, prompt, history=None)
    return response

我们先试试,只查询千问7B模型的效果

# test the case on plain LLM without vectorDB enhancement
question = '有没有中毒相关的新闻?'
answer = answer_question(question, '')
print(f'question: {question}\n' f'answer: {answer}')

再看看加入了专属知识后的效果

# test the case with knowledge
context = search_relevant_context(question, topk=1)
answer = answer_question(question, context)
print(f'question: {question}\n' f'answer: {answer}')

相关文章
|
1月前
|
Go 开发工具
百炼-千问模型通过openai接口构建assistant 等 go语言
由于阿里百炼平台通义千问大模型没有完善的go语言兼容openapi示例,并且官方答复assistant是不兼容openapi sdk的。 实际使用中发现是能够支持的,所以自己写了一个demo test示例,给大家做一个参考。
|
4月前
|
人工智能 开发者
通义千问三款主力模型再降价,最高降幅85%
通义千问三款主力模型再降价,最高降幅85%
658 12
通义千问三款主力模型再降价,最高降幅85%
|
2天前
|
人工智能 编解码
通义万相2.1:VBench榜单荣登第一!阿里通义万相最新视频生成模型,支持生成1080P长视频
万相2.1是阿里通义万相最新推出的视频生成模型,支持1080P无限长视频生成,具备复杂动作展现、物理规律还原、艺术风格转换等功能。
102 26
通义万相2.1:VBench榜单荣登第一!阿里通义万相最新视频生成模型,支持生成1080P长视频
|
4月前
|
自然语言处理 算法 云栖大会
通义万相发布视频生成模型,更懂中国风、中国话
通义万相发布视频生成模型,可一键创作影视级高清视频
748 13
|
4月前
|
API 云栖大会
通义千问升级旗舰模型Qwen-Max,性能接近GPT-4o
通义旗舰模型Qwen-Max全方位升级,性能接近GPT-4o
1748 11
|
17天前
|
人工智能 测试技术 开发者
通义发布最强开源多模态推理模型QVQ!
通义发布最强开源多模态推理模型QVQ!
294 18
|
2月前
|
人工智能 自然语言处理
ACE:阿里通义实验室推出的全能图像生成和编辑模型
ACE是阿里巴巴通义实验室推出的全能图像生成和编辑模型,基于扩散变换器,支持多模态输入和多任务处理。该模型通过长上下文条件单元(LCU)和统一条件格式,能够理解和执行自然语言指令,实现图像生成、编辑和多轮交互等复杂任务,显著提升视觉内容创作的效率和灵活性。
148 3
ACE:阿里通义实验室推出的全能图像生成和编辑模型
|
2月前
|
机器学习/深度学习 人工智能 安全
通义千问开源的QwQ模型,一个会思考的AI,百炼邀您第一时间体验
Qwen团队推出新成员QwQ-32B-Preview,专注于增强AI推理能力。通过深入探索和试验,该模型在数学和编程领域展现了卓越的理解力,但仍在学习和完善中。目前,QwQ-32B-Preview已上线阿里云百炼平台,提供免费体验。
|
2月前
|
机器学习/深度学习 人工智能 安全
千问团队开源会思考的QwQ模型,这模型成精了!
QwQ是由Qwen团队开发的大型语言模型,专注于增强AI的推理能力。此预览版本在数学和编程领域表现出色,但在其他领域仍有提升空间。模型具备深度自省和自我质疑的能力,通过逐步推理和假设检验,能够在复杂问题上取得突破性进展。QwQ不仅支持本地推理和Ollama直接运行,还提供了详细的微调指南,助力开发者根据特定需求定制模型。尽管QwQ在推理过程中存在语言切换和安全性等方面的局限性,Qwen团队仍致力于不断优化,推动模型向更高层次的智能迈进。[了解更多](https://modelscope.cn/studios/Qwen/QwQ-32B-preview)
千问团队开源会思考的QwQ模型,这模型成精了!
|
3月前
|
人工智能 自然语言处理 安全
【通义】AI视界|Adobe推出文生视频AI模型,迎战OpenAI和Meta
本文精选了过去24小时内的重要科技新闻,包括微软人工智能副总裁跳槽至OpenAI、Adobe推出文本生成视频的AI模型、Meta取消高端头显转而开发超轻量设备、谷歌与核能公司合作为数据中心供电,以及英伟达股价创下新高,市值接近3.4万亿美元。这些动态展示了科技行业的快速发展和激烈竞争。点击链接或扫描二维码获取更多资讯。

热门文章

最新文章