整体介绍
本教程演示如何使用向量检索服务（DashVector），结合LLM大模型等能力，来打造基于垂直领域专属知识等问答服务。其中LLM大模型能力，以及文本向量生成等能力，这里基于灵积模型服务上的通义千问 API以及Embedding API来接入。
考虑到有官方教程，亲测过，很多代码跑不起来，还有不少坑。我以一个0基础学习者的角色，参考官方代码重新写一个实践的guide，也带入了自己的一些理解。
大语言模型（LLM）作为自然语言处理领域的核心技术，具有丰富的自然语言处理能力。但其训练语料库具有一定的局限性，一般由普适知识、常识性知识，如维基百科、新闻、小说，和各种领域的专业知识组成。导致LLM在处理特定领域的知识表示和应用时存在一定的局限性，特别对于垂直领域内，或者企业内部等私域专属知识。特别是金融、医疗等比较封闭的行业。
一、创建资源
魔搭账号申请并领用算力
点到我的Notebook页面
绑定阿里云账号
免费申请并启动GPU，能申请免费36小时的GPU环境，不用就停掉，尽量省着点
模型服务灵积Dashscope免费领用
dashscope（灵积）是围绕AI各领域模型，通过标准化的API提供包括模型推理、模型微调训练在内的多种模型服务。
https://dashscope.console.aliyun.com/overview
免费领取灵积服务，并在api-key管理页面创建一个dashscope的api-key
免费开工通向量检索服务DashVector
DashVector是一款向量数据库，通过他的服务能有效提升向量检索效率，实现针对非结构化数据的高性能向量检索服务，可广泛应用于大模型搜索、多模态搜索、AI搜索、分子结构分析等几乎所有的 AI 搜索场景
https://dashvector.console.aliyun.com/cn-hangzhou/overview
到api-key管理页面创建一个key
到Cluster列表免费申领1个向量库实例（1个月免费）
这里看到的collection是在后续代码里把数据集加载进去的
创建好实例后，保存endpoiint
进入编码环境界面
来到我们的丘比特环境
二、代码是这样的
安装 灵积模型服务、向量数据库服务和transformer模型的依赖资源包
# 安装所需要的资源包 !pip install dashvector dashscope !pip install transformers_stream_generator python-dotenv !pip install --upgrade transformers
会看到这样的日志
Looking in indexes: https://mirrors.aliyun.com/pypi/simple Requirement already satisfied: dashvector in /opt/conda/lib/python3.10/site-packages (1.0.11) Requirement already satisfied: dashscope in /opt/conda/lib/python3.10/site-packages (1.14.1) Requirement already satisfied: aiohttp<4.0.0,>=3.1.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.9.3) Requirement already satisfied: certifi<2024.0.0,>=2023.7.22 in /opt/conda/lib/python3.10/site-packages (from dashvector) (2023.11.17) Requirement already satisfied: grpcio>=1.22.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.60.0) Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from dashvector) (1.26.3) Requirement already satisfied: protobuf<4.0.0,>=3.8.0 in /opt/conda/lib/python3.10/site-packages (from dashvector) (3.20.3) Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from dashscope) (2.31.0) Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.3.1) Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (23.2.0) Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.4.1) Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (6.0.4) Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (1.9.4) Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.1.0->dashvector) (4.0.3) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->dashscope) (1.26.16) DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.3.2 -> 24.0 [notice] To update, run: pip install --upgrade pip Looking in indexes: https://mirrors.aliyun.com/pypi/simple Requirement already satisfied: transformers_stream_generator in /opt/conda/lib/python3.10/site-packages (0.0.4) Requirement already satisfied: python-dotenv in /opt/conda/lib/python3.10/site-packages (1.0.1) Requirement already satisfied: transformers>=4.26.1 in /opt/conda/lib/python3.10/site-packages (from transformers_stream_generator) (4.38.2) Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (3.13.1) Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.20.3) Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (1.26.3) Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (23.1) Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (6.0.1) Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2023.12.25) Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (2.31.0) Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.15.1) Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (0.4.1) Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->transformers_stream_generator) (4.65.0) Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (2023.10.0) Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.26.1->transformers_stream_generator) (4.9.0) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (1.26.16) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->transformers_stream_generator) (2023.11.17) DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.3.2 -> 24.0 [notice] To update, run: pip install --upgrade pip Looking in indexes: https://mirrors.aliyun.com/pypi/simple Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (4.38.2) Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers) (3.13.1) Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.20.3) Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (1.26.3) Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers) (23.1) Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (6.0.1) Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers) (2023.12.25) Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers) (2.31.0) Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.15.1) Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers) (0.4.1) Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers) (4.65.0) Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (2023.10.0) Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.9.0) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (1.26.16) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers) (2023.11.17) DEPRECATION: pytorch-lightning 1.7.7 has a non-standard dependency specifier torch>=1.9.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pytorch-lightning or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.3.2 -> 24.0 [notice] To update, run: pip install --upgrade pip
下载我们的专属数据集
CEC-Corpus 是一些中文突发事件语料库
能看到里面有很多txt文档的预料信息
我们把他clone下来
# prepare news corpus as knowledge source !git clone https://github.com/shijiebei2009/CEC-Corpus.git
执行会看到这样的日志
初始化向量数据库对象，并以向量维度1536创建一个数据集
import dashscope import os from dotenv import load_dotenv from dashscope import TextEmbedding from dashvector import Client, Doc dashscope.api_key = '{你的魔搭apikey}' # 初始化DashVector向量库 dashvector_client = Client(api_key='{填写你的向量库apikey}',endpoint='{填写你的向量库endpoint}') # 定义一个数据集名称 collection_name = 'news_embeddings' # 如果重复就删掉这个数据集 dashvector_client.delete(collection_name) # 在向量数据库下创建一个数据集，向量维度设置为1536 rsp = dashvector_client.create(collection_name, 1536) collection = dashvector_client.get(collection_name)
这里你会发现你的dashvector多了一个名为news_embeddings的collection
定义一个函数，以遍历文件夹下所有的文件语料，写入向量数据库中
def prepare_data_from_dir(path, size): #把准备好的语料以适合的大小更新到DashVector batch_docs = [] for file in os.listdir(path): with open(path + '/' + file, 'r', encoding='utf-8') as f: batch_docs.append(f.read()) if len(batch_docs) == size: yield batch_docs[:] batch_docs.clear() if batch_docs: yield batch_docs
定义一个函数，将单个文件语料，写入向量数据库中 --这个函数后面没有用到
def prepare_data_from_file(path, size): # 把准备好的预料文件以合理的文档大小插入到 DashVector 中。 batch_docs = [] chunk_size = 12 with open(path, 'r', encoding='utf-8') as f: doc = '' count = 0 for line in f: if count < chunk_size and line.strip() != '': doc += line count += 1 if count == chunk_size: batch_docs.append(doc) if len(batch_docs) == size: yield batch_docs[:] batch_docs.clear() doc = '' count = 0 if batch_docs: yield batch_docs
def generate_embeddings(docs): # 通过魔搭的文本向量化api，将语料向量化 rsp = TextEmbedding.call(model=TextEmbedding.Models.text_embedding_v1, input=docs) embeddings = [record['embedding'] for record in rsp.output['embeddings']] return embeddings if isinstance(docs, list) else embeddings[0]
上面都是定义函数，这一步开始把原始文件词向量化到Dashvector中
id = 0 dir_name = 'CEC-Corpus/raw corpus/allSourceText' # 得到DashVector向量数据库对象 collection = dashvector_client.get(collection_name) # 最大批次大小设置为4 batch_size = 4 for news in list(prepare_data_from_dir(dir_name, batch_size)): ids = [id + i for i, _ in enumerate(news)] id += len(news) # 通过原始文件生成词向量 vectors = generate_embeddings(news) # upsert到向量库的collection中 ret = collection.upsert( [ Doc(id=str(id), vector=vector, fields={"raw": doc}) for id, doc, vector in zip(ids, news, vectors) ] ) print(ret)
这里会看到这样的原始文件词向量化的信息打印
这里我们可以回去 https://dashvector.console.aliyun.com/cn-hangzhou/cluster/wondashvector/collectionList 看看我们的collection中，已经有了很多数据
定义函数从collection中查询召回相关的信息优先级最高的一条，并尝试查询召回
def search_relevant_context(question, topk=1, client=dashvector_client): # 从collection中查询召回相关的信息 collection = client.get(collection_name) # 从向量数据库召回优先级最高的一条信息 rsp = collection.query(generate_embeddings(question), output_fields=['raw'], topk=topk) return "".join([item.fields['raw'] for item in rsp.output])
先试试看能不能成功召回
# query the top 1 results question = '有没有中毒相关的新闻？' context = search_relevant_context(question, topk=1) print(context)
下载并加载通义千问7B模型
加载通义千问7b模型
!git clone https://www.modelscope.cn/qwen/Qwen-7B-Chat.git
会看到这样的成功日志
初始化千问7B模型
# initialize qwen 7B model from modelscope import AutoModelForCausalLM, AutoTokenizer from modelscope import GenerationConfig tokenizer = AutoTokenizer.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("/mnt/workspace/Qwen-7B-Chat", device_map="auto", trust_remote_code=True).eval() model.generation_config = GenerationConfig.from_pretrained("/mnt/workspace/Qwen-7B-Chat", trust_remote_code=True) # 可指定不同的生成长度、top_p等相关超参
成功初始化会看到这样的信息
设定一个简单的问题，并开始我们的测试
设定提示词模版原因是能把之前从dashvector中检索的信息加入提示词
# define a prompt template for the vectorDB-enhanced LLM generation def answer_question(question, context): prompt = f'''请基于```内的内容回答问题。" ``` {context} ``` 我的问题是：{question}。 ''' history = None print(prompt) response, history = model.chat(tokenizer, prompt, history=None) return response
我们先试试，只查询千问7B模型的效果
# test the case on plain LLM without vectorDB enhancement question = '有没有中毒相关的新闻？' answer = answer_question(question, '') print(f'question: {question}\n' f'answer: {answer}')
再看看加入了专属知识后的效果
# test the case with knowledge context = search_relevant_context(question, topk=1) answer = answer_question(question, context) print(f'question: {question}\n' f'answer: {answer}')
