一、FastDeploy 安装
由于编译配置较为困难（本地环境可能有误），建议大家使用已编译好的whl包。
FastDeploy 预编编译Python Wheel包
FastDeploy提供了在Windows/Linux/Mac上的预先编译Python Wheel包，开发者可以直接下载后安装，也可以自行编译代码。
使用如下命令即可在Linux的Pythotn 3.8环境安装支持CPU部署的FastDeploy
python -m pip install fastdeploy_python-0.2.0-cp38-cp38-manylinux1_x86_64.whl
1.注意事项
- 不要重复安装
fastdeploy-python和fastdeploy-gpu-python - 如已安装CPU版本的
fastdeploy-python后，在安装GPU版本的fastdeploy-gpu-python，请先执行pip uninstall fastdeploy-python卸载已有版本
2.环境依赖
- cuda >= 11.2
- cudnn >= 8.0
3.下载地址
3.1 Linux x64平台
| GPU 安装包 | 硬件 | Python版本 |
| fastdeploy_gpu_python-0.2.0-cp36-cp36m-manylinux1_x86_64.whl | CPU/GPU | 3.6 |
| fastdeploy_gpu_python-0.2.0-cp37-cp37m-manylinux1_x86_64.whl | CPU/GPU | 3.7 |
| fastdeploy_gpu_python-0.2.0-cp38-cp38-manylinux1_x86_64.whl | CPU/GPU | 3.8 |
| fastdeploy_gpu_python-0.2.0-cp39-cp39-manylinux1_x86_64.whl | CPU/GPU | 3.9 |
3.2 Windows 10 x64平台
| CPU 安装包 | 硬件 | Python版本 |
| fastdeploy_python-0.2.0-cp38-cp38-win_amd64.whl | CPU | 3.8 |
| fastdeploy_python-0.2.0-cp39-cp39-win_amd64.whl | CPU | 3.9 |
| GPU 安装包 | 硬件 | Python版本 |
| fastdeploy_gpu_python-0.2.0-cp38-cp38-win_amd64.whl | CPU/GPU | 3.8 |
| fastdeploy_gpu_python-0.2.0-cp39-cp39-win_amd64.whl | CPU/GPU | 3.9 |
3.3 Linux aarch64平台
3.4 Mac OSX平台
| 架构 | 硬件 | 安装包 | Python版本 |
| x86_64 | CPU | fastdeploy_python-0.2.0-cp36-cp36m-macosx_10_14_x86_64.whl | 3.6 |
| x86_64 | CPU | fastdeploy_python-0.2.0-cp37-cp37m-macosx_10_14_x86_64.whl | 3.7 |
| x86_64 | CPU | fastdeploy_python-0.2.0-cp38-cp38-macosx_10_14_x86_64.whl | 3.8 |
| x86_64 | CPU | fastdeploy_python-0.2.0-cp39-cp39-macosx_10_14_x86_64.whl | 3.9 |
| arm64 | CPU | fastdeploy_python-0.2.0-cp38-cp38-macosx_11_0_arm64.whl | 3.8 |
| arm64 | CPU | fastdeploy_python-0.2.0-cp39-cp39-macosx_11_0_arm64.whl | 3.9 |
4.其它文档
二、实践
1.git下载
# git下载 !git clone -b develop https://github.com/PaddlePaddle/FastDeploy.git --depth=1
Cloning into 'FastDeploy'... remote: Enumerating objects: 1253, done.[K remote: Counting objects: 100% (1253/1253), done.[K remote: Compressing objects: 100% (965/965), done.[K remote: Total 1253 (delta 309), reused 1046 (delta 242), pack-reused 0[K Receiving objects: 100% (1253/1253), 2.37 MiB | 57.00 KiB/s, done. Resolving deltas: 100% (309/309), done. Checking connectivity... done.
2.查看系统环境
!python --version
Python 3.7.4
!nvidia-smi
Wed Aug 17 21:21:49 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla V100-SXM2... On | 00000000:43:00.0 Off | 0 | | N/A 42C P0 41W / 300W | 0MiB / 32510MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+
可见环境满足条件
- python 3.7
- cuda 11.
3.linux安装
!pip install https://bj.bcebos.com/paddlehub/fastdeploy/wheels/fastdeploy_gpu_python-0.2.0-cp37-cp37m-manylinux1_x86_64.whl
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting fastdeploy-gpu-python==0.2.0 Downloading https://bj.bcebos.com/paddlehub/fastdeploy/wheels/fastdeploy_gpu_python-0.2.0-cp37-cp37m-manylinux1_x86_64.whl (1508.4 MB) [2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.5/1.5 GB[0m [31m711.9 kB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m [?25hRequirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fastdeploy-gpu-python==0.2.0) (4.27.0) Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fastdeploy-gpu-python==0.2.0) (2.24.0) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (1.25.6) Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (2.8) Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (2019.9.11) Installing collected packages: fastdeploy-gpu-python Successfully installed fastdeploy-gpu-python-0.2.0 [1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m A new release of pip available: [0m[31;49m22.1.2[0m[39;49m -> [0m[32;49m22.2.2[0m [1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m To update, run: [0m[32;49mpip install --upgrade pip[0m
!python -m pip install --upgrade pip --use
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: pip in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (22.1.2) Collecting pip Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1f/2c/d9626f045e7b49a6225c6b09257861f24da78f4e5f23af2ddbdf852c99b8/pip-22.2.2-py3-none-any.whl (2.0 MB) [2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m2.0/2.0 MB[0m [31m4.1 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m [?25hInstalling collected packages: pip Successfully installed pip-22.2.2 [1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m A new release of pip available: [0m[31;49m22.1.2[0m[39;49m -> [0m[32;49m22.2.2[0m [1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m To update, run: [0m[32;49mpython -m pip install --upgrade pip[0m
4.运行
4.1下载PaddleSeg
!git clone https://gitee.com/PaddlePaddle/PaddleSeg.git --depth=1
Cloning into 'PaddleSeg'... remote: Enumerating objects: 1815, done.[K remote: Counting objects: 100% (1815/1815), done.[K remote: Compressing objects: 100% (1450/1450), done.[K remote: Total 1815 (delta 485), reused 1227 (delta 287), pack-reused 0[K Receiving objects: 100% (1815/1815), 48.94 MiB | 9.42 MiB/s, done. Resolving deltas: 100% (485/485), done. Checking connectivity... done.
4.2 获取预训练模型
%cd ~/PaddleSeg/ !mkdir bisenet %cd bisenet !wget https://paddleseg.bj.bcebos.com/dygraph/cityscapes/bisenet_cityscapes_1024x1024_160k/model.pdparams %cd ../..
/home/aistudio/PaddleSeg mkdir: cannot create directory ‘bisenet’: File exists /home/aistudio/PaddleSeg/bisenet --2022-08-17 21:41:56-- https://paddleseg.bj.bcebos.com/dygraph/cityscapes/bisenet_cityscapes_1024x1024_160k/model.pdparams Resolving paddleseg.bj.bcebos.com (paddleseg.bj.bcebos.com)... 100.67.200.6 Connecting to paddleseg.bj.bcebos.com (paddleseg.bj.bcebos.com)|100.67.200.6|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 15301992 (15M) [application/octet-stream] Saving to: ‘model.pdparams’ model.pdparams 100%[===================>] 14.59M 88.4MB/s in 0.2s 2022-08-17 21:41:56 (88.4 MB/s) - ‘model.pdparams’ saved [15301992/15301992] /home/aistudio
4.3导出静态模型
%cd ~/PaddleSeg/ # 设置1张可用的卡 !export CUDA_VISIBLE_DEVICES=0 # windows下请执行以下命令 # set CUDA_VISIBLE_DEVICES=0 !python export.py \ --config configs/bisenet/bisenet_cityscapes_1024x1024_160k.yml \ --model_path bisenet/model.pdparams \ --save_dir output \ --input_shape 1 3 512 1024
/home/aistudio/PaddleSeg /home/aistudio/PaddleSeg/paddleseg/models/losses/rmi_loss.py:78: DeprecationWarning: invalid escape sequence \i """ W0817 21:43:59.497787 2170 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 10.1 W0817 21:43:59.501492 2170 gpu_resources.cc:91] device: 0, cuDNN Version: 7.6. 2022-08-17 21:44:01 [INFO] Loaded trained params of model successfully. 2022-08-17 21:44:03 [INFO] Model is saved in output.
!ls output/
deploy.yaml model.pdiparams model.pdiparams.info model.pdmodel
三、预测
在部署前，需确认以下两个步骤
- 软硬件环境满足要求，参考FastDeploy环境要求
- FastDeploy Python whl包安装，参考FastDeploy Python安装
本目录下提供infer.py快速完成Unet在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成
- 下载部署示例代码 git clone https://github.com/PaddlePaddle/FastDeploy.git cd FastDeploy/examples/vision/segmentation/paddleseg/python # 下载Unet模型文件和测试图片 wget https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz tar -xvf Unet_cityscapes_without_argmax_infer.tgz wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png # CPU推理 python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device cpu # GPU推理 python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu # GPU上使用TensorRT推理 （注意：TensorRT推理第一次运行，有序列化模型的操作，有一定耗时，需要耐心等待） python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu --use_trt True
运行完成可视化结果如下图所示
1. PaddleSegModel Python接口
fd.vision.segmentation.PaddleSegModel(model_file, params_file, config_file, runtime_option=None, model_format=Frontend.PADDLE)
PaddleSeg模型加载和初始化，其中model_file, params_file以及config_file为训练模型导出的Paddle inference文件，具体请参考其文档说明模型导出
参数
- model_file(str): 模型文件路径
- params_file(str): 参数文件路径
- config_file(str): 推理部署配置文件
- runtime_option(RuntimeOption): 后端推理配置，默认为None，即采用默认配置
- model_format(Frontend): 模型格式，默认为Paddle格式
2.predict函数
PaddleSegModel.predict(input_image)码
模型预测结口，输入图像直接输出检测结果。
参数
- input_image(np.ndarray): 输入数据，注意需为HWC，BGR格式
返回
返回
fastdeploy.vision.SegmentationResult结构体，结构体说明参考文档视觉模型预测结果
3.类成员属性
3.1 预处理参数
用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果
- is_vertical_screen(bool): PP-HumanSeg系列模型通过设置此参数为
true表明输入图片是竖屏，即height大于width的图片
3.2 后处理参数
- with_softmax(bool): 当模型导出时，并未指定
with_softmax参数，可通过此设置此参数为true，将预测的输出分割标签（label_map）对应的概率结果(score_map)做softmax归一化处理
4.其它文档
