DeepRec编译生成whl包

简介: DeepRec编译生成whl包

ECS 使用Ubuntu 20.04系统

安装docker

apt update
apt install docker.io -y

拉取镜像

docker pull alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
deeprec-cpu-py36-ubuntu18.04: Pulling from alideeprec/deeprec-tianchi
171857c49d0f: Pull complete 
419640447d26: Pull complete 
61e52f862619: Pull complete 
d9e2325ed518: Pull complete 
3181a77c167e: Pull complete 
4d90b1065554: Pull complete 
6f645ab01471: Pull complete 
caf5cc9788dc: Pull complete 
18425f078e83: Pull complete 
2ff8095e796c: Pull complete 
6673f27730f3: Pull complete 
aaf6201115e6: Pull complete 
da571ade1cfe: Pull complete 
387146fbd3ee: Pull complete 
e56823d3e32f: Pull complete 
3ffae41a9164: Pull complete 
18cb5b9d0b83: Downloading  1.448GB/5.785GB
d6adfdb806f3: Download complete 
a5db3e1b51b4: Downloading  903.2MB/1.565GB

拉取完成

d6adfdb806f3: Pull complete 
a5db3e1b51b4: Pull complete 
Digest: sha256:193dbdcc72e72578ce774d97049eb4f016bb79735ffaec08f4ed5ed9b8433b97
Status: Downloaded newer image for alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
docker.io/alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04

运行镜像启动bash

sudo docker run -ti --name=tianchi_test --net=host -v /code/:/code alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04 bash

查看/tianchi目录

ls /tianchi/
data  models

获取DeepRec代码,也可以在使用docker pull前执行

cd /code
git clone https://github.com/alibaba/DeepRec
Cloning into 'DeepRec'...
remote: Enumerating objects: 678325, done.
remote: Counting objects: 100% (465/465), done.
remote: Compressing objects: 100% (292/292), done.
Receiving objects:  16% (112984/678325), 107.69 MiB | 11.86 MiB/s

编译配置使用默认选项

./configure 
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.26.1 installed.
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: 
XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with STAR support? [Y/n]: 
STAR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with PMEM support? [y/N]: 
No PMEM support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GPU_EV support? [y/N]: 
No GPU_EV support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Parquet Dataset support? [Y/n]: 
Parquet Dataset support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: 
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: 
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with MPI support? [y/N]: 
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: 


Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl_threadpool # Build with oneDNN support.
        --config=monolithic     # Config for mostly static monolithic build.
        --config=gdr            # Build with GDR support.
        --config=verbs          # Build with libverbs support.
        --config=ngraph         # Build with Intel nGraph support.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=noaws          # Disable AWS S3 filesystem support.
        --config=nogcp          # Disable GCP support.
        --config=nohdfs         # Disable HDFS support.
        --config=noignite       # Disable Apache Ignite support.
        --config=nokafka        # Disable Apache Kafka support.
        --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished

编译

bazel build  -c opt --config=opt  --config=mkl_threadpool --define build_with_mkl_dnn_v1_only=true //tensorflow/tools/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=106
INFO: Reading rc options for 'build' from /code/DeepRec/.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --copt=-w --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=v1
INFO: Reading rc options for 'build' from /code/DeepRec/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/home/pai/bin/python --action_env PYTHON_LIB_PATH=/home/pai/lib/python3.6/site-packages --python_path=/home/pai/bin/python --config=xla --config=star --config=parquet_dataset --action_env TF_USE_CCACHE=0 --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:v1 in file /code/DeepRec/.bazelrc: --define=tf_api_version=1 --action_env=TF2_BEHAVIOR=0
INFO: Found applicable config definition build:xla in file /code/DeepRec/.bazelrc: --action_env=TF_ENABLE_XLA=1 --define=with_xla_support=true
INFO: Found applicable config definition build:xla in file /code/DeepRec/.tf_configure.bazelrc: --define with_xla_support=true
INFO: Found applicable config definition build:star in file /code/DeepRec/.tf_configure.bazelrc: --define with_star_support=true
INFO: Found applicable config definition build:parquet_dataset in file /code/DeepRec/.tf_configure.bazelrc: --define with_parquet_dataset_support=true
INFO: Found applicable config definition build:opt in file /code/DeepRec/.tf_configure.bazelrc: --copt=-march=native --copt=-Wno-sign-compare --host_copt=-march=native --define with_default_optimizations=true
INFO: Found applicable config definition build:mkl_threadpool in file /code/DeepRec/.bazelrc: --define=build_with_mkl=true --define=enable_mkl=true --define=tensorflow_dnnl_contraction_kernel=0 --define=build_with_mkl_opensource=true --define=build_with_mkldnn_threadpool=true -c opt
Loading: 0 packages loaded
    Fetching @build_bazel_rules_swift; fetching 15s

2022-10-02-13-15-59.png

生成whl包并安装

./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

pip install /tmp/tensorflow_pkg/tensorflow-1.15.5+deeprec2208-cp36-cp36m-linux_x86_64.whl 
目录
相关文章
|
2月前
|
Linux 开发者
交叉编译工具链的下载与安装
交叉编译工具链的下载与安装
398 1
|
1月前
|
机器学习/深度学习 并行计算 计算机视觉
【入坑指南】| OpenCV4.8 + CUDA + 扩展模块支持编译
【入坑指南】| OpenCV4.8 + CUDA + 扩展模块支持编译
121 0
|
2月前
|
Python
python(pip)包/模块:如何离线安装?
python(pip)包/模块:如何离线安装?
21 0
|
2月前
|
Linux 数据库连接 数据库
Python如何将项目直接打包为一键整合包
Python如何将项目直接打包为一键整合包
139 0
|
2月前
|
Ubuntu Python
一键打包,随时运行,Python3项目虚拟环境一键整合包的制作(Venv)
之前我们介绍了[如何使用嵌入式 Python3 环境给项目制作一键整合包](https://v3u.cn/a_id_328),在使用嵌入式 Python 环境时,通常是作为另一个应用程序的一部分,而Python3虚拟环境是为了在开发过程中隔离项目所需的 Python 环境。虚拟环境允许我们在同一台计算机上的不同项目中使用不同的 Python 版本和软件包,而不会相互干扰。 本次我们利用Python3自带的虚拟环境(venv)功能来给项目制作一键整合包。
一键打包,随时运行,Python3项目虚拟环境一键整合包的制作(Venv)
|
Java TensorFlow 算法框架/工具
Tensorflow Lite移动平台编译|Bazel实践
如果不做定制化操作,我们不需要自己编译TensorFlow Lite Android库。我们可以直接使用位于MavenCentral的TensorFlow Lite AAR。但是在某些情况下,我们需要本地编译TensorFlow Lite。例如,您可能正在构建一个包含operations selected from TensorFlow的自定义二进制文件,或者需要对TensorFlow Lite进行局部修改。
367 0
|
存储 安全 Linux
python|如何将编写的模块打包上传至pypi
python|如何将编写的模块打包上传至pypi
232 0
|
并行计算 C++ 异构计算
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(二)
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(二)
165 0
|
并行计算 计算机视觉
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(一)
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(一)
185 0
|
Python
Python 3.8.8 几个常用库的下载和离线包的制作安装
Python 3.8.8 几个常用库的下载和离线包的制作安装
2168 0