DeepRec编译生成whl包

简介: DeepRec编译生成whl包

ECS 使用Ubuntu 20.04系统

安装docker

apt update
apt install docker.io -y

拉取镜像

docker pull alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
deeprec-cpu-py36-ubuntu18.04: Pulling from alideeprec/deeprec-tianchi
171857c49d0f: Pull complete 
419640447d26: Pull complete 
61e52f862619: Pull complete 
d9e2325ed518: Pull complete 
3181a77c167e: Pull complete 
4d90b1065554: Pull complete 
6f645ab01471: Pull complete 
caf5cc9788dc: Pull complete 
18425f078e83: Pull complete 
2ff8095e796c: Pull complete 
6673f27730f3: Pull complete 
aaf6201115e6: Pull complete 
da571ade1cfe: Pull complete 
387146fbd3ee: Pull complete 
e56823d3e32f: Pull complete 
3ffae41a9164: Pull complete 
18cb5b9d0b83: Downloading  1.448GB/5.785GB
d6adfdb806f3: Download complete 
a5db3e1b51b4: Downloading  903.2MB/1.565GB

拉取完成

d6adfdb806f3: Pull complete 
a5db3e1b51b4: Pull complete 
Digest: sha256:193dbdcc72e72578ce774d97049eb4f016bb79735ffaec08f4ed5ed9b8433b97
Status: Downloaded newer image for alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
docker.io/alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04

运行镜像启动bash

sudo docker run -ti --name=tianchi_test --net=host -v /code/:/code alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04 bash

查看/tianchi目录

ls /tianchi/
data  models

获取DeepRec代码,也可以在使用docker pull前执行

cd /code
git clone https://github.com/alibaba/DeepRec
Cloning into 'DeepRec'...
remote: Enumerating objects: 678325, done.
remote: Counting objects: 100% (465/465), done.
remote: Compressing objects: 100% (292/292), done.
Receiving objects:  16% (112984/678325), 107.69 MiB | 11.86 MiB/s

编译配置使用默认选项

./configure 
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.26.1 installed.
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: 
XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with STAR support? [Y/n]: 
STAR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with PMEM support? [y/N]: 
No PMEM support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GPU_EV support? [y/N]: 
No GPU_EV support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Parquet Dataset support? [Y/n]: 
Parquet Dataset support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: 
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: 
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with MPI support? [y/N]: 
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: 


Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl_threadpool # Build with oneDNN support.
        --config=monolithic     # Config for mostly static monolithic build.
        --config=gdr            # Build with GDR support.
        --config=verbs          # Build with libverbs support.
        --config=ngraph         # Build with Intel nGraph support.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=noaws          # Disable AWS S3 filesystem support.
        --config=nogcp          # Disable GCP support.
        --config=nohdfs         # Disable HDFS support.
        --config=noignite       # Disable Apache Ignite support.
        --config=nokafka        # Disable Apache Kafka support.
        --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished

编译

bazel build  -c opt --config=opt  --config=mkl_threadpool --define build_with_mkl_dnn_v1_only=true //tensorflow/tools/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=106
INFO: Reading rc options for 'build' from /code/DeepRec/.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --copt=-w --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=v1
INFO: Reading rc options for 'build' from /code/DeepRec/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/home/pai/bin/python --action_env PYTHON_LIB_PATH=/home/pai/lib/python3.6/site-packages --python_path=/home/pai/bin/python --config=xla --config=star --config=parquet_dataset --action_env TF_USE_CCACHE=0 --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:v1 in file /code/DeepRec/.bazelrc: --define=tf_api_version=1 --action_env=TF2_BEHAVIOR=0
INFO: Found applicable config definition build:xla in file /code/DeepRec/.bazelrc: --action_env=TF_ENABLE_XLA=1 --define=with_xla_support=true
INFO: Found applicable config definition build:xla in file /code/DeepRec/.tf_configure.bazelrc: --define with_xla_support=true
INFO: Found applicable config definition build:star in file /code/DeepRec/.tf_configure.bazelrc: --define with_star_support=true
INFO: Found applicable config definition build:parquet_dataset in file /code/DeepRec/.tf_configure.bazelrc: --define with_parquet_dataset_support=true
INFO: Found applicable config definition build:opt in file /code/DeepRec/.tf_configure.bazelrc: --copt=-march=native --copt=-Wno-sign-compare --host_copt=-march=native --define with_default_optimizations=true
INFO: Found applicable config definition build:mkl_threadpool in file /code/DeepRec/.bazelrc: --define=build_with_mkl=true --define=enable_mkl=true --define=tensorflow_dnnl_contraction_kernel=0 --define=build_with_mkl_opensource=true --define=build_with_mkldnn_threadpool=true -c opt
Loading: 0 packages loaded
    Fetching @build_bazel_rules_swift; fetching 15s

2022-10-02-13-15-59.png

生成whl包并安装

./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

pip install /tmp/tensorflow_pkg/tensorflow-1.15.5+deeprec2208-cp36-cp36m-linux_x86_64.whl 
目录
相关文章
|
2月前
|
Linux 数据库连接 数据库
Python如何将项目直接打包为一键整合包
Python如何将项目直接打包为一键整合包
49 0
|
5月前
|
定位技术 数据处理 Python
Anaconda环境GDAL库基于whl文件的配置方法
Anaconda环境GDAL库基于whl文件的配置方法
|
Linux 语音技术 Python
如何解决kaldi的依赖库mkl安装失败的问题
通过搜索大量的资料,但都发现不太适用。现在将失败的症状和解决方法分享一下,希望能给读者提供一些帮助。
如何解决kaldi的依赖库mkl安装失败的问题
|
5月前
ffmpeg的下载、编译与安装
ffmpeg的下载、编译与安装
106 0
|
12月前
|
并行计算 计算机视觉
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(一)
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(一)
135 0
|
12月前
|
并行计算 C++ 异构计算
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(二)
【CUDA学习笔记】第八篇:源码编译OpenCV+CUDA模块(完整源码打包一次成功编译)(二)
154 0
|
12月前
|
Python
Python 3.8.8 几个常用库的下载和离线包的制作安装
Python 3.8.8 几个常用库的下载和离线包的制作安装
1834 0
|
Python
【Python依赖库降低版本】
【Python依赖库降低版本】
202 0
|
Java
VS2017编译OpenJDK,编译通过的工程包下载链接
VS2017编译OpenJDK,编译通过的工程包下载链接
57 0
|
C++ Python
Python 技术篇-将项目打包成whl文件实例演示,whl包的制作方法
Python 技术篇-将项目打包成whl文件实例演示,whl包的制作方法
979 0
Python 技术篇-将项目打包成whl文件实例演示,whl包的制作方法