DeepRec编译生成whl包

简介: DeepRec编译生成whl包

ECS 使用Ubuntu 20.04系统

安装docker

apt update
apt install docker.io -y

拉取镜像

docker pull alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
deeprec-cpu-py36-ubuntu18.04: Pulling from alideeprec/deeprec-tianchi
171857c49d0f: Pull complete 
419640447d26: Pull complete 
61e52f862619: Pull complete 
d9e2325ed518: Pull complete 
3181a77c167e: Pull complete 
4d90b1065554: Pull complete 
6f645ab01471: Pull complete 
caf5cc9788dc: Pull complete 
18425f078e83: Pull complete 
2ff8095e796c: Pull complete 
6673f27730f3: Pull complete 
aaf6201115e6: Pull complete 
da571ade1cfe: Pull complete 
387146fbd3ee: Pull complete 
e56823d3e32f: Pull complete 
3ffae41a9164: Pull complete 
18cb5b9d0b83: Downloading  1.448GB/5.785GB
d6adfdb806f3: Download complete 
a5db3e1b51b4: Downloading  903.2MB/1.565GB

拉取完成

d6adfdb806f3: Pull complete 
a5db3e1b51b4: Pull complete 
Digest: sha256:193dbdcc72e72578ce774d97049eb4f016bb79735ffaec08f4ed5ed9b8433b97
Status: Downloaded newer image for alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
docker.io/alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04

运行镜像启动bash

sudo docker run -ti --name=tianchi_test --net=host -v /code/:/code alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04 bash

查看/tianchi目录

ls /tianchi/
data  models

获取DeepRec代码,也可以在使用docker pull前执行

cd /code
git clone https://github.com/alibaba/DeepRec
Cloning into 'DeepRec'...
remote: Enumerating objects: 678325, done.
remote: Counting objects: 100% (465/465), done.
remote: Compressing objects: 100% (292/292), done.
Receiving objects:  16% (112984/678325), 107.69 MiB | 11.86 MiB/s

编译配置使用默认选项

./configure 
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.26.1 installed.
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: 
XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with STAR support? [Y/n]: 
STAR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with PMEM support? [y/N]: 
No PMEM support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GPU_EV support? [y/N]: 
No GPU_EV support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Parquet Dataset support? [Y/n]: 
Parquet Dataset support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: 
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: 
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with MPI support? [y/N]: 
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: 


Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl_threadpool # Build with oneDNN support.
        --config=monolithic     # Config for mostly static monolithic build.
        --config=gdr            # Build with GDR support.
        --config=verbs          # Build with libverbs support.
        --config=ngraph         # Build with Intel nGraph support.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=noaws          # Disable AWS S3 filesystem support.
        --config=nogcp          # Disable GCP support.
        --config=nohdfs         # Disable HDFS support.
        --config=noignite       # Disable Apache Ignite support.
        --config=nokafka        # Disable Apache Kafka support.
        --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished

编译

bazel build  -c opt --config=opt  --config=mkl_threadpool --define build_with_mkl_dnn_v1_only=true //tensorflow/tools/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=106
INFO: Reading rc options for 'build' from /code/DeepRec/.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --copt=-w --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=v1
INFO: Reading rc options for 'build' from /code/DeepRec/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/home/pai/bin/python --action_env PYTHON_LIB_PATH=/home/pai/lib/python3.6/site-packages --python_path=/home/pai/bin/python --config=xla --config=star --config=parquet_dataset --action_env TF_USE_CCACHE=0 --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:v1 in file /code/DeepRec/.bazelrc: --define=tf_api_version=1 --action_env=TF2_BEHAVIOR=0
INFO: Found applicable config definition build:xla in file /code/DeepRec/.bazelrc: --action_env=TF_ENABLE_XLA=1 --define=with_xla_support=true
INFO: Found applicable config definition build:xla in file /code/DeepRec/.tf_configure.bazelrc: --define with_xla_support=true
INFO: Found applicable config definition build:star in file /code/DeepRec/.tf_configure.bazelrc: --define with_star_support=true
INFO: Found applicable config definition build:parquet_dataset in file /code/DeepRec/.tf_configure.bazelrc: --define with_parquet_dataset_support=true
INFO: Found applicable config definition build:opt in file /code/DeepRec/.tf_configure.bazelrc: --copt=-march=native --copt=-Wno-sign-compare --host_copt=-march=native --define with_default_optimizations=true
INFO: Found applicable config definition build:mkl_threadpool in file /code/DeepRec/.bazelrc: --define=build_with_mkl=true --define=enable_mkl=true --define=tensorflow_dnnl_contraction_kernel=0 --define=build_with_mkl_opensource=true --define=build_with_mkldnn_threadpool=true -c opt
Loading: 0 packages loaded
    Fetching @build_bazel_rules_swift; fetching 15s

2022-10-02-13-15-59.png

生成whl包并安装

./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

pip install /tmp/tensorflow_pkg/tensorflow-1.15.5+deeprec2208-cp36-cp36m-linux_x86_64.whl 
目录
相关文章
|
存储 Rust 安全
Rust中的线程基础:创建与管理
本文将深入探讨Rust中的线程基础,包括线程的创建、同步与通信机制,以及Rust提供的线程管理工具。通过理解这些概念,读者将能够更好地利用Rust的并发特性,编写高效且安全的多线程程序。
|
JSON Java 定位技术
【Android App】GPS获取定位经纬度和根据经纬度获取详细地址讲解及实战(附源码和演示 超详细)
【Android App】GPS获取定位经纬度和根据经纬度获取详细地址讲解及实战(附源码和演示 超详细)
5379 1
|
网络协议 算法 网络性能优化
【流媒体】推流与拉流简介
【流媒体】推流与拉流简介
2175 0
|
消息中间件 存储 Kafka
Kafka 与 SQS:事件流工具深入比较
【8月更文挑战第13天】
600 0
|
人工智能 搜索推荐
强大的AI搜索引擎——秘塔AI搜索
【2月更文挑战第17天】强大的AI搜索引擎——秘塔AI搜索
5217 2
强大的AI搜索引擎——秘塔AI搜索
|
人工智能 Ubuntu Shell
【AI征文】DeepRec编译过程中新手最容易踩的3个坑
3个坑分别是: 一、windows不能通过Docker Desktop编译 二、没有改官方代码的目录导致名称占用 三、因不明原因导致configure能查找到,但是无法使用
711 1
【AI征文】DeepRec编译过程中新手最容易踩的3个坑
|
前端开发 Java 关系型数据库
xxl-job 登入功能集成 OIDC 统一认证
xxl-job 是一款 java 开发的、开源的分布式任务调度系统,自带了登录认证功能,不支持对接、扩展 LDAP 、OIDC 等标准认证系统,考虑到单独维护 xxl-job 自有的用户系统不方便,以及存在人员离职、调岗、权限变动等需要及时调整用户权限的情况,需要接入公司统一的 OIDC 认证系统
2231 0
|
机器学习/深度学习
牛客刷题(矩阵加圣诞树)(下)
牛客刷题(矩阵加圣诞树)
牛客刷题(矩阵加圣诞树)(下)
|
监控 Kubernetes Cloud Native
阿里云与 Kubecost 合作,容器服务 ACK 支持使用 Kubecost 进行成本管理
阿里云与 Kubecost 合作,容器服务 ACK 支持使用 Kubecost 进行成本管理
阿里云与 Kubecost 合作,容器服务 ACK 支持使用 Kubecost 进行成本管理