ECS 使用Ubuntu 20.04系统
安装docker
apt update
apt install docker.io -y
拉取镜像
docker pull alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
deeprec-cpu-py36-ubuntu18.04: Pulling from alideeprec/deeprec-tianchi
171857c49d0f: Pull complete
419640447d26: Pull complete
61e52f862619: Pull complete
d9e2325ed518: Pull complete
3181a77c167e: Pull complete
4d90b1065554: Pull complete
6f645ab01471: Pull complete
caf5cc9788dc: Pull complete
18425f078e83: Pull complete
2ff8095e796c: Pull complete
6673f27730f3: Pull complete
aaf6201115e6: Pull complete
da571ade1cfe: Pull complete
387146fbd3ee: Pull complete
e56823d3e32f: Pull complete
3ffae41a9164: Pull complete
18cb5b9d0b83: Downloading 1.448GB/5.785GB
d6adfdb806f3: Download complete
a5db3e1b51b4: Downloading 903.2MB/1.565GB
拉取完成
d6adfdb806f3: Pull complete
a5db3e1b51b4: Pull complete
Digest: sha256:193dbdcc72e72578ce774d97049eb4f016bb79735ffaec08f4ed5ed9b8433b97
Status: Downloaded newer image for alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
docker.io/alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04
运行镜像启动bash
sudo docker run -ti --name=tianchi_test --net=host -v /code/:/code alideeprec/deeprec-tianchi:deeprec-cpu-py36-ubuntu18.04 bash
查看/tianchi目录
ls /tianchi/
data models
获取DeepRec代码,也可以在使用docker pull前执行
cd /code
git clone https://github.com/alibaba/DeepRec
Cloning into 'DeepRec'...
remote: Enumerating objects: 678325, done.
remote: Counting objects: 100% (465/465), done.
remote: Compressing objects: 100% (292/292), done.
Receiving objects: 16% (112984/678325), 107.69 MiB | 11.86 MiB/s
编译配置使用默认选项
./configure
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.26.1 installed.
Do you wish to build TensorFlow with XLA JIT support? [Y/n]:
XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with STAR support? [Y/n]:
STAR support will be enabled for TensorFlow.
Do you wish to build TensorFlow with PMEM support? [y/N]:
No PMEM support will be enabled for TensorFlow.
Do you wish to build TensorFlow with GPU_EV support? [y/N]:
No GPU_EV support will be enabled for TensorFlow.
Do you wish to build TensorFlow with Parquet Dataset support? [Y/n]:
Parquet Dataset support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl_threadpool # Build with oneDNN support.
--config=monolithic # Config for mostly static monolithic build.
--config=gdr # Build with GDR support.
--config=verbs # Build with libverbs support.
--config=ngraph # Build with Intel nGraph support.
--config=numa # Build with NUMA support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
--config=v2 # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=noignite # Disable Apache Ignite support.
--config=nokafka # Disable Apache Kafka support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished
编译
bazel build -c opt --config=opt --config=mkl_threadpool --define build_with_mkl_dnn_v1_only=true //tensorflow/tools/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=106
INFO: Reading rc options for 'build' from /code/DeepRec/.bazelrc:
'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --copt=-w --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=v1
INFO: Reading rc options for 'build' from /code/DeepRec/.tf_configure.bazelrc:
'build' options: --action_env PYTHON_BIN_PATH=/home/pai/bin/python --action_env PYTHON_LIB_PATH=/home/pai/lib/python3.6/site-packages --python_path=/home/pai/bin/python --config=xla --config=star --config=parquet_dataset --action_env TF_USE_CCACHE=0 --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:v1 in file /code/DeepRec/.bazelrc: --define=tf_api_version=1 --action_env=TF2_BEHAVIOR=0
INFO: Found applicable config definition build:xla in file /code/DeepRec/.bazelrc: --action_env=TF_ENABLE_XLA=1 --define=with_xla_support=true
INFO: Found applicable config definition build:xla in file /code/DeepRec/.tf_configure.bazelrc: --define with_xla_support=true
INFO: Found applicable config definition build:star in file /code/DeepRec/.tf_configure.bazelrc: --define with_star_support=true
INFO: Found applicable config definition build:parquet_dataset in file /code/DeepRec/.tf_configure.bazelrc: --define with_parquet_dataset_support=true
INFO: Found applicable config definition build:opt in file /code/DeepRec/.tf_configure.bazelrc: --copt=-march=native --copt=-Wno-sign-compare --host_copt=-march=native --define with_default_optimizations=true
INFO: Found applicable config definition build:mkl_threadpool in file /code/DeepRec/.bazelrc: --define=build_with_mkl=true --define=enable_mkl=true --define=tensorflow_dnnl_contraction_kernel=0 --define=build_with_mkl_opensource=true --define=build_with_mkldnn_threadpool=true -c opt
Loading: 0 packages loaded
Fetching @build_bazel_rules_swift; fetching 15s
生成whl包并安装
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-1.15.5+deeprec2208-cp36-cp36m-linux_x86_64.whl