直接使用
请打开基于SwinTransformer的图像分类示例，并点击右上角 “ 在DSW中打开” 。
EasyCV图像分类-SwinTransformer
Swin Transformer是由微软亚研院提出的一种transformer-based通用骨干网络，通过Shifted windows结构来平衡计算效率及local window之间信息交互的问题，目前被广泛应用于图像分类、目标识别、图像分割领域，并取得sota的结果。
本文将介绍如何使用EasyCV快速使用Swin Transformer 进行图像分类模型的训练、推理
运行环境要求
PAI-Pytorch镜像 or 原生Pytorch1.5+以上环境 GPU机器， 内存32G以上
安装依赖包
1、获取torch和cuda版本，并根据版本号修改mmcv安装命令，安装对应版本的mmcv和nvidia-dali
import torch import os os.environ['CUDA']='cu' + torch.version.cuda.replace('.', '') os.environ['Torch']='torch'+torch.version.__version__.replace('+PAI', '') !echo "cuda version: $CUDA" !echo "pytorch version: $Torch"
[2022-07-29 06:44:32,493.493 dsw-142697-674d9f9968-cnbqm:6003 INFO utils.py:30] NOTICE: PAIDEBUGGER is turned off.
cuda version: cu101 pytorch version: torch1.8.2
# install some python deps ! pip install mmcv-full==1.4.4 -f https://download.openmmlab.com/mmcv/dist/${CUDA}/${Torch}/index.html ! pip install http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/third_party/nvidia_dali_cuda100-0.25.0-1535750-py3-none-manylinux2014_x86_64.whl
3、 安装EasyCV算法包
#pip install pai-easycv ! echo y | pip uninstall pai-easycv easycv ! pip install pai-easycv
4、 简单验证
from easycv.apis import *
[2022-07-29 06:46:00,778.778 dsw-142697-674d9f9968-cnbqm:6003 INFO font_manager.py:1423] Generating new fontManager, this may take some time...
Cifar10 分类
下面示例介绍如何利用cifar10数据，使用swin-tiny模型快速进行图像分类模型的训练评估、模型预测过程
数据准备
下载cifar10数据，解压到data/cifar目录， 目录结构如下
data/cifar └── cifar-10-batches-py ├── batches.meta ├── data_batch_1 ├── data_batch_2 ├── data_batch_3 ├── data_batch_4 ├── data_batch_5 ├── readme.html ├── read.py └── test_batch
! mkdir -p data/cifar && wget http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/cifar10/cifar-10-python.tar.gz && tar -zxf cifar-10-python.tar.gz -C data/cifar/
--2022-07-29 06:46:38-- http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/cifar10/cifar-10-python.tar.gz Resolving pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com... 39.98.20.13 Connecting to pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com|39.98.20.13|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 170498071 (163M) [application/gzip] Saving to: ‘cifar-10-python.tar.gz’ cifar-10-python.tar 100%[===================>] 162.60M 13.5MB/s in 14s 2022-07-29 06:46:52 (12.0 MB/s) - ‘cifar-10-python.tar.gz’ saved [170498071/170498071]
训练模型
下载训练配置文件
! rm -rf swintiny_b64_5e_jpg.py ! wget https://raw.githubusercontent.com/alibaba/EasyCV/master/configs/classification/cifar10/swintiny_b64_5e_jpg.py
--2022-07-29 06:47:00-- https://raw.githubusercontent.com/alibaba/EasyCV/master/configs/classification/cifar10/swintiny_b64_5e_jpg.py Resolving raw.githubusercontent.com... 185.199.111.133, 185.199.108.133, 185.199.109.133, ... Connecting to raw.githubusercontent.com|185.199.111.133|:443... connected. ^C
使用单卡gpu进行训练和验证集评估
! python -m easycv.tools.train swintiny_b64_5e_jpg.py --work_dir work_dirs/classification/cifar10/swin_tiny
[2022-07-29 06:47:58,145.145 dsw-142697-674d9f9968-cnbqm:6158 INFO utils.py:30] NOTICE: PAIDEBUGGER is turned off. Read base config from /home/pai/lib/python3.6/site-packages/easycv/configs/base.py /home/pai/lib/python3.6/site-packages/easycv/utils/setup_env.py:37: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. f'Setting OMP_NUM_THREADS environment variable for each process ' /home/pai/lib/python3.6/site-packages/easycv/utils/setup_env.py:47: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. f'Setting MKL_NUM_THREADS environment variable for each process ' 2022-07-29 06:48:04,949 - easycv - INFO - Environment info: ------------------------------------------------------------ sys.platform: linux Python: 3.6.12 |Anaconda, Inc.| (default, Sep 8 2020, 23:10:56) [GCC 7.3.0] CUDA available: True CUDA_HOME: /usr/local/cuda NVCC: Cuda compilation tools, release 10.1, V10.1.243 GPU 0: Tesla V100-SXM2-16GB GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.8.2+PAI PyTorch compiling details: PyTorch built with: - GCC 7.5 - C++ Version: 201402 - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v1.7.0 (Git Hash N/A) - OpenMP 201511 (a.k.a. OpenMP 4.5) - NNPACK is enabled - CPU capability usage: AVX2 - CUDA Runtime 10.1 - NVCC architecture flags: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_75,code=compute_75 - CuDNN 7.6.5 - Magma 2.5.2 - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.1, CUDNN_VERSION=7.6.5, CXX_COMPILER=/usr/lib/ccache/c++, CXX_FLAGS=-D_GLIBCXX_USE_CXX11_ABI=0 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, TorchVision: 0.9.2+cu101 OpenCV: 4.4.0 MMCV: 1.4.4 EasyCV: 0.5.0 ------------------------------------------------------------ 2022-07-29 06:48:04,949 - easycv - INFO - Distributed training: False 2022-07-29 06:48:04,949 - easycv - INFO - Config: /home/pai/lib/python3.6/site-packages/easycv/configs/base.py train_cfg = {} test_cfg = {} optimizer_config = dict() # grad_clip, coalesce, bucket_size_mb # yapf:disable log_config = dict( interval=50, hooks=[ dict(type='TextLoggerHook'), # dict(type='TensorboardLoggerHook') ]) # yapf:enable # runtime settings dist_params = dict(backend='nccl') cudnn_benchmark = False log_level = 'INFO' load_from = None resume_from = None workflow = [('train', 1)] /mnt/workspace/swintiny_b64_5e_jpg.py _base_ = '../../base.py' # model settings model = dict( type='Classification', backbone=dict( type='PytorchImageModelWrapper', model_name='swin_tiny_patch4_window7_224', num_classes=0, ), head=dict(type='ClsHead', in_channels=768, with_fc=True, num_classes=10)) # dataset settings class_list = [ 'airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck' ] data_source_cfg = dict(type='ClsSourceCifar10', root='data/cifar/') dataset_type = 'ClsDataset' img_norm_cfg = dict(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) train_pipeline = [ dict(type='RandomResizedCrop', size=224), dict(type='RandomHorizontalFlip'), dict( type='MMRandAugment', num_policies=2, total_level=10, magnitude_level=9, magnitude_std=0.5, hparams=dict( pad_val=[round(x * 255) for x in img_norm_cfg['mean'][::-1]], interpolation='bicubic')), dict( type='MMRandomErasing', erase_prob=0.25, mode='rand', min_area_ratio=0.02, max_area_ratio=1 / 3, fill_color=[x * 255 for x in img_norm_cfg['mean'][::-1]], fill_std=[x * 255 for x in img_norm_cfg['std'][::-1]]), dict(type='ToTensor'), dict(type='Normalize', **img_norm_cfg), dict(type='Collect', keys=['img', 'gt_labels']) ] test_pipeline = [ dict(type='Resize', size=256), dict(type='CenterCrop', size=224), dict(type='ToTensor'), dict(type='Normalize', **img_norm_cfg), dict(type='Collect', keys=['img', 'gt_labels']) ] data = dict( imgs_per_gpu=64, workers_per_gpu=2, train=dict( type=dataset_type, data_source=dict(split='train', **data_source_cfg), pipeline=train_pipeline), val=dict( type=dataset_type, data_source=dict(split='test', **data_source_cfg), pipeline=test_pipeline), test=dict( type=dataset_type, data_source=dict(split='test', **data_source_cfg), pipeline=test_pipeline)) # additional hooks eval_config = dict(initial=True, interval=1, gpu_collect=True) eval_pipelines = [ dict( mode='test', data=data['val'], dist_eval=True, evaluators=[dict(type='ClsEvaluator', topk=(1, 5))], ) ] custom_hooks = [] # optimizer paramwise_options = { 'norm': dict(weight_decay=0.), 'bias': dict(weight_decay=0.), 'absolute_pos_embed': dict(weight_decay=0.), 'relative_position_bias_table': dict(weight_decay=0.) } optimizer = dict( type='AdamW', lr=1e-4, weight_decay=0.05, eps=1e-8, betas=(0.9, 0.999), paramwise_options=paramwise_options) optimizer_config = dict(grad_clip=dict(max_norm=5.0), update_interval=2) # learning policy lr_config = dict( policy='CosineAnnealing', by_epoch=False, min_lr_ratio=1e-2, warmup='linear', warmup_ratio=1e-3, warmup_iters=20, warmup_by_epoch=True) checkpoint_config = dict(interval=5) # runtime settings total_epochs = 5 # log setting log_config = dict(interval=50) # export config export = dict(export_neck=True) 2022-07-29 06:48:04,950 - easycv - INFO - Config Dict: {"train_cfg": {}, "test_cfg": {}, "optimizer_config": {"grad_clip": {"max_norm": 5.0}, "update_interval": 2}, "log_config": {"interval": 50, "hooks": [{"type": "TextLoggerHook"}]}, "dist_params": {"backend": "nccl"}, "cudnn_benchmark": false, "log_level": "INFO", "load_from": null, "resume_from": null, "workflow": [["train", 1]], "model": {"type": "Classification", "backbone": {"type": "PytorchImageModelWrapper", "model_name": "swin_tiny_patch4_window7_224", "num_classes": 0}, "head": {"type": "ClsHead", "in_channels": 768, "with_fc": true, "num_classes": 10}}, "class_list": ["airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck"], "data_source_cfg": {"type": "ClsSourceCifar10", "root": "data/cifar/"}, "dataset_type": "ClsDataset", "img_norm_cfg": {"mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}, "train_pipeline": [{"type": "RandomResizedCrop", "size": 224}, {"type": "RandomHorizontalFlip"}, {"type": "MMRandAugment", "num_policies": 2, "total_level": 10, "magnitude_level": 9, "magnitude_std": 0.5, "hparams": {"pad_val": [104, 116, 124], "interpolation": "bicubic"}}, {"type": "MMRandomErasing", "erase_prob": 0.25, "mode": "rand", "min_area_ratio": 0.02, "max_area_ratio": 0.3333333333333333, "fill_color": [103.53, 116.28, 123.675], "fill_std": [57.375, 57.120000000000005, 58.395]}, {"type": "ToTensor"}, {"type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}, {"type": "Collect", "keys": ["img", "gt_labels"]}], "test_pipeline": [{"type": "Resize", "size": 256}, {"type": "CenterCrop", "size": 224}, {"type": "ToTensor"}, {"type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}, {"type": "Collect", "keys": ["img", "gt_labels"]}], "data": {"imgs_per_gpu": 64, "workers_per_gpu": 2, "train": {"type": "ClsDataset", "data_source": {"split": "train", "type": "ClsSourceCifar10", "root": "data/cifar/"}, "pipeline": [{"type": "RandomResizedCrop", "size": 224}, {"type": "RandomHorizontalFlip"}, {"type": "MMRandAugment", "num_policies": 2, "total_level": 10, "magnitude_level": 9, "magnitude_std": 0.5, "hparams": {"pad_val": [104, 116, 124], "interpolation": "bicubic"}}, {"type": "MMRandomErasing", "erase_prob": 0.25, "mode": "rand", "min_area_ratio": 0.02, "max_area_ratio": 0.3333333333333333, "fill_color": [103.53, 116.28, 123.675], "fill_std": [57.375, 57.120000000000005, 58.395]}, {"type": "ToTensor"}, {"type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}, {"type": "Collect", "keys": ["img", "gt_labels"]}]}, "val": {"type": "ClsDataset", "data_source": {"split": "test", "type": "ClsSourceCifar10", "root": "data/cifar/"}, "pipeline": [{"type": "Resize", "size": 256}, {"type": "CenterCrop", "size": 224}, {"type": "ToTensor"}, {"type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}, {"type": "Collect", "keys": ["img", "gt_labels"]}]}, "test": {"type": "ClsDataset", "data_source": {"split": "test", "type": "ClsSourceCifar10", "root": "data/cifar/"}, "pipeline": [{"type": "Resize", "size": 256}, {"type": "CenterCrop", "size": 224}, {"type": "ToTensor"}, {"type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}, {"type": "Collect", "keys": ["img", "gt_labels"]}]}}, "eval_config": {"initial": true, "interval": 1, "gpu_collect": true}, "eval_pipelines": [{"mode": "test", "data": {"type": "ClsDataset", "data_source": {"split": "test", "type": "ClsSourceCifar10", "root": "data/cifar/"}, "pipeline": [{"type": "Resize", "size": 256}, {"type": "CenterCrop", "size": 224}, {"type": "ToTensor"}, {"type": "Normalize", "mean": [0.485, 0.456, 0.406], "std": [0.229, 0.224, 0.225]}, {"type": "Collect", "keys": ["img", "gt_labels"]}]}, "dist_eval": true, "evaluators": [{"type": "ClsEvaluator", "topk": [1, 5]}]}], "custom_hooks": [], "paramwise_options": {"norm": {"weight_decay": 0.0}, "bias": {"weight_decay": 0.0}, "absolute_pos_embed": {"weight_decay": 0.0}, "relative_position_bias_table": {"weight_decay": 0.0}}, "optimizer": {"type": "AdamW", "lr": 0.0001, "weight_decay": 0.05, "eps": 1e-08, "betas": [0.9, 0.999], "paramwise_options": {"norm": {"weight_decay": 0.0}, "bias": {"weight_decay": 0.0}, "absolute_pos_embed": {"weight_decay": 0.0}, "relative_position_bias_table": {"weight_decay": 0.0}}}, "lr_config": {"policy": "CosineAnnealing", "by_epoch": false, "min_lr_ratio": 0.01, "warmup": "linear", "warmup_ratio": 0.001, "warmup_iters": 20, "warmup_by_epoch": true}, "checkpoint_config": {"interval": 5}, "total_epochs": 5, "export": {"export_neck": true}, "work_dir": "work_dirs/classification/cifar10/swin_tiny", "oss_work_dir": null, "gpus": 1} 2022-07-29 06:48:04,950 - easycv - INFO - GPU INFO : Tesla V100-SXM2-16GB 2022-07-29 06:48:05,430 - easycv - INFO - load model from default path: https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/classification/timm/swint/swin_tiny_patch4_window7_224.pth Downloading: "https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/classification/timm/swint/swin_tiny_patch4_window7_224.pth" to /root/.cache/torch/hub/checkpoints/swin_tiny_patch4_window7_224.pth Classification( (backbone): PytorchImageModelWrapper( (model): SwinTransformer( (patch_embed): PatchEmbed( (proj): Conv2d(3, 96, kernel_size=(4, 4), stride=(4, 4)) (norm): LayerNorm((96,), eps=1e-05, elementwise_affine=True) ) (pos_drop): Dropout(p=0.0, inplace=False) (layers): Sequential( (0): BasicLayer( dim=96, input_resolution=(56, 56), depth=2 (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=96, out_features=288, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=96, out_features=96, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): Identity() (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=96, out_features=384, bias=True) (act): GELU() (fc2): Linear(in_features=384, out_features=96, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=96, out_features=288, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=96, out_features=96, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=96, out_features=384, bias=True) (act): GELU() (fc2): Linear(in_features=384, out_features=96, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) ) (downsample): PatchMerging( input_resolution=(56, 56), dim=96 (reduction): Linear(in_features=384, out_features=192, bias=False) (norm): LayerNorm((384,), eps=1e-05, elementwise_affine=True) ) ) (1): BasicLayer( dim=192, input_resolution=(28, 28), depth=2 (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=192, out_features=576, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=192, out_features=192, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=192, out_features=768, bias=True) (act): GELU() (fc2): Linear(in_features=768, out_features=192, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=192, out_features=576, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=192, out_features=192, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=192, out_features=768, bias=True) (act): GELU() (fc2): Linear(in_features=768, out_features=192, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) ) (downsample): PatchMerging( input_resolution=(28, 28), dim=192 (reduction): Linear(in_features=768, out_features=384, bias=False) (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True) ) ) (2): BasicLayer( dim=384, input_resolution=(14, 14), depth=6 (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=384, out_features=1152, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=384, out_features=384, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=384, out_features=1536, bias=True) (act): GELU() (fc2): Linear(in_features=1536, out_features=384, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=384, out_features=1152, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=384, out_features=384, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=384, out_features=1536, bias=True) (act): GELU() (fc2): Linear(in_features=1536, out_features=384, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (2): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=384, out_features=1152, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=384, out_features=384, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=384, out_features=1536, bias=True) (act): GELU() (fc2): Linear(in_features=1536, out_features=384, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (3): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=384, out_features=1152, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=384, out_features=384, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=384, out_features=1536, bias=True) (act): GELU() (fc2): Linear(in_features=1536, out_features=384, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (4): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=384, out_features=1152, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=384, out_features=384, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=384, out_features=1536, bias=True) (act): GELU() (fc2): Linear(in_features=1536, out_features=384, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (5): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=384, out_features=1152, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=384, out_features=384, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=384, out_features=1536, bias=True) (act): GELU() (fc2): Linear(in_features=1536, out_features=384, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) ) (downsample): PatchMerging( input_resolution=(14, 14), dim=384 (reduction): Linear(in_features=1536, out_features=768, bias=False) (norm): LayerNorm((1536,), eps=1e-05, elementwise_affine=True) ) ) (3): BasicLayer( dim=768, input_resolution=(7, 7), depth=2 (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=768, out_features=2304, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=768, out_features=768, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=768, out_features=3072, bias=True) (act): GELU() (fc2): Linear(in_features=3072, out_features=768, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear(in_features=768, out_features=2304, bias=True) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear(in_features=768, out_features=768, bias=True) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=768, out_features=3072, bias=True) (act): GELU() (fc2): Linear(in_features=3072, out_features=768, bias=True) (drop): Dropout(p=0.0, inplace=False) ) ) ) ) ) (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (avgpool): AdaptiveAvgPool1d(output_size=1) (head): Identity() ) ) (head_0): ClsHead( (criterion): CrossEntropyLossWithLabelSmooth( (log_softmax): LogSoftmax(dim=1) (nllloss): NLLLoss() (cls): Identity() ) (fc_cls): Linear(in_features=768, out_features=10, bias=True) ) (avg_pool): AdaptiveAvgPool2d(output_size=(1, 1)) (activate_fn): Softmax(dim=1) ) data shuffle: True GPU INFO : Tesla V100-SXM2-16GB paramwise_options -- backbone.model.patch_embed.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.patch_embed.norm.weight: weight_decay=0.0 paramwise_options -- backbone.model.patch_embed.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.patch_embed.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.0.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.blocks.1.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.downsample.norm.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.downsample.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.0.downsample.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.0.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.blocks.1.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.downsample.norm.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.downsample.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.1.downsample.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.0.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.1.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.2.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.3.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.4.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.blocks.5.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.downsample.norm.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.downsample.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.2.downsample.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.0.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.norm1.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.norm1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.attn.relative_position_bias_table: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.attn.qkv.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.attn.proj.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.norm2.weight: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.norm2.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.mlp.fc1.bias: weight_decay=0.0 paramwise_options -- backbone.model.layers.3.blocks.1.mlp.fc2.bias: weight_decay=0.0 paramwise_options -- backbone.model.norm.weight: weight_decay=0.0 paramwise_options -- backbone.model.norm.bias: weight_decay=0.0 paramwise_options -- backbone.model.norm.bias: weight_decay=0.0 paramwise_options -- head_0.fc_cls.bias: weight_decay=0.0 2022-07-29 06:48:19,488 - easycv - INFO - open validate hook 2022-07-29 06:48:19,916 - easycv - INFO - register EvaluationHook {'initial': True, 'evaluators': [<easycv.core.evaluation.classification_eval.ClsEvaluator object at 0x7f928dab1198>]} 2022-07-29 06:48:19,916 - easycv - INFO - Start running, host: root@dsw-142697-674d9f9968-cnbqm, work_dir: /mnt/workspace/work_dirs/classification/cifar10/swin_tiny 2022-07-29 06:48:19,917 - easycv - INFO - Hooks will be executed in the following order: before_run: (VERY_HIGH ) CosineAnnealingLrUpdaterHook (ABOVE_NORMAL) OptimizerHook (NORMAL ) CheckpointHook (NORMAL ) EvalHook (NORMAL ) BestCkptSaverHook (VERY_LOW ) PreLoggerHook (VERY_LOW ) TextLoggerHook -------------------- before_train_epoch: (VERY_HIGH ) CosineAnnealingLrUpdaterHook (LOW ) IterTimerHook (VERY_LOW ) PreLoggerHook (VERY_LOW ) TextLoggerHook -------------------- before_train_iter: (VERY_HIGH ) CosineAnnealingLrUpdaterHook (LOW ) IterTimerHook -------------------- after_train_iter: (ABOVE_NORMAL) OptimizerHook (NORMAL ) CheckpointHook (LOW ) IterTimerHook (VERY_LOW ) PreLoggerHook (VERY_LOW ) TextLoggerHook -------------------- after_train_epoch: (NORMAL ) CheckpointHook (NORMAL ) EvalHook (NORMAL ) BestCkptSaverHook (VERY_LOW ) PreLoggerHook (VERY_LOW ) TextLoggerHook -------------------- before_val_epoch: (LOW ) IterTimerHook (VERY_LOW ) PreLoggerHook (VERY_LOW ) TextLoggerHook -------------------- before_val_iter: (LOW ) IterTimerHook -------------------- after_val_iter: (LOW ) IterTimerHook -------------------- after_val_epoch: (VERY_LOW ) PreLoggerHook (VERY_LOW ) TextLoggerHook -------------------- after_run: (VERY_LOW ) TextLoggerHook -------------------- 2022-07-29 06:48:19,918 - easycv - INFO - workflow: [('train', 1)], max: 5 epochs 2022-07-29 06:48:19,919 - easycv - INFO - Checkpoints will be saved to /mnt/workspace/work_dirs/classification/cifar10/swin_tiny by HardDiskBackend. [>>>>>>>>>>>>>>>>>>>>>>>>] 10000/10000, 484.2 task/s, elapsed: 21s, ETA: 0s 2022-07-29 06:48:40,618 - easycv - INFO - SaveBest metric_name: ['ClsEvaluator_neck_top1'] 2022-07-29 06:48:40,848 - easycv - INFO - End SaveBest metric 2022-07-29 06:48:59,610 - easycv - INFO - Epoch [1][50/782] lr: 4.128e-07, eta: 0:24:05, time: 0.375, data_time: 0.055, memory: 7543, loss: 1.1466 2022-07-29 06:49:16,016 - easycv - INFO - Epoch [1][100/782] lr: 7.312e-07, eta: 0:22:18, time: 0.328, data_time: 0.008, memory: 7543, loss: 1.1416 2022-07-29 06:49:32,443 - easycv - INFO - Epoch [1][150/782] lr: 1.048e-06, eta: 0:21:32, time: 0.329, data_time: 0.008, memory: 7543, loss: 1.1374 2022-07-29 06:49:48,878 - easycv - INFO - Epoch [1][200/782] lr: 1.362e-06, eta: 0:21:01, time: 0.329, data_time: 0.008, memory: 7543, loss: 1.1291 2022-07-29 06:50:05,331 - easycv - INFO - Epoch [1][250/782] lr: 1.674e-06, eta: 0:20:36, time: 0.329, data_time: 0.009, memory: 7543, loss: 1.1226 2022-07-29 06:50:21,803 - easycv - INFO - Epoch [1][300/782] lr: 1.981e-06, eta: 0:20:14, time: 0.329, data_time: 0.009, memory: 7543, loss: 1.1092 2022-07-29 06:50:38,274 - easycv - INFO - Epoch [1][350/782] lr: 2.284e-06, eta: 0:19:53, time: 0.329, data_time: 0.009, memory: 7543, loss: 1.0938 2022-07-29 06:50:54,750 - easycv - INFO - Epoch [1][400/782] lr: 2.582e-06, eta: 0:19:34, time: 0.330, data_time: 0.009, memory: 7543, loss: 1.0705 2022-07-29 06:51:11,245 - easycv - INFO - Epoch [1][450/782] lr: 2.873e-06, eta: 0:19:16, time: 0.330, data_time: 0.009, memory: 7543, loss: 1.0445 2022-07-29 06:51:27,733 - easycv - INFO - Epoch [1][500/782] lr: 3.158e-06, eta: 0:18:57, time: 0.330, data_time: 0.009, memory: 7543, loss: 0.9987 2022-07-29 06:51:44,241 - easycv - INFO - Epoch [1][550/782] lr: 3.436e-06, eta: 0:18:40, time: 0.330, data_time: 0.009, memory: 7543, loss: 0.9462 2022-07-29 06:52:00,791 - easycv - INFO - Epoch [1][600/782] lr: 3.705e-06, eta: 0:18:22, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.8853 2022-07-29 06:52:17,339 - easycv - INFO - Epoch [1][650/782] lr: 3.966e-06, eta: 0:18:05, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.8349 2022-07-29 06:52:33,915 - easycv - INFO - Epoch [1][700/782] lr: 4.218e-06, eta: 0:17:48, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.7912 2022-07-29 06:52:50,481 - easycv - INFO - Epoch [1][750/782] lr: 4.459e-06, eta: 0:17:31, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.7605 [>>>>>>>>>>>>>>>>>>>>>>>>] 10000/10000, 531.5 task/s, elapsed: 19s, ETA: 0s 2022-07-29 06:53:19,733 - easycv - INFO - SaveBest metric_name: ['ClsEvaluator_neck_top1'] 2022-07-29 06:53:20,617 - easycv - INFO - End SaveBest metric 2022-07-29 06:53:20,618 - easycv - INFO - Epoch(val) [1][782] ClsEvaluator_neck_top1: 81.7700, neck_top1: 81.7700, neck_top5: 99.2500 2022-07-29 06:53:39,552 - easycv - INFO - Epoch [2][50/782] lr: 4.833e-06, eta: 0:16:33, time: 0.378, data_time: 0.056, memory: 7543, loss: 0.7205 2022-07-29 06:53:56,044 - easycv - INFO - Epoch [2][100/782] lr: 5.046e-06, eta: 0:16:18, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.6969 2022-07-29 06:54:12,570 - easycv - INFO - Epoch [2][150/782] lr: 5.248e-06, eta: 0:16:03, time: 0.331, data_time: 0.008, memory: 7543, loss: 0.6773 2022-07-29 06:54:29,102 - easycv - INFO - Epoch [2][200/782] lr: 5.437e-06, eta: 0:15:48, time: 0.331, data_time: 0.008, memory: 7543, loss: 0.6596 2022-07-29 06:54:45,636 - easycv - INFO - Epoch [2][250/782] lr: 5.613e-06, eta: 0:15:33, time: 0.331, data_time: 0.008, memory: 7543, loss: 0.6641 2022-07-29 06:55:02,177 - easycv - INFO - Epoch [2][300/782] lr: 5.777e-06, eta: 0:15:17, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.6380 2022-07-29 06:55:18,721 - easycv - INFO - Epoch [2][350/782] lr: 5.927e-06, eta: 0:15:02, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.6203 2022-07-29 06:55:35,271 - easycv - INFO - Epoch [2][400/782] lr: 6.064e-06, eta: 0:14:46, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.6249 2022-07-29 06:55:51,796 - easycv - INFO - Epoch [2][450/782] lr: 6.187e-06, eta: 0:14:31, time: 0.330, data_time: 0.009, memory: 7543, loss: 0.6121 2022-07-29 06:56:08,342 - easycv - INFO - Epoch [2][500/782] lr: 6.296e-06, eta: 0:14:15, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.6135 2022-07-29 06:56:24,905 - easycv - INFO - Epoch [2][550/782] lr: 6.390e-06, eta: 0:13:59, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.6061 2022-07-29 06:56:41,469 - easycv - INFO - Epoch [2][600/782] lr: 6.470e-06, eta: 0:13:43, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5945 2022-07-29 06:56:58,045 - easycv - INFO - Epoch [2][650/782] lr: 6.536e-06, eta: 0:13:28, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.5942 2022-07-29 06:57:14,617 - easycv - INFO - Epoch [2][700/782] lr: 6.587e-06, eta: 0:13:12, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5834 2022-07-29 06:57:31,196 - easycv - INFO - Epoch [2][750/782] lr: 6.623e-06, eta: 0:12:56, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.5798 [>>>>>>>>>>>>>>>>>>>>>>>>] 10000/10000, 531.7 task/s, elapsed: 19s, ETA: 0s 2022-07-29 06:58:00,449 - easycv - INFO - SaveBest metric_name: ['ClsEvaluator_neck_top1'] 2022-07-29 06:58:01,307 - easycv - INFO - End SaveBest metric 2022-07-29 06:58:01,307 - easycv - INFO - Epoch(val) [2][782] ClsEvaluator_neck_top1: 92.1600, neck_top1: 92.1600, neck_top5: 99.8700 2022-07-29 06:58:20,225 - easycv - INFO - Epoch [3][50/782] lr: 6.651e-06, eta: 0:12:18, time: 0.378, data_time: 0.056, memory: 7543, loss: 0.5729 2022-07-29 06:58:36,710 - easycv - INFO - Epoch [3][100/782] lr: 6.649e-06, eta: 0:12:02, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.5642 2022-07-29 06:58:53,211 - easycv - INFO - Epoch [3][150/782] lr: 6.632e-06, eta: 0:11:47, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.5743 2022-07-29 06:59:09,736 - easycv - INFO - Epoch [3][200/782] lr: 6.601e-06, eta: 0:11:31, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.5744 2022-07-29 06:59:26,269 - easycv - INFO - Epoch [3][250/782] lr: 6.557e-06, eta: 0:11:15, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5725 2022-07-29 06:59:42,796 - easycv - INFO - Epoch [3][300/782] lr: 6.499e-06, eta: 0:11:00, time: 0.331, data_time: 0.008, memory: 7543, loss: 0.5746 2022-07-29 06:59:59,337 - easycv - INFO - Epoch [3][350/782] lr: 6.427e-06, eta: 0:10:44, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5600 2022-07-29 07:00:15,894 - easycv - INFO - Epoch [3][400/782] lr: 6.342e-06, eta: 0:10:28, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5519 2022-07-29 07:00:32,462 - easycv - INFO - Epoch [3][450/782] lr: 6.245e-06, eta: 0:10:13, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5552 2022-07-29 07:00:49,025 - easycv - INFO - Epoch [3][500/782] lr: 6.135e-06, eta: 0:09:57, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5465 2022-07-29 07:01:05,586 - easycv - INFO - Epoch [3][550/782] lr: 6.014e-06, eta: 0:09:41, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5589 2022-07-29 07:01:22,157 - easycv - INFO - Epoch [3][600/782] lr: 5.882e-06, eta: 0:09:25, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5475 2022-07-29 07:01:38,728 - easycv - INFO - Epoch [3][650/782] lr: 5.739e-06, eta: 0:09:09, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5512 2022-07-29 07:01:55,309 - easycv - INFO - Epoch [3][700/782] lr: 5.585e-06, eta: 0:08:53, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.5482 2022-07-29 07:02:11,879 - easycv - INFO - Epoch [3][750/782] lr: 5.423e-06, eta: 0:08:37, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5387 [>>>>>>>>>>>>>>>>>>>>>>>>] 10000/10000, 528.8 task/s, elapsed: 19s, ETA: 0s 2022-07-29 07:02:41,242 - easycv - INFO - SaveBest metric_name: ['ClsEvaluator_neck_top1'] 2022-07-29 07:02:42,104 - easycv - INFO - End SaveBest metric 2022-07-29 07:02:42,105 - easycv - INFO - Epoch(val) [3][782] ClsEvaluator_neck_top1: 94.1200, neck_top1: 94.1200, neck_top5: 99.9400 2022-07-29 07:03:01,065 - easycv - INFO - Epoch [4][50/782] lr: 5.137e-06, eta: 0:08:06, time: 0.378, data_time: 0.057, memory: 7543, loss: 0.5471 2022-07-29 07:03:17,556 - easycv - INFO - Epoch [4][100/782] lr: 4.953e-06, eta: 0:07:50, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.5307 2022-07-29 07:03:34,078 - easycv - INFO - Epoch [4][150/782] lr: 4.761e-06, eta: 0:07:34, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.5333 2022-07-29 07:03:50,597 - easycv - INFO - Epoch [4][200/782] lr: 4.564e-06, eta: 0:07:18, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.5435 2022-07-29 07:04:07,136 - easycv - INFO - Epoch [4][250/782] lr: 4.361e-06, eta: 0:07:02, time: 0.331, data_time: 0.008, memory: 7543, loss: 0.5269 2022-07-29 07:04:23,685 - easycv - INFO - Epoch [4][300/782] lr: 4.154e-06, eta: 0:06:47, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5276 2022-07-29 07:04:40,256 - easycv - INFO - Epoch [4][350/782] lr: 3.943e-06, eta: 0:06:31, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5447 2022-07-29 07:04:56,816 - easycv - INFO - Epoch [4][400/782] lr: 3.729e-06, eta: 0:06:15, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5268 2022-07-29 07:05:13,366 - easycv - INFO - Epoch [4][450/782] lr: 3.514e-06, eta: 0:05:59, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5448 2022-07-29 07:05:29,954 - easycv - INFO - Epoch [4][500/782] lr: 3.297e-06, eta: 0:05:43, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.5443 2022-07-29 07:05:46,529 - easycv - INFO - Epoch [4][550/782] lr: 3.080e-06, eta: 0:05:27, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5295 2022-07-29 07:06:03,082 - easycv - INFO - Epoch [4][600/782] lr: 2.865e-06, eta: 0:05:11, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5321 2022-07-29 07:06:19,638 - easycv - INFO - Epoch [4][650/782] lr: 2.651e-06, eta: 0:04:55, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5335 2022-07-29 07:06:36,215 - easycv - INFO - Epoch [4][700/782] lr: 2.440e-06, eta: 0:04:39, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.5321 2022-07-29 07:06:52,785 - easycv - INFO - Epoch [4][750/782] lr: 2.233e-06, eta: 0:04:23, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5322 [>>>>>>>>>>>>>>>>>>>>>>>>] 10000/10000, 528.3 task/s, elapsed: 19s, ETA: 0s 2022-07-29 07:07:22,162 - easycv - INFO - SaveBest metric_name: ['ClsEvaluator_neck_top1'] 2022-07-29 07:07:23,026 - easycv - INFO - End SaveBest metric 2022-07-29 07:07:23,027 - easycv - INFO - Epoch(val) [4][782] ClsEvaluator_neck_top1: 95.0700, neck_top1: 95.0700, neck_top5: 99.9400 2022-07-29 07:07:41,981 - easycv - INFO - Epoch [5][50/782] lr: 1.905e-06, eta: 0:03:54, time: 0.378, data_time: 0.056, memory: 7543, loss: 0.5289 2022-07-29 07:07:58,498 - easycv - INFO - Epoch [5][100/782] lr: 1.713e-06, eta: 0:03:39, time: 0.330, data_time: 0.008, memory: 7543, loss: 0.5190 2022-07-29 07:08:15,024 - easycv - INFO - Epoch [5][150/782] lr: 1.528e-06, eta: 0:03:23, time: 0.331, data_time: 0.008, memory: 7543, loss: 0.5357 2022-07-29 07:08:31,553 - easycv - INFO - Epoch [5][200/782] lr: 1.352e-06, eta: 0:03:07, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5239 2022-07-29 07:08:48,096 - easycv - INFO - Epoch [5][250/782] lr: 1.185e-06, eta: 0:02:51, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5078 2022-07-29 07:09:04,662 - easycv - INFO - Epoch [5][300/782] lr: 1.029e-06, eta: 0:02:35, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5215 2022-07-29 07:09:21,204 - easycv - INFO - Epoch [5][350/782] lr: 8.847e-07, eta: 0:02:19, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5220 2022-07-29 07:09:37,797 - easycv - INFO - Epoch [5][400/782] lr: 7.525e-07, eta: 0:02:03, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.5129 2022-07-29 07:09:54,371 - easycv - INFO - Epoch [5][450/782] lr: 6.336e-07, eta: 0:01:46, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5150 2022-07-29 07:10:10,955 - easycv - INFO - Epoch [5][500/782] lr: 5.291e-07, eta: 0:01:30, time: 0.332, data_time: 0.009, memory: 7543, loss: 0.5194 2022-07-29 07:10:27,518 - easycv - INFO - Epoch [5][550/782] lr: 4.399e-07, eta: 0:01:14, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5339 2022-07-29 07:10:44,072 - easycv - INFO - Epoch [5][600/782] lr: 3.667e-07, eta: 0:00:58, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5248 2022-07-29 07:11:00,640 - easycv - INFO - Epoch [5][650/782] lr: 3.107e-07, eta: 0:00:42, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5257 2022-07-29 07:11:17,204 - easycv - INFO - Epoch [5][700/782] lr: 2.725e-07, eta: 0:00:26, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5295 2022-07-29 07:11:33,772 - easycv - INFO - Epoch [5][750/782] lr: 2.530e-07, eta: 0:00:10, time: 0.331, data_time: 0.009, memory: 7543, loss: 0.5255 2022-07-29 07:11:44,196 - easycv - INFO - Saving checkpoint at 5 epochs [>>>>>>>>>>>>>>>>>>>>>>>>] 10000/10000, 528.2 task/s, elapsed: 19s, ETA: 0s 2022-07-29 07:12:03,717 - easycv - INFO - SaveBest metric_name: ['ClsEvaluator_neck_top1'] 2022-07-29 07:12:04,532 - easycv - INFO - End SaveBest metric 2022-07-29 07:12:04,533 - easycv - INFO - Epoch(val) [5][782] ClsEvaluator_neck_top1: 95.3400, neck_top1: 95.3400, neck_top5: 99.9400
导出模型
模型训练完成，使用export命令导出模型进行推理，导出的模型包含推理时所需的预处理信息、后处理信息
# 查看训练产生的pt文件 ! ls work_dirs/classification/cifar10/swin_tiny*
20220729_064804.log ClsEvaluator_neck_top1_best.pth 20220729_064804.log.json epoch_5.pth
ClsEvaluator_neck_top1_best.pth 是训练过程中产生的acc最高的pth,导出该模型
! python -m easycv.tools.export swintiny_b64_5e_jpg.py work_dirs/classification/cifar10/swin_tiny/ClsEvaluator_neck_top1_best.pth work_dirs/classification/cifar10/swin_tiny/best_export.pth
[2022-07-29 07:12:35,065.065 dsw-142697-674d9f9968-cnbqm:6548 INFO utils.py:30] NOTICE: PAIDEBUGGER is turned off. swintiny_b64_5e_jpg.py Read base config from /home/pai/lib/python3.6/site-packages/easycv/configs/base.py 2022-07-29 07:12:43,302 - easycv - INFO - load model from default path: https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/classification/timm/swint/swin_tiny_patch4_window7_224.pth load checkpoint from local path: work_dirs/classification/cifar10/swin_tiny/ClsEvaluator_neck_top1_best.pth
预测
下载测试图片
! wget http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/cifar10/qince_data/predict/aeroplane_s_000004.png
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file. ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled. --2022-07-29 07:12:49-- http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/cifar10/qince_data/predict/aeroplane_s_000004.png Resolving pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com... 39.98.20.13 Connecting to pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com|39.98.20.13|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2391 (2.3K) [image/png] Saving to: ‘aeroplane_s_000004.png’ aeroplane_s_000004. 100%[===================>] 2.33K --.-KB/s in 0s 2022-07-29 07:12:49 (426 MB/s) - ‘aeroplane_s_000004.png’ saved [2391/2391]
import cv2 from easycv.predictors.classifier import TorchClassifier output_ckpt = 'work_dirs/classification/cifar10/swin_tiny/best_export.pth' tcls = TorchClassifier(output_ckpt, topk=1) img = cv2.imread('aeroplane_s_000004.png') # input image should be RGB order img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) output = tcls.predict([img]) print(output)
