@toc)
参考论文：ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices作者：Xiangyu Zhang ，Xinyu Zhou，Mengxiao Lin，Jian Sun
1、论文摘要
我们引入了一个高效计算的CNN结构名字叫做shuffleNet，这个结构被设计用来解决部署算力非常有限的移动设备问题，这个新的结构使用了两个新的操作,pointwise group convolution 和 channel shuffle能够在极大减少计算量的同时保持一定的精度。我们在ImageNet classification和MS COCO目标检测数据集上做实验论证了ShuffleNet和其他的结构相比有着很好的性能。比如,相比于mobilenet，shufflenet在ImageNet 分类任务上有着更低的top-1错误率(错误率是7.8%)需要的计算量为40MFLOPs。在一个ARM-based移动设备,ShuffleNet相比于AlexNet实现了保持一定的精度的同时，实现了13A的速度。
2 Group Convolution（分组卷积）
第一个创新点就是分组1*1卷积
简单来说分组卷积就是将特征图分为不同的组，再对每组特征图分别进行卷积。这里的分组一般都是分为 n 个等份，理论上其实不是等份也可以，不 过一般为了实现方便都是分为等份。分组卷积的好处主要是可以减少模型的计算量和训练参 数，同时对模型准确率影响不大，甚至有可能会提高模型准确率。
在分组卷积中，每个卷积核只处理部分通道，比如上图中，红色卷积核只处理红色的通道，绿色卷积核只处理绿色通道，黄色卷积核只处理黄色通道。此时每个卷积核有2个通道，每个卷积核生成一张特征图。
下面我们通过几个图来详细了解一下，下图为普通卷积
图片来源：《深度学习从0到1》
图中的 Conv 表示卷积。
这里特征图的大小和卷积和的大小都不是重点内容，所以图中没有标出，我们只要能看 出 6 个特征图卷积后得到 12 个特征图就可以了。不过为了让大家理解分组卷积的计算量和 权值数量这里我们举例计算一下，假设特征图大小是 28×28，卷积核大小为 5×5，Same Padding。卷积层权值数量为 5×5×6×12+12=1812，乘法计算量为 5×5×28×28×6× 12=1411200。
下面我们看一下分组卷积，分组卷积一般都是把特征图分为 n 个等份，然后再对 n 个等 份的特征图分别卷积，这里的 n 可以人为设置，如图所示。
图中的 Conv 表示卷积。
为了跟普通卷积对比，所以这里分组卷积的例子输入也是 6 个特征图，输出也是 12 个特 征图。这里我们可以看到把 6 个特征图分为了 3 组，每组 2 个特征图，每组分别进行卷积， 卷积后得到 4 个特征图。最后再把 3 个组共 12 个特征图组合起来。假设特征图大小是 28× 28，卷积核大小为 5×5，Same Padding。这里卷积层权值数量为 5×5×2×4× 3+12=612，乘法计算量为 5×5×28×28×2×4×3=470400。权值数量和计算量都约为普通 卷积的 1/3。
分组卷积过程也可以描述如下：
- 假设输入的形状为 $H × W × C$，用 k 个 $h\times w$ 的卷积核对其进行卷积操作；
- 把输入分为g组，每组形状为$H × W × (C/g)$(假设可以整除)
- 把卷积核也分为g组，每组为k/g(假设可整除)个$h\times w$卷积核
- 按顺序，每组的输入和该组内的卷积核分别做标准卷积操作，输出 g 组形状为$H'× W' × (k/g)$;
将这 g 组特征合并起来,得到最终形状为$H'× W' × k$的特征;
下图为分两组时的举例：
图片地址
或者看下图
左边标准卷积，每个卷积核处理12个通道右边分组卷积，假设输入的12个通道分为3组，每个卷积核只处理4个通道
3 Channel Shuffle(通道重排)
图1（a）说明了两个堆叠组卷积层的情况。很明显，某个组的输出只与组内的输入有关。此属性会阻塞通道组之间的信息流并削弱表示。
如果我们允许组卷积从不同组中获取输入数据（如图 1（b）所示），输入和输出通道将完全相关。
对于上一层生成的特征图，我们可以先将每组中的通道划分为几个子组，然后将不同的子组馈入下一层中的每个组。这可以通过通道混洗操作有效而优雅地实现（图 1（c））：假设一个卷积层具有 g 个组，其输出有 g × n 个通道；我们首先将输出通道维度重塑为 (g, n)，转置然后将其展平作为下一层的输入。请注意，即使两个卷积的组数不同，该操作仍然有效。此外，channel shuffle 也是可微分的，这意味着它可以嵌入到网络结构中进行端到端训练。
举个例子来说，如下图。分组卷积生成的三组特征图，第一组1~4；第二组5~8；第三组9~12。先将特征图重塑，为三行N列的矩形。然后进行转置，变成N行三列。最后压平，从二维tensor变成一维tensor，每一组的特征图交叉组合在一起。实现各组之间的信息交融。
4、ShuffleNet Unit
图 2. ShuffleNet 单元。a) 具有深度卷积 (DWConv) [3, 12] 的残差单元 [9]；
b) 具有逐点组卷积 (GConv) 和通道混洗的 ShuffleNet 单元；
c) ShuffleNet 单元，stride = 2.
这里需要注意(b)中第一个分组卷积降维，然后通道重排，在进行3*3的DepthWise卷积，最后使用1*1的分组卷积升维，目的还是让残差前后的shape一致，保证可以进行Add操作。
(c)图是ShuffleNet的下采样模块。注意，这里左右分支不是使用Add操作，因为最后左右分支的shape是不一致的。==右分支的通道数和左分支的通道数叠加 == 输出特征图的通道数out_channel(重点，和上面的残差是不一样的)==，残差边上使用了池化窗口为3*3，stride=2的平均池化==Add是逐元素求个，Concat是某个方向的叠加，这里是再通道方向叠加。==
5、ShuffleNet网络结构设计
上图为不同分组数的ShuffleNet网络结构，通常我们将g=3的那一个网络作为baseline Network
我们在表 1 中展示了整体 ShuffleNet 架构。所提出的网络主要由一组 ShuffleNet 单元组成，分为三个阶段。每个阶段的第一个构建块应用 stride = 2。一个阶段内的其他超参数保持不变，下一个阶段的输出通道加倍。与 [9] 类似，我们将每个 ShuffleNet 的瓶颈通道数设置为输出通道的 1/4
上表中stage2的第一个block上不用GConv,用普通的1*1卷积，因为此时输入通道数只有24，太少了。且每个stage中的第一个block的stride=2（对应ShuffleNet Unit中的下采样模块,c图），其他block的stride=1(对应ShuffleNet基本模块，图b)
6、代码复现
这里只是简单复现下，细节问题并没有管，比如那个stage2的第一个block上不用GConv,用普通的1*1卷积就没管。
import tensorflow as tf
from tensorflow.keras.layers import concatenate, Conv2D, Activation, BatchNormalization, DepthwiseConv2D
from tensorflow.keras.layers import add, AvgPool2D,MaxPool2D,GlobalAveragePooling2D,Dense
from tensorflow.keras.models import Model
from plot_model import plot_model6.1 Channel Shuffle模块
# 通道重排,跨组信息交互
def channel_shuffle(inputs, num_groups):
# 先得到输入特征图的shape，b:batch size，h,w:一张图的size，c:通道数
b, h, w, c = inputs.shape
# 确定shape = [b, h, w, num_groups, c//num_groups]。通道维度原来是一个长为c的一维tensor，变成num_groups行n列的矩阵
# 在通道维度上将特征图reshape为num_groups行n列的矩阵
x_reshaped = tf.reshape(inputs, [-1, h, w, num_groups, c // num_groups])
# 确定转置的矩形的shape = [b, h, w, c//num_groups, num_groups]
# 矩阵转置，最后两个维度从num_groups行n列变成n行num_groups列
x_transposed = tf.transpose(x_reshaped, [0, 1, 2, 4, 3])
# 重新排列，shotcut和x的通道像素交叉排列，通道维度重新变成一维tensor
output = tf.reshape(x_transposed, [-1, h, w, c])
# 返回通道维度交叉排序后的tensor
return output6.2 分组卷积模块
我这里tensorflow版本为2.0，我看官网API中高版本的Conv2D参数中已经有了groups属性了。
def group_conv(inputs, filters, kernel, strides, num_groups):
conv_side_layers_tmp = tf.split(inputs, num_groups, axis=3)
conv_side_layers = []
for layer in conv_side_layers_tmp:
conv_side_layers.append(tf.keras.layers.Conv2D(filters // num_groups, kernel, strides, padding='same')(layer))
x = concatenate(conv_side_layers, axis=-1)
return x6.3 普通卷积模块
# 普通卷积：卷积+批标准化+ReLU激活
def conv(inputs, filters, kernel_size, stride=1):
x = Conv2D(filters, kernel_size, stride, padding='same', use_bias=False)(inputs)
x = BatchNormalization()(x)
x = Activation('relu')(x)
return x6.4 DepthWise卷积
# DWConv:深度可分离卷积块(论文中DWConv卷积核全是3*3,步长有1和2两种)
def depthwise_conv_bn(inputs, kernel_size, stride=1):
x = DepthwiseConv2D(kernel_size=kernel_size,
strides=stride,
padding='same',
use_bias=False)(inputs)
x = BatchNormalization()(x)
return x注意，这里没有用ReLU
6.5 ShuffleNetV1基本模块
# ShuffleNetV1基本模块(Add)
def shuffleNetUnitA(inputs, num_groups):
in_channels = inputs.shape[-1]
out_channels = in_channels
bottleneck_channels = out_channels // 4
# 1*1分组卷积降维
x = group_conv(inputs, bottleneck_channels, kernel=1, strides=1, num_groups=num_groups)
x = BatchNormalization()(x)
x = Activation('relu')(x)
# Channel Shuffle
x = channel_shuffle(x, num_groups)
# 3*3 DWConv
x = depthwise_conv_bn(x, kernel_size=3, stride=1)
# 1*1分组卷积升维(要保证残差连接前后的shape一致)
x = group_conv(x, out_channels, kernel=1, strides=1, num_groups=num_groups)
x = BatchNormalization()(x)
x = add([inputs, x])
x = Activation('relu')(x)
return x6.6 ShuffleNetV1下采样模块
# ShuffleNetV1下采样模块(下采样模块，concat)
def shuffleNetUnitB(inputs, out_channels, num_groups):
in_channels = inputs.shape[-1]
# 右分支的通道数和左分支的通道数叠加 == 输出特征图的通道数out_channel(重点，和上面的残差是不一样的)
out_channels -= in_channels
bottleneck_channels = out_channels // 4
# （1）右分支
# 1*1 GConv
x = group_conv(inputs, bottleneck_channels, kernel=1, strides=1, num_groups=num_groups)
x = BatchNormalization()(x)
x = Activation('relu')(x)
# Channel Shuffle
x = channel_shuffle(x, num_groups)
# 3*3 DWConv,stide=2
x = depthwise_conv_bn(x, kernel_size=3, stride=2)
# 1*1 GConv
x = group_conv(x, out_channels, kernel=1, strides=1, num_groups=num_groups)
x = BatchNormalization()(x)
# (2)左分支:3*3 AVG Pool,stride=2
y = AvgPool2D(pool_size=3, strides=2, padding='same')(inputs)
# 在通道维度上堆叠
x = concatenate([y, x], axis=-1)
x = Activation('relu')(x)
return x6.5和6.6的通道数需要结合论文好好看，要不你看不懂为什么会这样设计，原论文中都有解释。
6.6 stage
每个stage中的第一个block的stride=2(即下采样模块)，其他block的stride=1(即基本模块)
def stage(inputs, out_channels, num_groups, n):
# 每个stage中的第一个block的stride=2(即下采样模块)，其他block的stride=1(即基本模块)
# 都是按照论文搭建的，要去看论文原文，要不你绝对不理解为什么这样搭建，嘿嘿。
x = shuffleNetUnitB(inputs, out_channels, num_groups)
for _ in range(n):
x = shuffleNetUnitA(x, num_groups)
return x6.7 网络搭建
# first_stage_channels为第一个stage的输出通道数
# num_groups为分组数量
def ShuffleNet(inputs, first_stage_channels, num_groups,num_classes):
# 构建网络输入tensor
inputs = tf.keras.Input(shape=inputs)
# 论文中先用了一个普通卷积和池化
x = Conv2D(filters=24,
kernel_size=3,
strides=2,
padding='same')(inputs)
x = MaxPool2D(pool_size=3, strides=2, padding='same')(x)
# 三个stage,每个stage的第一个block的stride=2
# 同一个stage内的其他超参数不变，下一个stage的输出通道数加倍(这个可以通过论文中的表格看出，原文也给了)
# n为分组卷积的分组数量，论文中用g表示
x = stage(x, first_stage_channels, num_groups, n=3)
x = stage(x, first_stage_channels * 2, num_groups, n=7)
x = stage(x, first_stage_channels * 4, num_groups, n=3)
x = GlobalAveragePooling2D()(x)
# 我看过其他大佬的文章说compile的时候再用softmax，那样更稳定，有时间再试试吧
x = Dense(num_classes, activation='softmax')(x)
# 完整网络架构
model = Model(inputs=inputs, outputs=x)
return model6.8 自定义数据集测试
超参数
# 类别数
num_classes = 17
# 批次大小
batch_size = 32
# 周期数
epochs = 100
# 图片大小
image_size = 224查看模型摘要
model=ShuffleNet(inputs[224,224,3],first_stage_channels=240,num_groups=3,num_classes=17)
model.summary()Model: "functional_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 224, 224, 3) 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, 112, 112, 24) 672 input_1[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 56, 56, 24) 0 conv2d[0][0]
__________________________________________________________________________________________________
tf_op_layer_split (TensorFlowOp [(None, 56, 56, 8), 0 max_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][1]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][2]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 56, 56, 54) 0 conv2d_1[0][0]
conv2d_2[0][0]
conv2d_3[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 56, 56, 54) 216 concatenate[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, 56, 56, 54) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape (TensorFlow [(None, 56, 56, 3, 1 0 activation[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose (TensorFl [(None, 56, 56, 18, 0 tf_op_layer_Reshape[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_1 (TensorFl [(None, 56, 56, 54)] 0 tf_op_layer_Transpose[0][0]
__________________________________________________________________________________________________
depthwise_conv2d (DepthwiseConv (None, 28, 28, 54) 486 tf_op_layer_Reshape_1[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 28, 28, 54) 216 depthwise_conv2d[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_1 (TensorFlow [(None, 28, 28, 18), 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][1]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][2]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 28, 28, 216) 0 conv2d_4[0][0]
conv2d_5[0][0]
conv2d_6[0][0]
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, 28, 28, 24) 0 max_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 28, 28, 216) 864 concatenate_1[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 28, 28, 240) 0 average_pooling2d[0][0]
batch_normalization_2[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, 28, 28, 240) 0 concatenate_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_2 (TensorFlow [(None, 28, 28, 80), 0 activation_1[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][1]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][2]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 28, 28, 60) 0 conv2d_7[0][0]
conv2d_8[0][0]
conv2d_9[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 28, 28, 60) 240 concatenate_3[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, 28, 28, 60) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_2 (TensorFl [(None, 28, 28, 3, 2 0 activation_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_1 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_3 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_1[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_1 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 28, 28, 60) 240 depthwise_conv2d_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_3 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][1]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][2]
__________________________________________________________________________________________________
concatenate_4 (Concatenate) (None, 28, 28, 240) 0 conv2d_10[0][0]
conv2d_11[0][0]
conv2d_12[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 28, 28, 240) 960 concatenate_4[0][0]
__________________________________________________________________________________________________
add (Add) (None, 28, 28, 240) 0 activation_1[0][0]
batch_normalization_5[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, 28, 28, 240) 0 add[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_4 (TensorFlow [(None, 28, 28, 80), 0 activation_3[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][1]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][2]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 28, 28, 60) 0 conv2d_13[0][0]
conv2d_14[0][0]
conv2d_15[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 28, 28, 60) 240 concatenate_5[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 28, 28, 60) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_4 (TensorFl [(None, 28, 28, 3, 2 0 activation_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_2 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_5 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_2[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_5[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 28, 28, 60) 240 depthwise_conv2d_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_5 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][1]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][2]
__________________________________________________________________________________________________
concatenate_6 (Concatenate) (None, 28, 28, 240) 0 conv2d_16[0][0]
conv2d_17[0][0]
conv2d_18[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 28, 28, 240) 960 concatenate_6[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 28, 28, 240) 0 activation_3[0][0]
batch_normalization_8[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 28, 28, 240) 0 add_1[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_6 (TensorFlow [(None, 28, 28, 80), 0 activation_5[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][1]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][2]
__________________________________________________________________________________________________
concatenate_7 (Concatenate) (None, 28, 28, 60) 0 conv2d_19[0][0]
conv2d_20[0][0]
conv2d_21[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 28, 28, 60) 240 concatenate_7[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, 28, 28, 60) 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_6 (TensorFl [(None, 28, 28, 3, 2 0 activation_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_3 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_7 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_3[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_3 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_7[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 28, 28, 60) 240 depthwise_conv2d_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_7 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][1]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][2]
__________________________________________________________________________________________________
concatenate_8 (Concatenate) (None, 28, 28, 240) 0 conv2d_22[0][0]
conv2d_23[0][0]
conv2d_24[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 28, 28, 240) 960 concatenate_8[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 28, 28, 240) 0 activation_5[0][0]
batch_normalization_11[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, 28, 28, 240) 0 add_2[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_8 (TensorFlow [(None, 28, 28, 80), 0 activation_7[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][1]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][2]
__________________________________________________________________________________________________
concatenate_9 (Concatenate) (None, 28, 28, 60) 0 conv2d_25[0][0]
conv2d_26[0][0]
conv2d_27[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 28, 28, 60) 240 concatenate_9[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, 28, 28, 60) 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_8 (TensorFl [(None, 28, 28, 3, 2 0 activation_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_4 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_9 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_4[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_4 (DepthwiseCo (None, 14, 14, 60) 540 tf_op_layer_Reshape_9[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 14, 14, 60) 240 depthwise_conv2d_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_9 (TensorFlow [(None, 14, 14, 20), 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][1]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][2]
__________________________________________________________________________________________________
concatenate_10 (Concatenate) (None, 14, 14, 240) 0 conv2d_28[0][0]
conv2d_29[0][0]
conv2d_30[0][0]
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 14, 14, 240) 0 activation_7[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 14, 14, 240) 960 concatenate_10[0][0]
__________________________________________________________________________________________________
concatenate_11 (Concatenate) (None, 14, 14, 480) 0 average_pooling2d_1[0][0]
batch_normalization_14[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 14, 14, 480) 0 concatenate_11[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_10 (TensorFlo [(None, 14, 14, 160) 0 activation_9[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][1]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][2]
__________________________________________________________________________________________________
concatenate_12 (Concatenate) (None, 14, 14, 120) 0 conv2d_31[0][0]
conv2d_32[0][0]
conv2d_33[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 14, 14, 120) 480 concatenate_12[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, 14, 14, 120) 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_10 (TensorF [(None, 14, 14, 3, 4 0 activation_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_5 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_11 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_5[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_5 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_11[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_11 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][1]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][2]
__________________________________________________________________________________________________
concatenate_13 (Concatenate) (None, 14, 14, 480) 0 conv2d_34[0][0]
conv2d_35[0][0]
conv2d_36[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 14, 14, 480) 1920 concatenate_13[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 14, 14, 480) 0 activation_9[0][0]
batch_normalization_17[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, 14, 14, 480) 0 add_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_12 (TensorFlo [(None, 14, 14, 160) 0 activation_11[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][1]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][2]
__________________________________________________________________________________________________
concatenate_14 (Concatenate) (None, 14, 14, 120) 0 conv2d_37[0][0]
conv2d_38[0][0]
conv2d_39[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 14, 14, 120) 480 concatenate_14[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, 14, 14, 120) 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_12 (TensorF [(None, 14, 14, 3, 4 0 activation_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_6 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_13 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_6[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_6 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_13[0][0]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_13 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][1]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][2]
__________________________________________________________________________________________________
concatenate_15 (Concatenate) (None, 14, 14, 480) 0 conv2d_40[0][0]
conv2d_41[0][0]
conv2d_42[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 14, 14, 480) 1920 concatenate_15[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 14, 14, 480) 0 activation_11[0][0]
batch_normalization_20[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, 14, 14, 480) 0 add_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_14 (TensorFlo [(None, 14, 14, 160) 0 activation_13[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][1]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][2]
__________________________________________________________________________________________________
concatenate_16 (Concatenate) (None, 14, 14, 120) 0 conv2d_43[0][0]
conv2d_44[0][0]
conv2d_45[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 14, 14, 120) 480 concatenate_16[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, 14, 14, 120) 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_14 (TensorF [(None, 14, 14, 3, 4 0 activation_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_7 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_15 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_7[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_7 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_15[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_15 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][1]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][2]
__________________________________________________________________________________________________
concatenate_17 (Concatenate) (None, 14, 14, 480) 0 conv2d_46[0][0]
conv2d_47[0][0]
conv2d_48[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 14, 14, 480) 1920 concatenate_17[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 14, 14, 480) 0 activation_13[0][0]
batch_normalization_23[0][0]
__________________________________________________________________________________________________
activation_15 (Activation) (None, 14, 14, 480) 0 add_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_16 (TensorFlo [(None, 14, 14, 160) 0 activation_15[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][1]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][2]
__________________________________________________________________________________________________
concatenate_18 (Concatenate) (None, 14, 14, 120) 0 conv2d_49[0][0]
conv2d_50[0][0]
conv2d_51[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 14, 14, 120) 480 concatenate_18[0][0]
__________________________________________________________________________________________________
activation_16 (Activation) (None, 14, 14, 120) 0 batch_normalization_24[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_16 (TensorF [(None, 14, 14, 3, 4 0 activation_16[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_8 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_16[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_17 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_8[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_8 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_17[0][0]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_17 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][1]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][2]
__________________________________________________________________________________________________
concatenate_19 (Concatenate) (None, 14, 14, 480) 0 conv2d_52[0][0]
conv2d_53[0][0]
conv2d_54[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 14, 14, 480) 1920 concatenate_19[0][0]
__________________________________________________________________________________________________
add_6 (Add) (None, 14, 14, 480) 0 activation_15[0][0]
batch_normalization_26[0][0]
__________________________________________________________________________________________________
activation_17 (Activation) (None, 14, 14, 480) 0 add_6[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_18 (TensorFlo [(None, 14, 14, 160) 0 activation_17[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][1]
__________________________________________________________________________________________________
conv2d_57 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][2]
__________________________________________________________________________________________________
concatenate_20 (Concatenate) (None, 14, 14, 120) 0 conv2d_55[0][0]
conv2d_56[0][0]
conv2d_57[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 14, 14, 120) 480 concatenate_20[0][0]
__________________________________________________________________________________________________
activation_18 (Activation) (None, 14, 14, 120) 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_18 (TensorF [(None, 14, 14, 3, 4 0 activation_18[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_9 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_18[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_19 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_9[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_9 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_19[0][0]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_9[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_19 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
conv2d_58 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][0]
__________________________________________________________________________________________________
conv2d_59 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][1]
__________________________________________________________________________________________________
conv2d_60 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][2]
__________________________________________________________________________________________________
concatenate_21 (Concatenate) (None, 14, 14, 480) 0 conv2d_58[0][0]
conv2d_59[0][0]
conv2d_60[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 14, 14, 480) 1920 concatenate_21[0][0]
__________________________________________________________________________________________________
add_7 (Add) (None, 14, 14, 480) 0 activation_17[0][0]
batch_normalization_29[0][0]
__________________________________________________________________________________________________
activation_19 (Activation) (None, 14, 14, 480) 0 add_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_20 (TensorFlo [(None, 14, 14, 160) 0 activation_19[0][0]
__________________________________________________________________________________________________
conv2d_61 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][1]
__________________________________________________________________________________________________
conv2d_63 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][2]
__________________________________________________________________________________________________
concatenate_22 (Concatenate) (None, 14, 14, 120) 0 conv2d_61[0][0]
conv2d_62[0][0]
conv2d_63[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 14, 14, 120) 480 concatenate_22[0][0]
__________________________________________________________________________________________________
activation_20 (Activation) (None, 14, 14, 120) 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_20 (TensorF [(None, 14, 14, 3, 4 0 activation_20[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_10 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_20[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_21 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_10[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_10 (DepthwiseC (None, 14, 14, 120) 1080 tf_op_layer_Reshape_21[0][0]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_21 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
conv2d_64 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][0]
__________________________________________________________________________________________________
conv2d_65 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][1]
__________________________________________________________________________________________________
conv2d_66 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][2]
__________________________________________________________________________________________________
concatenate_23 (Concatenate) (None, 14, 14, 480) 0 conv2d_64[0][0]
conv2d_65[0][0]
conv2d_66[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 480) 1920 concatenate_23[0][0]
__________________________________________________________________________________________________
add_8 (Add) (None, 14, 14, 480) 0 activation_19[0][0]
batch_normalization_32[0][0]
__________________________________________________________________________________________________
activation_21 (Activation) (None, 14, 14, 480) 0 add_8[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_22 (TensorFlo [(None, 14, 14, 160) 0 activation_21[0][0]
__________________________________________________________________________________________________
conv2d_67 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][0]
__________________________________________________________________________________________________
conv2d_68 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][1]
__________________________________________________________________________________________________
conv2d_69 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][2]
__________________________________________________________________________________________________
concatenate_24 (Concatenate) (None, 14, 14, 120) 0 conv2d_67[0][0]
conv2d_68[0][0]
conv2d_69[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 120) 480 concatenate_24[0][0]
__________________________________________________________________________________________________
activation_22 (Activation) (None, 14, 14, 120) 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_22 (TensorF [(None, 14, 14, 3, 4 0 activation_22[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_11 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_22[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_23 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_11[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_11 (DepthwiseC (None, 14, 14, 120) 1080 tf_op_layer_Reshape_23[0][0]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_11[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_23 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
conv2d_70 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][0]
__________________________________________________________________________________________________
conv2d_71 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][1]
__________________________________________________________________________________________________
conv2d_72 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][2]
__________________________________________________________________________________________________
concatenate_25 (Concatenate) (None, 14, 14, 480) 0 conv2d_70[0][0]
conv2d_71[0][0]
conv2d_72[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 480) 1920 concatenate_25[0][0]
__________________________________________________________________________________________________
add_9 (Add) (None, 14, 14, 480) 0 activation_21[0][0]
batch_normalization_35[0][0]
__________________________________________________________________________________________________
activation_23 (Activation) (None, 14, 14, 480) 0 add_9[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_24 (TensorFlo [(None, 14, 14, 160) 0 activation_23[0][0]
__________________________________________________________________________________________________
conv2d_73 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][0]
__________________________________________________________________________________________________
conv2d_74 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][1]
__________________________________________________________________________________________________
conv2d_75 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][2]
__________________________________________________________________________________________________
concatenate_26 (Concatenate) (None, 14, 14, 120) 0 conv2d_73[0][0]
conv2d_74[0][0]
conv2d_75[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 120) 480 concatenate_26[0][0]
__________________________________________________________________________________________________
activation_24 (Activation) (None, 14, 14, 120) 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_24 (TensorF [(None, 14, 14, 3, 4 0 activation_24[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_12 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_24[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_25 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_12[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_12 (DepthwiseC (None, 7, 7, 120) 1080 tf_op_layer_Reshape_25[0][0]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 7, 7, 120) 480 depthwise_conv2d_12[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_25 (TensorFlo [(None, 7, 7, 40), ( 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
conv2d_76 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][0]
__________________________________________________________________________________________________
conv2d_77 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][1]
__________________________________________________________________________________________________
conv2d_78 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][2]
__________________________________________________________________________________________________
concatenate_27 (Concatenate) (None, 7, 7, 480) 0 conv2d_76[0][0]
conv2d_77[0][0]
conv2d_78[0][0]
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 7, 7, 480) 0 activation_23[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 7, 7, 480) 1920 concatenate_27[0][0]
__________________________________________________________________________________________________
concatenate_28 (Concatenate) (None, 7, 7, 960) 0 average_pooling2d_2[0][0]
batch_normalization_38[0][0]
__________________________________________________________________________________________________
activation_25 (Activation) (None, 7, 7, 960) 0 concatenate_28[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_26 (TensorFlo [(None, 7, 7, 320), 0 activation_25[0][0]
__________________________________________________________________________________________________
conv2d_79 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][0]
__________________________________________________________________________________________________
conv2d_80 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][1]
__________________________________________________________________________________________________
conv2d_81 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][2]
__________________________________________________________________________________________________
concatenate_29 (Concatenate) (None, 7, 7, 240) 0 conv2d_79[0][0]
conv2d_80[0][0]
conv2d_81[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 7, 7, 240) 960 concatenate_29[0][0]
__________________________________________________________________________________________________
activation_26 (Activation) (None, 7, 7, 240) 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_26 (TensorF [(None, 7, 7, 3, 80) 0 activation_26[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_13 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_26[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_27 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_13[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_13 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_27[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_13[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_27 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_40[0][0]
__________________________________________________________________________________________________
conv2d_82 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][0]
__________________________________________________________________________________________________
conv2d_83 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][1]
__________________________________________________________________________________________________
conv2d_84 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][2]
__________________________________________________________________________________________________
concatenate_30 (Concatenate) (None, 7, 7, 960) 0 conv2d_82[0][0]
conv2d_83[0][0]
conv2d_84[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 7, 7, 960) 3840 concatenate_30[0][0]
__________________________________________________________________________________________________
add_10 (Add) (None, 7, 7, 960) 0 activation_25[0][0]
batch_normalization_41[0][0]
__________________________________________________________________________________________________
activation_27 (Activation) (None, 7, 7, 960) 0 add_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_28 (TensorFlo [(None, 7, 7, 320), 0 activation_27[0][0]
__________________________________________________________________________________________________
conv2d_85 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][0]
__________________________________________________________________________________________________
conv2d_86 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][1]
__________________________________________________________________________________________________
conv2d_87 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][2]
__________________________________________________________________________________________________
concatenate_31 (Concatenate) (None, 7, 7, 240) 0 conv2d_85[0][0]
conv2d_86[0][0]
conv2d_87[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 7, 7, 240) 960 concatenate_31[0][0]
__________________________________________________________________________________________________
activation_28 (Activation) (None, 7, 7, 240) 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_28 (TensorF [(None, 7, 7, 3, 80) 0 activation_28[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_14 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_28[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_29 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_14[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_14 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_29[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_29 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_43[0][0]
__________________________________________________________________________________________________
conv2d_88 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][0]
__________________________________________________________________________________________________
conv2d_89 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][1]
__________________________________________________________________________________________________
conv2d_90 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][2]
__________________________________________________________________________________________________
concatenate_32 (Concatenate) (None, 7, 7, 960) 0 conv2d_88[0][0]
conv2d_89[0][0]
conv2d_90[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 7, 7, 960) 3840 concatenate_32[0][0]
__________________________________________________________________________________________________
add_11 (Add) (None, 7, 7, 960) 0 activation_27[0][0]
batch_normalization_44[0][0]
__________________________________________________________________________________________________
activation_29 (Activation) (None, 7, 7, 960) 0 add_11[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_30 (TensorFlo [(None, 7, 7, 320), 0 activation_29[0][0]
__________________________________________________________________________________________________
conv2d_91 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][0]
__________________________________________________________________________________________________
conv2d_92 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][1]
__________________________________________________________________________________________________
conv2d_93 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][2]
__________________________________________________________________________________________________
concatenate_33 (Concatenate) (None, 7, 7, 240) 0 conv2d_91[0][0]
conv2d_92[0][0]
conv2d_93[0][0]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 7, 7, 240) 960 concatenate_33[0][0]
__________________________________________________________________________________________________
activation_30 (Activation) (None, 7, 7, 240) 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_30 (TensorF [(None, 7, 7, 3, 80) 0 activation_30[0][0]
__________________________________________________________________________________________________
tf_op_layer_Transpose_15 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_30[0][0]
__________________________________________________________________________________________________
tf_op_layer_Reshape_31 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_15[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_15 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_31[0][0]
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_15[0][0]
__________________________________________________________________________________________________
tf_op_layer_split_31 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_46[0][0]
__________________________________________________________________________________________________
conv2d_94 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][0]
__________________________________________________________________________________________________
conv2d_95 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][1]
__________________________________________________________________________________________________
conv2d_96 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][2]
__________________________________________________________________________________________________
concatenate_34 (Concatenate) (None, 7, 7, 960) 0 conv2d_94[0][0]
conv2d_95[0][0]
conv2d_96[0][0]
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 7, 7, 960) 3840 concatenate_34[0][0]
__________________________________________________________________________________________________
add_12 (Add) (None, 7, 7, 960) 0 activation_29[0][0]
batch_normalization_47[0][0]
__________________________________________________________________________________________________
activation_31 (Activation) (None, 7, 7, 960) 0 add_12[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 960) 0 activation_31[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 17) 16337 global_average_pooling2d[0][0]
==================================================================================================
Total params: 902,741
Trainable params: 879,053
Non-trainable params: 23,688数据增强
# 训练集数据进行数据增强
train_datagen = ImageDataGenerator(
rotation_range=20, # 随机旋转度数
width_shift_range=0.1, # 随机水平平移
height_shift_range=0.1, # 随机竖直平移
rescale=1 / 255, # 数据归一化
shear_range=10, # 随机错切变换
zoom_range=0.1, # 随机放大
horizontal_flip=True, # 水平翻转
brightness_range=(0.7, 1.3), # 亮度变化
fill_mode='nearest', # 填充方式
)
# 测试集数据只需要归一化就可以
test_datagen = ImageDataGenerator(
rescale=1 / 255, # 数据归一化
)数据生成器
# 训练集数据生成器，可以在训练时自动产生数据进行训练
# 从'data/train'获得训练集数据
# 获得数据后会把图片resize为image_size×image_size的大小
# generator每次会产生batch_size个数据
train_generator = train_datagen.flow_from_directory(
'../data/train',
target_size=(image_size, image_size),
batch_size=batch_size,
)
# 测试集数据生成器
test_generator = test_datagen.flow_from_directory(
'../data/test',
target_size=(image_size, image_size),
batch_size=batch_size,
)
# 字典的键为17个文件夹的名字，值为对应的分类编号
print(train_generator.class_indices)
回调设置
# 学习率调节函数，逐渐减小学习率
def adjust_learning_rate(epoch):
# 前40周期
if epoch<=40:
lr = 1e-4
# 前40到80周期
elif epoch>40 and epoch<=80:
lr = 1e-5
# 80到100周期
else:
lr = 1e-6
return lr
# 定义优化器
adam = Adam(lr=1e-4)
# 读取模型
checkpoint_save_path = "./checkpoint/ShuffleNetV1.ckpt"
if os.path.exists(checkpoint_save_path + '.index'):
print('-------------load the model-----------------')
model.load_weights(checkpoint_save_path)
# 保存模型
cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
save_weights_only=True,
save_best_only=True)
# 定义学习率衰减策略
callbacks = []
callbacks.append(LearningRateScheduler(adjust_learning_rate))
callbacks.append(cp_callback)# 定义优化器，loss function，训练过程中计算准确率
model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy'])
# Tensorflow2.1版本(包括2.1)之后可以直接使用fit训练模型
history = model.fit(x=train_generator,epochs=epochs,validation_data=test_generator,callbacks=callbacks)Epoch 1/100
34/34 [==============================] - 18s 526ms/step - loss: 2.9157 - accuracy: 0.1544 - val_loss: 2.8353 - val_accuracy: 0.0588
Epoch 2/100
34/34 [==============================] - 16s 478ms/step - loss: 2.2678 - accuracy: 0.2822 - val_loss: 2.8387 - val_accuracy: 0.0588
Epoch 3/100
34/34 [==============================] - 16s 483ms/step - loss: 1.9987 - accuracy: 0.3778 - val_loss: 2.8482 - val_accuracy: 0.0588
Epoch 4/100
34/34 [==============================] - 16s 481ms/step - loss: 1.8269 - accuracy: 0.4200 - val_loss: 2.8607 - val_accuracy: 0.0625
Epoch 5/100
34/34 [==============================] - 17s 487ms/step - loss: 1.7022 - accuracy: 0.4651 - val_loss: 2.8800 - val_accuracy: 0.0588
Epoch 6/100
34/34 [==============================] - 16s 479ms/step - loss: 1.5436 - accuracy: 0.5138 - val_loss: 2.9101 - val_accuracy: 0.0588
Epoch 7/100
34/34 [==============================] - 16s 484ms/step - loss: 1.4806 - accuracy: 0.5221 - val_loss: 2.9455 - val_accuracy: 0.0588
Epoch 8/100
34/34 [==============================] - 17s 486ms/step - loss: 1.3658 - accuracy: 0.5699 - val_loss: 3.0113 - val_accuracy: 0.0588
Epoch 9/100
34/34 [==============================] - 16s 478ms/step - loss: 1.3244 - accuracy: 0.5772 - val_loss: 3.0792 - val_accuracy: 0.0588
Epoch 10/100
34/34 [==============================] - 16s 480ms/step - loss: 1.2401 - accuracy: 0.6029 - val_loss: 3.2077 - val_accuracy: 0.0588
Epoch 11/100
34/34 [==============================] - 16s 480ms/step - loss: 1.1940 - accuracy: 0.6048 - val_loss: 3.3061 - val_accuracy: 0.0588
Epoch 12/100
34/34 [==============================] - 16s 480ms/step - loss: 1.1689 - accuracy: 0.6213 - val_loss: 3.4508 - val_accuracy: 0.0772
Epoch 13/100
34/34 [==============================] - 16s 481ms/step - loss: 1.1282 - accuracy: 0.6241 - val_loss: 3.5842 - val_accuracy: 0.0919
Epoch 14/100
34/34 [==============================] - 16s 479ms/step - loss: 1.0759 - accuracy: 0.6572 - val_loss: 3.5697 - val_accuracy: 0.1140
Epoch 15/100
34/34 [==============================] - 17s 488ms/step - loss: 1.0264 - accuracy: 0.6618 - val_loss: 3.6231 - val_accuracy: 0.1176
Epoch 16/100
34/34 [==============================] - 16s 475ms/step - loss: 1.0210 - accuracy: 0.6719 - val_loss: 3.3065 - val_accuracy: 0.1471
Epoch 17/100
34/34 [==============================] - 16s 477ms/step - loss: 0.9543 - accuracy: 0.6939 - val_loss: 2.9929 - val_accuracy: 0.2169
Epoch 18/100
34/34 [==============================] - 17s 493ms/step - loss: 0.9405 - accuracy: 0.6939 - val_loss: 2.3133 - val_accuracy: 0.2941
Epoch 19/100
34/34 [==============================] - 17s 488ms/step - loss: 0.9069 - accuracy: 0.7031 - val_loss: 1.8180 - val_accuracy: 0.4081
Epoch 20/100
34/34 [==============================] - 17s 491ms/step - loss: 0.8797 - accuracy: 0.7050 - val_loss: 1.7820 - val_accuracy: 0.4926
Epoch 21/100
34/34 [==============================] - 17s 493ms/step - loss: 0.8911 - accuracy: 0.7215 - val_loss: 1.4993 - val_accuracy: 0.5625
Epoch 22/100
34/34 [==============================] - 17s 496ms/step - loss: 0.8480 - accuracy: 0.7215 - val_loss: 1.4438 - val_accuracy: 0.5699
Epoch 23/100
34/34 [==============================] - 17s 496ms/step - loss: 0.7861 - accuracy: 0.7472 - val_loss: 1.2822 - val_accuracy: 0.6103
Epoch 24/100
34/34 [==============================] - 16s 477ms/step - loss: 0.8111 - accuracy: 0.7307 - val_loss: 1.7404 - val_accuracy: 0.5404
Epoch 25/100
34/34 [==============================] - 16s 477ms/step - loss: 0.7855 - accuracy: 0.7316 - val_loss: 1.4184 - val_accuracy: 0.6287
Epoch 26/100
34/34 [==============================] - 17s 491ms/step - loss: 0.7679 - accuracy: 0.7555 - val_loss: 1.2750 - val_accuracy: 0.6213
Epoch 27/100
34/34 [==============================] - 16s 481ms/step - loss: 0.7451 - accuracy: 0.7454 - val_loss: 1.4109 - val_accuracy: 0.5882
Epoch 28/100
34/34 [==============================] - 16s 482ms/step - loss: 0.7201 - accuracy: 0.7408 - val_loss: 1.3238 - val_accuracy: 0.6507
Epoch 29/100
34/34 [==============================] - 16s 475ms/step - loss: 0.7238 - accuracy: 0.7629 - val_loss: 1.2997 - val_accuracy: 0.6360
Epoch 30/100
34/34 [==============================] - 16s 482ms/step - loss: 0.6587 - accuracy: 0.7849 - val_loss: 1.2941 - val_accuracy: 0.6471
Epoch 31/100
34/34 [==============================] - 17s 488ms/step - loss: 0.6508 - accuracy: 0.7858 - val_loss: 1.4063 - val_accuracy: 0.6397
Epoch 32/100
34/34 [==============================] - 17s 495ms/step - loss: 0.6294 - accuracy: 0.8006 - val_loss: 1.1997 - val_accuracy: 0.6691
Epoch 33/100
34/34 [==============================] - 16s 482ms/step - loss: 0.6579 - accuracy: 0.7665 - val_loss: 1.2146 - val_accuracy: 0.6434
Epoch 34/100
34/34 [==============================] - 16s 481ms/step - loss: 0.6207 - accuracy: 0.7941 - val_loss: 1.3010 - val_accuracy: 0.6360
Epoch 35/100
34/34 [==============================] - 16s 480ms/step - loss: 0.5873 - accuracy: 0.8033 - val_loss: 1.2448 - val_accuracy: 0.6618
Epoch 36/100
34/34 [==============================] - 16s 477ms/step - loss: 0.5755 - accuracy: 0.7941 - val_loss: 1.2584 - val_accuracy: 0.6434
Epoch 37/100
34/34 [==============================] - 17s 491ms/step - loss: 0.5668 - accuracy: 0.8189 - val_loss: 1.1914 - val_accuracy: 0.6728
Epoch 38/100
34/34 [==============================] - 16s 485ms/step - loss: 0.5527 - accuracy: 0.8107 - val_loss: 1.2917 - val_accuracy: 0.6434
Epoch 39/100
34/34 [==============================] - 16s 475ms/step - loss: 0.5715 - accuracy: 0.8088 - val_loss: 1.2613 - val_accuracy: 0.6581
Epoch 40/100
34/34 [==============================] - 16s 477ms/step - loss: 0.5286 - accuracy: 0.8272 - val_loss: 1.2632 - val_accuracy: 0.6581
Epoch 41/100
34/34 [==============================] - 17s 495ms/step - loss: 0.5098 - accuracy: 0.8327 - val_loss: 1.1116 - val_accuracy: 0.6985
Epoch 42/100
34/34 [==============================] - 17s 493ms/step - loss: 0.4543 - accuracy: 0.8428 - val_loss: 1.0116 - val_accuracy: 0.7206
Epoch 43/100
34/34 [==============================] - 17s 492ms/step - loss: 0.4145 - accuracy: 0.8695 - val_loss: 0.9978 - val_accuracy: 0.7206
Epoch 44/100
34/34 [==============================] - 17s 489ms/step - loss: 0.4421 - accuracy: 0.8612 - val_loss: 0.9956 - val_accuracy: 0.7169
Epoch 45/100
34/34 [==============================] - 17s 486ms/step - loss: 0.4015 - accuracy: 0.8750 - val_loss: 0.9964 - val_accuracy: 0.7206
Epoch 46/100
34/34 [==============================] - 16s 476ms/step - loss: 0.3937 - accuracy: 0.8805 - val_loss: 1.0131 - val_accuracy: 0.7279
Epoch 47/100
34/34 [==============================] - 16s 477ms/step - loss: 0.4492 - accuracy: 0.8539 - val_loss: 1.0172 - val_accuracy: 0.7243
Epoch 48/100
34/34 [==============================] - 16s 483ms/step - loss: 0.4238 - accuracy: 0.8649 - val_loss: 1.0143 - val_accuracy: 0.7279
Epoch 49/100
34/34 [==============================] - 16s 480ms/step - loss: 0.4192 - accuracy: 0.8658 - val_loss: 1.0148 - val_accuracy: 0.7206
Epoch 50/100
34/34 [==============================] - 16s 479ms/step - loss: 0.4071 - accuracy: 0.8695 - val_loss: 1.0148 - val_accuracy: 0.7316
Epoch 51/100
34/34 [==============================] - 16s 483ms/step - loss: 0.4115 - accuracy: 0.8557 - val_loss: 1.0180 - val_accuracy: 0.7463
Epoch 52/100
34/34 [==============================] - 16s 480ms/step - loss: 0.4106 - accuracy: 0.8704 - val_loss: 1.0214 - val_accuracy: 0.7353
Epoch 53/100
34/34 [==============================] - 16s 477ms/step - loss: 0.4026 - accuracy: 0.8713 - val_loss: 1.0171 - val_accuracy: 0.7390
Epoch 54/100
34/34 [==============================] - 16s 476ms/step - loss: 0.4073 - accuracy: 0.8750 - val_loss: 1.0067 - val_accuracy: 0.7426
Epoch 55/100
34/34 [==============================] - 16s 485ms/step - loss: 0.3608 - accuracy: 0.8842 - val_loss: 1.0039 - val_accuracy: 0.7500
Epoch 56/100
34/34 [==============================] - 16s 478ms/step - loss: 0.4101 - accuracy: 0.8658 - val_loss: 1.0056 - val_accuracy: 0.7426
Epoch 57/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3797 - accuracy: 0.8787 - val_loss: 0.9983 - val_accuracy: 0.7426
Epoch 58/100
34/34 [==============================] - 16s 484ms/step - loss: 0.4077 - accuracy: 0.8695 - val_loss: 1.0055 - val_accuracy: 0.7390
Epoch 59/100
34/34 [==============================] - 16s 474ms/step - loss: 0.4265 - accuracy: 0.8649 - val_loss: 1.0111 - val_accuracy: 0.7500
Epoch 60/100
34/34 [==============================] - 17s 492ms/step - loss: 0.3475 - accuracy: 0.8961 - val_loss: 0.9925 - val_accuracy: 0.7500
Epoch 61/100
34/34 [==============================] - 17s 497ms/step - loss: 0.3883 - accuracy: 0.8796 - val_loss: 0.9693 - val_accuracy: 0.7390
Epoch 62/100
34/34 [==============================] - 17s 494ms/step - loss: 0.3615 - accuracy: 0.8934 - val_loss: 0.9608 - val_accuracy: 0.7463
Epoch 63/100
34/34 [==============================] - 16s 480ms/step - loss: 0.4017 - accuracy: 0.8695 - val_loss: 0.9706 - val_accuracy: 0.7610
Epoch 64/100
34/34 [==============================] - 17s 495ms/step - loss: 0.3727 - accuracy: 0.8851 - val_loss: 0.9587 - val_accuracy: 0.7574
Epoch 65/100
34/34 [==============================] - 16s 480ms/step - loss: 0.4053 - accuracy: 0.8750 - val_loss: 0.9707 - val_accuracy: 0.7390
Epoch 66/100
34/34 [==============================] - 16s 481ms/step - loss: 0.3722 - accuracy: 0.8860 - val_loss: 0.9836 - val_accuracy: 0.7426
Epoch 67/100
34/34 [==============================] - 16s 477ms/step - loss: 0.3841 - accuracy: 0.8824 - val_loss: 0.9597 - val_accuracy: 0.7426
Epoch 68/100
34/34 [==============================] - 16s 480ms/step - loss: 0.4085 - accuracy: 0.8621 - val_loss: 0.9813 - val_accuracy: 0.7390
Epoch 69/100
34/34 [==============================] - 16s 479ms/step - loss: 0.3578 - accuracy: 0.8943 - val_loss: 0.9673 - val_accuracy: 0.7463
Epoch 70/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3628 - accuracy: 0.8915 - val_loss: 0.9789 - val_accuracy: 0.7353
Epoch 71/100
34/34 [==============================] - 17s 486ms/step - loss: 0.3904 - accuracy: 0.8686 - val_loss: 0.9738 - val_accuracy: 0.7537
Epoch 72/100
34/34 [==============================] - 16s 484ms/step - loss: 0.3585 - accuracy: 0.8943 - val_loss: 0.9827 - val_accuracy: 0.7574
Epoch 73/100
34/34 [==============================] - 16s 479ms/step - loss: 0.3593 - accuracy: 0.8879 - val_loss: 0.9722 - val_accuracy: 0.7574
Epoch 74/100
34/34 [==============================] - 16s 481ms/step - loss: 0.3636 - accuracy: 0.8869 - val_loss: 0.9719 - val_accuracy: 0.7500
Epoch 75/100
34/34 [==============================] - 16s 479ms/step - loss: 0.3776 - accuracy: 0.8787 - val_loss: 0.9690 - val_accuracy: 0.7500
Epoch 76/100
34/34 [==============================] - 17s 490ms/step - loss: 0.3815 - accuracy: 0.8768 - val_loss: 0.9569 - val_accuracy: 0.7574
Epoch 77/100
34/34 [==============================] - 16s 480ms/step - loss: 0.3608 - accuracy: 0.8833 - val_loss: 0.9737 - val_accuracy: 0.7537
Epoch 78/100
34/34 [==============================] - 17s 496ms/step - loss: 0.3801 - accuracy: 0.8722 - val_loss: 0.9393 - val_accuracy: 0.7610
Epoch 79/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3385 - accuracy: 0.8943 - val_loss: 0.9577 - val_accuracy: 0.7500
Epoch 80/100
34/34 [==============================] - 16s 477ms/step - loss: 0.3565 - accuracy: 0.8869 - val_loss: 0.9693 - val_accuracy: 0.7537
Epoch 81/100
34/34 [==============================] - 16s 482ms/step - loss: 0.3614 - accuracy: 0.8961 - val_loss: 0.9911 - val_accuracy: 0.7574
Epoch 82/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3581 - accuracy: 0.8814 - val_loss: 0.9848 - val_accuracy: 0.7574
Epoch 83/100
34/34 [==============================] - 16s 479ms/step - loss: 0.3880 - accuracy: 0.8833 - val_loss: 0.9779 - val_accuracy: 0.7574
Epoch 84/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3549 - accuracy: 0.8851 - val_loss: 0.9744 - val_accuracy: 0.7537
Epoch 85/100
34/34 [==============================] - 17s 487ms/step - loss: 0.3552 - accuracy: 0.8934 - val_loss: 0.9736 - val_accuracy: 0.7537
Epoch 86/100
34/34 [==============================] - 16s 483ms/step - loss: 0.3681 - accuracy: 0.8842 - val_loss: 0.9715 - val_accuracy: 0.7537
Epoch 87/100
34/34 [==============================] - 16s 479ms/step - loss: 0.3416 - accuracy: 0.8934 - val_loss: 0.9662 - val_accuracy: 0.7537
Epoch 88/100
34/34 [==============================] - 17s 486ms/step - loss: 0.3562 - accuracy: 0.8934 - val_loss: 0.9662 - val_accuracy: 0.7537
Epoch 89/100
34/34 [==============================] - 17s 485ms/step - loss: 0.3682 - accuracy: 0.8851 - val_loss: 0.9636 - val_accuracy: 0.7500
Epoch 90/100
34/34 [==============================] - 16s 480ms/step - loss: 0.3739 - accuracy: 0.8787 - val_loss: 0.9614 - val_accuracy: 0.7500
Epoch 91/100
34/34 [==============================] - 16s 483ms/step - loss: 0.3442 - accuracy: 0.8943 - val_loss: 0.9631 - val_accuracy: 0.7500
Epoch 92/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3598 - accuracy: 0.8915 - val_loss: 0.9624 - val_accuracy: 0.7500
Epoch 93/100
34/34 [==============================] - 16s 474ms/step - loss: 0.3845 - accuracy: 0.8778 - val_loss: 0.9641 - val_accuracy: 0.7500
Epoch 94/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3526 - accuracy: 0.8915 - val_loss: 0.9655 - val_accuracy: 0.7574
Epoch 95/100
34/34 [==============================] - 16s 477ms/step - loss: 0.3581 - accuracy: 0.8869 - val_loss: 0.9652 - val_accuracy: 0.7574
Epoch 96/100
34/34 [==============================] - 16s 477ms/step - loss: 0.3392 - accuracy: 0.8897 - val_loss: 0.9608 - val_accuracy: 0.7537
Epoch 97/100
34/34 [==============================] - 16s 478ms/step - loss: 0.3525 - accuracy: 0.8943 - val_loss: 0.9628 - val_accuracy: 0.7537
Epoch 98/100
34/34 [==============================] - 16s 480ms/step - loss: 0.3532 - accuracy: 0.8943 - val_loss: 0.9632 - val_accuracy: 0.7537
Epoch 99/100
34/34 [==============================] - 16s 479ms/step - loss: 0.3759 - accuracy: 0.8851 - val_loss: 0.9571 - val_accuracy: 0.7574
Epoch 100/100
34/34 [==============================] - 16s 476ms/step - loss: 0.3292 - accuracy: 0.8971 - val_loss: 0.9545 - val_accuracy: 0.7537没有预训练的权重，训练出来的结果肯定不是特别好，但是还算能接受吧。
可视化
# 画出训练集准确率曲线图
plt.plot(np.arange(epochs),history.history['accuracy'],c='b',label='train_accuracy')
# 画出验证集准确率曲线图
plt.plot(np.arange(epochs),history.history['val_accuracy'],c='y',label='val_accuracy')
# 图例
plt.legend()
# x坐标描述
plt.xlabel('epochs')
# y坐标描述
plt.ylabel('accuracy')
# 显示图像
plt.show()
# 画出训练集loss曲线图
plt.plot(np.arange(epochs),history.history['loss'],c='b',label='train_loss')
# 画出验证集loss曲线图
plt.plot(np.arange(epochs),history.history['val_loss'],c='y',label='val_loss')
# 图例
plt.legend()
# x坐标描述
plt.xlabel('epochs')
# y坐标描述
plt.ylabel('loss')
# 显示图像
plt.show()
模型结构大图
这里给出1000分类的模型结构图，图太大，翻起来不好翻，需要pdf的留下邮箱我发给你。
References
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, & Jian Sun (2018). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices computer vision and pattern recognition.
8.3 使用Tensorflow2搭建ShuffleNetv2
