@toc
参考论文：Squeeze-and-Excitation Networks作者：Jie Hu，Li Shen，Samuel Albanie，Gang Sun，Enhua Wu
论文中给出的源码链接：https://github.com/hujie-frank/SENet
1、SeNet简介
SENet 是 ImageNet Challenge 图像识别比赛 2017 年的冠军，是来自 Momenta 公司 的团队完成。他们提出了 Squeeze-and-Excitation Networks（简称 SENet）。SENet 不是独立的模型设计，只对模型的一种优化。一般 SENet 都会结合其它模型一起使用，比如 SENet 用于 ResNet-50 中我们就把这个模型称为 SE-ResNet-50，比如 SENet 用于 Inception-ResNet-v2 中我们就把这个模型称为 SE- Inception-ResNet-v2。最早提出 SENet 的论文是《Squeeze-and-Excitation Networks》。
SENet 的模型优化思路很有意思，主要是针对特征的 channel 进行优化。
我们可以想象在进行图像识别的时候，卷积计算后生成了很多特征图，不同的滤波器会 得到不同的特征图，不同的特征图代表从图像中提取的不同的特征。我们得到了这么多的特征图，按理来说某些特征图的应该更重要，某些特征图应该没这么重要，并不是所有特征图 都一样的重要。所以 SENet 的核心思想就是给特征图增加注意力和门控机制，增强重要的特征图的信息，减弱不重要的特征图的信息。
2、注意力和门控机制
我们看一下 SENet 的名字 Squeeze-and-Excitation Networks。其中的“Squeeze”中文意思是“挤压”，在模型中 的实际操作其实是压缩特征图的特征，作者使用的压缩特征图的特征的方式是 avg pooling 平均池化。这个大家应该很熟悉了，求一个特征图所有值的平均值，把 avg pooling 计算后的结果作为这个特征图压缩后的特征。比如一共有 64 个特征图，“Squeeze”计算后我们 就会得到 64 个值，代表 64 个特征图压缩后的特征。
“Excitation”中文意思是“激发”，在模型中的实际操作是调节特征图信号强弱，作者 使用的方式是给“Squeeze”计算后的结果加上两个全连接层，最终输出每个特征图对应的 激活值，激活值可以改变特征图信号的强弱。每个特征图乘以它所对应的激活值，得到特征 图的输出，然后再传给下一层。
2.1 SENet block
原论文中给出的图如下所示：
具体实现如下：
- 对输入进来的特征层进行全局平均池化。
- 然后进行两次全连接(这两个全连接可用
1*1卷积代替)，第一次全连接神经元个数较少，第二次全连接神经元个数和输入特征层个数相同。 - 在完成两次全连接之后，再取一次sigmoid讲值固定到0-1之间，此时我们获得了输入特征层每一个通道的权值(0-1之间)。
- 在获得这个权值之后，讲这个权值与原输入特征层相乘即可。
2.2 SENet block的Tensorflow实现
import math
import tensorflow as tf
from tensorflow.keras import backend as K
from tensorflow.keras.layers import (Activation, Add, Concatenate, Conv1D, Conv2D, Dense,
GlobalAveragePooling2D, GlobalMaxPooling2D, Lambda, BatchNormalization,
Reshape, multiply, Input)
from tensorflow.keras.models import Model
from plot_model import plot_modeldef se_block(input_feature, ratio=16, name=""):
# 获得通道数[h,w,c]
channel = input_feature.shape[-1]
# 全局平均池化 [c]
se_feature = GlobalAveragePooling2D()(input_feature)
# [1,1,c]
se_feature = Reshape((1, 1, channel))(se_feature)
# 接下来两次全连接(也可以使用两个1*1卷积)
# 使用两个全连接
se_feature = Dense(channel // ratio,
activation='relu',
kernel_initializer='he_normal',
use_bias=False,
name="se_block_one_" + str(name))(se_feature)
se_feature = Dense(channel,
kernel_initializer='he_normal',
use_bias=False,
name="se_block_two_" + str(name))(se_feature)
# 使用两个1*1卷积
# se_feature = Conv2D(channel // ratio,
# activation='relu',
# kernel_size=[1, 1],
# kernel_initializer='he_normal',
# use_bias=False,
# name="se_block_one_" + str(name))(se_feature)
#
# se_feature = Conv2D(channel,
# kernel_size=[1,1],
# kernel_initializer='he_normal',
# use_bias=False,
# name="se_block_two_" + str(name))(se_feature)
# 使用sigmoid获得输入特征层每一个通道的权值(0-1之间)
se_feature = Activation('sigmoid')(se_feature)
# 将权值乘上原输入特征层即可。
out = multiply([input_feature, se_feature])
return out
这里可以将两个全连接层替换为两个1*1卷积，就是注释掉的那部分代码。
if __name__ == '__main__':
inputs = Input([26, 26, 512])
x = se_block(inputs)
model = Model(inputs, x)
model.summary()
plot_model(model,to_file='img/SeNet.png',show_shapes=True)
3、Se-Inception模块与SE-ResNet模块
图 2. 原始 Inception 模块（左）和 SEInception 模块（右）的架构。
图中的 Global pooling 表示全局池化；W 表示图片宽度；H 表示图片高度；C 表示图片 通道数；FC 表示全连接层；r 表示缩减率，意思是通道数在第一个全连接层缩减多少，总之 就是一个超参数，不用细究，一般取值为 16。
图 3. 原始残差模块（左）和 SEResNet 模块（右）的架构。
图中的 Global pooling 表示全局池化；W 表示图片宽度；H 表示图片高度；C 表示图片 通道数；FC 表示全连接层；r 表示缩减率，意思是通道数在第一个全连接层缩减多少，总之 就是一个超参数，不用细究，一般取值为 16。
4、SE-残差结构的理解
图片来源：《深度学习从0到1》
普通的 ResNet 中的残差结构如下图所示
图中的 Conv 表示卷积；Batch Norm 表示批量标准化；Identity mapping 表示恒等映射；batch 表示批次。
普通的残差结构我们就不需要多说了，我前面也有论文的精读博客。下面我们看一下加上了 Squeeze-and-Excitation 模块后的残差结构如下图所示。
加上 Squeeze-and-Excitation 模块后的残差结构主要变化是在原来的残差结构最后一个 卷积层后面进行 Squeeze-and-Excitation 的操作。Squeeze 就是先做平均池化，得到每一个特征图的压缩特征。图中特征图大小为 56×56，所以池化的窗口大小也是 56×56。池化 过后就是 Excitation 操作，前面我们有提到 Excitation 操作有两个全连接层，这是 SENet 原始论文中的做法，实际我们在写程序的时候也可以用两个窗口大小 1×1 的卷积层的替代，效果跟全连接是一样的。
Excitation 操作部分最后的激活函数是 Sigmoid 函数，作者在这里 使用 Sigmoid 函数主要是利用 Sigmoid 函数输出范围是 0-1 这个特性，让 Excitation 的输 出激活值可以起到一个门控的作用。Excitation 的输出的激活值会乘以原始残差结构最后一 个卷积层的输出结果，对特征图的数值大小进行控制。如果是重要的特征图，会保持比较大 的数值；如果是不重要的特征图，特征图的数值就会变小。
5、ResNet-50，SE-ResNet-50，SE-ResNeXt-50(32×4d)模型结构
（左）ResNet-50 [13]。 （中）SE-ResNet-50。 （右）带有 32×4d 模板的 SE-ResNeXt-50。括号内列出了残差构建块的特定参数设置的形状和操作，外部显示了阶段中堆叠块的数量。 fc后面的内括号表示SE模块中两个全连接层的输出维度。图中的 Output size 表示输出大小；conv 表示卷积；max pool 表示最大池化；stride 表示步长；fc 表示全连接层；global average pool 表示全局平均池化；C 表示分组数。
6、SeNet结果分析
基础模型增加 SE 模块后会使得整体模型的参数增加 10%左右，计算量增加不多，一般 来说模型的效果也会有所提升。作者使用多个模型在 ImageNet 数据集上进行了测试，下图为多个模型在 ImageNet 验证集测试结果。
图中 original 表示模型原始论文中的结果；re-implementation 表示 SENet 作者重新训 练模型的结果；SENet 表示给这些模型加上 SE 模块后的结果；top-1 err.表示 top1 错误 率；top-5 err.表示 top5 错误率；GFLOPs 表示计算量。
图中结果可以看出，图中测试的所有模型只要加上 SE 模块，错误率都能降低，并且模型 浮点计算量没有太大变化。
下图 也能看出加上 SE 模块后模型效果可以变得更好：
图中的 epochs 表示周期；Top-1 error 表示 Top1 错误率；train 表示训练集；val 表示 验证集。
SENet 论文的最后，作者还给了一组很有意思的图。作者用 ImageNet 数据集训练了一 个 SE-ResNet-50，然后选出 4 个种类(goldfish,pug,plane,cliff)的图片，统计这 4 个种类在 SE-ResNet-50 模型的每个 SE 模块的特征图的激活情况，如下图所示。
图中的 all 表示所有 1000 个种类的平均值；goldfish 表示金鱼；pug 表示哈巴狗； plane 表示飞机；cliff 表示悬崖；channel index 表示通道；activation 表示激活值
作者观察实验结果得到 3 个结论：
- 第一，不同种类的物体在浅层激活分布情况是类似的，如图中的 SE_2_3 和 SE_3_4。也 就是不管是识别哪种物体，浅层的卷积层中，重要的特征图总是比较固定的那些。
- 第二，在更深层一些的位置，不同种类在不同的特征图激活分布不同，因为不同类别对 特征有不同的偏好，如图中的 SE_4_6 和 SE_5_1。低层特征通常更普遍，识别不同种类物体可以使用类似的滤波器，而高层特征通常包含更多细节，识别不同种类物体需要使用不同的滤波器。
- 第三，在模型的最后阶段，SE_5_2 呈现出饱和状态，其中大部分激活值都接近于 1，也 有一些接近于 0。对于激活值为 1 的特征图，相当于 SE 模块不存在。在网络的最后一个 SE 模块 SE_5_3，不同种类有着类似的分布，只是尺度不同。也就是说 SE_5_2 和 SE_5_3 相对 来说没有前面的一些 SE 模块重要，作者通过实验发现删除最后一个阶段的 SE 模块，总体参 数可以显著减少，性能只有一点损失(<0.1%的 Top1 错误率)。
7、SE-ResNet50图像识别
import numpy as np
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Input,Dense,Dropout,Conv2D,MaxPool2D,Flatten,GlobalAvgPool2D,BatchNormalization,Activation,Add,ZeroPadding2D,Multiply
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt
from tensorflow.keras.callbacks import LearningRateScheduler
from tensorflow.keras.models import Model
from tensorflow.keras.utils import plot_model# 类别数
num_classes = 17
# 批次大小
batch_size = 32
# 周期数
epochs = 100
# 图片大小
image_size = 224# 训练集数据进行数据增强
train_datagen = ImageDataGenerator(
rotation_range = 20, # 随机旋转度数
width_shift_range = 0.1, # 随机水平平移
height_shift_range = 0.1,# 随机竖直平移
rescale = 1/255, # 数据归一化
shear_range = 10, # 随机错切变换
zoom_range = 0.1, # 随机放大
horizontal_flip = True, # 水平翻转
brightness_range=(0.7, 1.3), # 亮度变化
fill_mode = 'nearest', # 填充方式
)
# 测试集数据只需要归一化就可以
test_datagen = ImageDataGenerator(
rescale = 1/255, # 数据归一化
) # 训练集数据生成器，可以在训练时自动产生数据进行训练
# 从'data/train'获得训练集数据
# 获得数据后会把图片resize为image_size×image_size的大小
# generator每次会产生batch_size个数据
train_generator = train_datagen.flow_from_directory(
'data/train',
target_size=(image_size,image_size),
batch_size=batch_size,
)
# 测试集数据生成器
test_generator = test_datagen.flow_from_directory(
'data/test',
target_size=(image_size,image_size),
batch_size=batch_size,
)
# 字典的键为17个文件夹的名字，值为对应的分类编号
train_generator.class_indices
# SE模块
def ChannelSE(input_tensor, reduction=16):
# 获得信号通道数
channels = input_tensor.shape[-1]
# SE模块
x = GlobalAvgPool2D()(input_tensor)
# 把2维数据再变成4维(?,1,1,?)
x = x[:, None, None, :]
# 卷积替代全连接层
x = Conv2D(filters=channels//reduction,kernel_size=1,strides=1)(x)
x = Activation('relu')(x)
x = Conv2D(filters=channels,kernel_size=1,strides=1)(x)
x = Activation('sigmoid')(x)
x = Multiply()([input_tensor, x])
return x
# 定义残差单元
def block(x, filters, strides=1, conv_shortcut=True, reduction=16):
# projection shortcut
if conv_shortcut == True:
shortcut = Conv2D(filters*4,kernel_size=1,strides=strides,padding='valid')(x)
# epsilon为BN公式中防止分母为零的值
shortcut = BatchNormalization(epsilon=1.001e-5)(shortcut)
else:
# identity_shortcut
shortcut = x
# 3个卷积层
x = Conv2D(filters=filters,kernel_size=1,strides=strides,padding='valid')(x)
x = BatchNormalization(epsilon=1.001e-5)(x)
x = Activation('relu')(x)
x = Conv2D(filters=filters,kernel_size=3,strides=1,padding='same')(x)
x = BatchNormalization(epsilon=1.001e-5)(x)
x = Activation('relu')(x)
x = Conv2D(filters=filters*4,kernel_size=1,strides=1,padding='valid')(x)
x = BatchNormalization(epsilon=1.001e-5)(x)
# SE模块
x = ChannelSE(x, reduction=reduction)
x = Add()([x, shortcut])
x = Activation('relu')(x)
return x
# 堆叠残差单元
def stack(x, filters, blocks, strides):
x = block(x, filters, strides=strides)
for i in range(blocks-1):
x = block(x, filters, conv_shortcut=False)
return x
# 定义SE-ResNet50
inputs = Input(shape=(image_size,image_size,3))
# 填充3圈0，填充后图像从224×224变成230×230
x = ZeroPadding2D((3, 3))(inputs)
x= Conv2D(filters=64,kernel_size=7,strides=2,padding='valid')(x)
x = BatchNormalization(epsilon=1.001e-5)(x)
x = Activation('relu')(x)
# 填充1圈0
x = ZeroPadding2D((1, 1))(x)
x = MaxPool2D(pool_size=3,strides=2,padding='valid')(x)
# 堆叠残差结构
# blocks表示堆叠数量
x = stack(x, filters=64, blocks=3, strides=1)
x = stack(x, filters=128, blocks=4, strides=2)
x = stack(x, filters=256, blocks=6, strides=2)
x = stack(x, filters=512, blocks=3, strides=2)
# 根据特征图大小进行平均池化，池化后得到2维数据
x = GlobalAvgPool2D()(x)
x = Dense(num_classes, activation='softmax')(x)
# 定义模型
model = Model(inputs=inputs,outputs=x)# 学习率调节函数，逐渐减小学习率
def adjust_learning_rate(epoch):
# 前40周期
if epoch<=40:
lr = 1e-4
# 前40到80周期
elif epoch>40 and epoch<=80:
lr = 1e-5
# 80到100周期
else:
lr = 1e-6
return lr# 定义优化器
adam = Adam(lr=1e-4)
# 定义学习率衰减策略
callbacks = []
callbacks.append(LearningRateScheduler(adjust_learning_rate))
# 定义优化器，loss function，训练过程中计算准确率
model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy'])
# Tensorflow2.1版本之前可以使用fit_generator训练模型
# history = model.fit_generator(train_generator,steps_per_epoch=len(train_generator),epochs=epochs,validation_data=test_generator,validation_steps=len(test_generator))
# Tensorflow2.1版本(包括2.1)之后可以直接使用fit训练模型
history = model.fit(x=train_generator,epochs=epochs,validation_data=test_generator,callbacks=callbacks)Epoch 1/100
34/34 [==============================] - 19s 549ms/step - loss: 2.5603 - accuracy: 0.1930 - val_loss: 2.8569 - val_accuracy: 0.0588
Epoch 2/100
34/34 [==============================] - 17s 501ms/step - loss: 1.9323 - accuracy: 0.3603 - val_loss: 3.0023 - val_accuracy: 0.0588
Epoch 3/100
34/34 [==============================] - 18s 519ms/step - loss: 1.5821 - accuracy: 0.4614 - val_loss: 3.3108 - val_accuracy: 0.0588
Epoch 4/100
34/34 [==============================] - 17s 508ms/step - loss: 1.4403 - accuracy: 0.5285 - val_loss: 3.5824 - val_accuracy: 0.0588
Epoch 5/100
34/34 [==============================] - 17s 510ms/step - loss: 1.2837 - accuracy: 0.5781 - val_loss: 4.0358 - val_accuracy: 0.0625
Epoch 6/100
34/34 [==============================] - 17s 507ms/step - loss: 1.2065 - accuracy: 0.6075 - val_loss: 4.3552 - val_accuracy: 0.0588
Epoch 7/100
34/34 [==============================] - 18s 515ms/step - loss: 1.1144 - accuracy: 0.6259 - val_loss: 4.0170 - val_accuracy: 0.0846
Epoch 8/100
34/34 [==============================] - 17s 507ms/step - loss: 1.0141 - accuracy: 0.6572 - val_loss: 4.4925 - val_accuracy: 0.0625
Epoch 9/100
34/34 [==============================] - 17s 510ms/step - loss: 0.9487 - accuracy: 0.6857 - val_loss: 4.7573 - val_accuracy: 0.0993
Epoch 10/100
34/34 [==============================] - 17s 503ms/step - loss: 0.8503 - accuracy: 0.7132 - val_loss: 4.5246 - val_accuracy: 0.0919
Epoch 11/100
34/34 [==============================] - 17s 499ms/step - loss: 0.8037 - accuracy: 0.7298 - val_loss: 4.4767 - val_accuracy: 0.1213
Epoch 12/100
34/34 [==============================] - 17s 507ms/step - loss: 0.7193 - accuracy: 0.7518 - val_loss: 4.1744 - val_accuracy: 0.2243
Epoch 13/100
34/34 [==============================] - 17s 506ms/step - loss: 0.6467 - accuracy: 0.7904 - val_loss: 3.3856 - val_accuracy: 0.2537
Epoch 14/100
34/34 [==============================] - 17s 511ms/step - loss: 0.6490 - accuracy: 0.7748 - val_loss: 3.3443 - val_accuracy: 0.2941
Epoch 15/100
34/34 [==============================] - 17s 501ms/step - loss: 0.6287 - accuracy: 0.7914 - val_loss: 2.8897 - val_accuracy: 0.3603
Epoch 16/100
34/34 [==============================] - 17s 510ms/step - loss: 0.5383 - accuracy: 0.8088 - val_loss: 2.5651 - val_accuracy: 0.4338
Epoch 17/100
34/34 [==============================] - 18s 515ms/step - loss: 0.5573 - accuracy: 0.8033 - val_loss: 2.0221 - val_accuracy: 0.5000
Epoch 18/100
34/34 [==============================] - 17s 506ms/step - loss: 0.5236 - accuracy: 0.8217 - val_loss: 2.5051 - val_accuracy: 0.4485
Epoch 19/100
34/34 [==============================] - 17s 511ms/step - loss: 0.5003 - accuracy: 0.8355 - val_loss: 2.3877 - val_accuracy: 0.5000
Epoch 20/100
34/34 [==============================] - 17s 513ms/step - loss: 0.4758 - accuracy: 0.8290 - val_loss: 1.5135 - val_accuracy: 0.6728
Epoch 21/100
34/34 [==============================] - 17s 511ms/step - loss: 0.4111 - accuracy: 0.8621 - val_loss: 1.4329 - val_accuracy: 0.6176
Epoch 22/100
34/34 [==============================] - 17s 509ms/step - loss: 0.3885 - accuracy: 0.8768 - val_loss: 1.5033 - val_accuracy: 0.6434
Epoch 23/100
34/34 [==============================] - 17s 507ms/step - loss: 0.3462 - accuracy: 0.8851 - val_loss: 2.2281 - val_accuracy: 0.5000
Epoch 24/100
34/34 [==============================] - 17s 510ms/step - loss: 0.3518 - accuracy: 0.8824 - val_loss: 2.0414 - val_accuracy: 0.6176
Epoch 25/100
34/34 [==============================] - 17s 506ms/step - loss: 0.3619 - accuracy: 0.8704 - val_loss: 1.8381 - val_accuracy: 0.6287
Epoch 26/100
34/34 [==============================] - 17s 513ms/step - loss: 0.3551 - accuracy: 0.8814 - val_loss: 1.4278 - val_accuracy: 0.6507
Epoch 27/100
34/34 [==============================] - 18s 516ms/step - loss: 0.3022 - accuracy: 0.9017 - val_loss: 1.3334 - val_accuracy: 0.6838
Epoch 28/100
34/34 [==============================] - 17s 508ms/step - loss: 0.2860 - accuracy: 0.9072 - val_loss: 1.3219 - val_accuracy: 0.6875
Epoch 29/100
34/34 [==============================] - 17s 510ms/step - loss: 0.2219 - accuracy: 0.9219 - val_loss: 0.9909 - val_accuracy: 0.7132
Epoch 30/100
34/34 [==============================] - 17s 507ms/step - loss: 0.2143 - accuracy: 0.9228 - val_loss: 1.1023 - val_accuracy: 0.7353
Epoch 31/100
34/34 [==============================] - 17s 514ms/step - loss: 0.2513 - accuracy: 0.9136 - val_loss: 1.4077 - val_accuracy: 0.6691
Epoch 32/100
34/34 [==============================] - 17s 511ms/step - loss: 0.2759 - accuracy: 0.8980 - val_loss: 2.3026 - val_accuracy: 0.5772
Epoch 33/100
34/34 [==============================] - 17s 511ms/step - loss: 0.2204 - accuracy: 0.9219 - val_loss: 1.7835 - val_accuracy: 0.6213
Epoch 34/100
34/34 [==============================] - 17s 506ms/step - loss: 0.2390 - accuracy: 0.9228 - val_loss: 1.9852 - val_accuracy: 0.6103
Epoch 35/100
34/34 [==============================] - 17s 504ms/step - loss: 0.1942 - accuracy: 0.9301 - val_loss: 1.1830 - val_accuracy: 0.6765
Epoch 36/100
34/34 [==============================] - 18s 516ms/step - loss: 0.1756 - accuracy: 0.9403 - val_loss: 1.2185 - val_accuracy: 0.6985
Epoch 37/100
34/34 [==============================] - 17s 510ms/step - loss: 0.1713 - accuracy: 0.9403 - val_loss: 1.2769 - val_accuracy: 0.7243
Epoch 38/100
34/34 [==============================] - 17s 509ms/step - loss: 0.1709 - accuracy: 0.9430 - val_loss: 1.6684 - val_accuracy: 0.6654
Epoch 39/100
34/34 [==============================] - 17s 506ms/step - loss: 0.1908 - accuracy: 0.9467 - val_loss: 1.4263 - val_accuracy: 0.7096
Epoch 40/100
34/34 [==============================] - 17s 502ms/step - loss: 0.1768 - accuracy: 0.9338 - val_loss: 1.2879 - val_accuracy: 0.7096
Epoch 41/100
34/34 [==============================] - 17s 513ms/step - loss: 0.1483 - accuracy: 0.9504 - val_loss: 1.0808 - val_accuracy: 0.7390
Epoch 42/100
34/34 [==============================] - 17s 511ms/step - loss: 0.1057 - accuracy: 0.9623 - val_loss: 0.8448 - val_accuracy: 0.7868
Epoch 43/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0670 - accuracy: 0.9807 - val_loss: 0.7289 - val_accuracy: 0.8162
Epoch 44/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0768 - accuracy: 0.9779 - val_loss: 0.6560 - val_accuracy: 0.8162
Epoch 45/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0568 - accuracy: 0.9871 - val_loss: 0.6800 - val_accuracy: 0.8199
Epoch 46/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0612 - accuracy: 0.9871 - val_loss: 0.6611 - val_accuracy: 0.8309
Epoch 47/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0499 - accuracy: 0.9899 - val_loss: 0.6868 - val_accuracy: 0.8015
Epoch 48/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0561 - accuracy: 0.9816 - val_loss: 0.6607 - val_accuracy: 0.8051
Epoch 49/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0418 - accuracy: 0.9926 - val_loss: 0.6790 - val_accuracy: 0.8015
Epoch 50/100
34/34 [==============================] - 18s 516ms/step - loss: 0.0383 - accuracy: 0.9917 - val_loss: 0.6684 - val_accuracy: 0.8162
Epoch 51/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0416 - accuracy: 0.9917 - val_loss: 0.6851 - val_accuracy: 0.8051
Epoch 52/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0435 - accuracy: 0.9881 - val_loss: 0.6635 - val_accuracy: 0.8051
Epoch 53/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0457 - accuracy: 0.9917 - val_loss: 0.6653 - val_accuracy: 0.8199
Epoch 54/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0444 - accuracy: 0.9862 - val_loss: 0.7060 - val_accuracy: 0.8125
Epoch 55/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0497 - accuracy: 0.9862 - val_loss: 0.7086 - val_accuracy: 0.8051
Epoch 56/100
34/34 [==============================] - 18s 516ms/step - loss: 0.0355 - accuracy: 0.9926 - val_loss: 0.6972 - val_accuracy: 0.8162
Epoch 57/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0407 - accuracy: 0.9917 - val_loss: 0.7006 - val_accuracy: 0.8088
Epoch 58/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0429 - accuracy: 0.9890 - val_loss: 0.6793 - val_accuracy: 0.8162
Epoch 59/100
34/34 [==============================] - 17s 509ms/step - loss: 0.0327 - accuracy: 0.9936 - val_loss: 0.6627 - val_accuracy: 0.8199
Epoch 60/100
34/34 [==============================] - 17s 504ms/step - loss: 0.0449 - accuracy: 0.9890 - val_loss: 0.6477 - val_accuracy: 0.8088
Epoch 61/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0349 - accuracy: 0.9926 - val_loss: 0.6539 - val_accuracy: 0.8125
Epoch 62/100
34/34 [==============================] - 17s 500ms/step - loss: 0.0273 - accuracy: 0.9963 - val_loss: 0.6503 - val_accuracy: 0.8235
Epoch 63/100
34/34 [==============================] - 17s 509ms/step - loss: 0.0287 - accuracy: 0.9972 - val_loss: 0.6243 - val_accuracy: 0.8346
Epoch 64/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0306 - accuracy: 0.9954 - val_loss: 0.6159 - val_accuracy: 0.8272
Epoch 65/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0272 - accuracy: 0.9936 - val_loss: 0.6473 - val_accuracy: 0.8125
Epoch 66/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0257 - accuracy: 0.9972 - val_loss: 0.6518 - val_accuracy: 0.8088
Epoch 67/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0325 - accuracy: 0.9954 - val_loss: 0.6649 - val_accuracy: 0.8015
Epoch 68/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0269 - accuracy: 0.9963 - val_loss: 0.6461 - val_accuracy: 0.8088
Epoch 69/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0278 - accuracy: 0.9954 - val_loss: 0.6525 - val_accuracy: 0.8088
Epoch 70/100
34/34 [==============================] - 17s 513ms/step - loss: 0.0347 - accuracy: 0.9899 - val_loss: 0.6667 - val_accuracy: 0.8088
Epoch 71/100
34/34 [==============================] - 17s 500ms/step - loss: 0.0296 - accuracy: 0.9936 - val_loss: 0.6512 - val_accuracy: 0.8235
Epoch 72/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0373 - accuracy: 0.9899 - val_loss: 0.6657 - val_accuracy: 0.8125
Epoch 73/100
34/34 [==============================] - 17s 500ms/step - loss: 0.0307 - accuracy: 0.9954 - val_loss: 0.6822 - val_accuracy: 0.8051
Epoch 74/100
34/34 [==============================] - 17s 499ms/step - loss: 0.0332 - accuracy: 0.9899 - val_loss: 0.6935 - val_accuracy: 0.8162
Epoch 75/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0187 - accuracy: 0.9991 - val_loss: 0.6491 - val_accuracy: 0.8162
Epoch 76/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0243 - accuracy: 0.9963 - val_loss: 0.6576 - val_accuracy: 0.8272
Epoch 77/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0227 - accuracy: 0.9945 - val_loss: 0.6988 - val_accuracy: 0.8162
Epoch 78/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0205 - accuracy: 0.9954 - val_loss: 0.7424 - val_accuracy: 0.8162
Epoch 79/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0298 - accuracy: 0.9936 - val_loss: 0.6804 - val_accuracy: 0.8309
Epoch 80/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0246 - accuracy: 0.9936 - val_loss: 0.7717 - val_accuracy: 0.8015
Epoch 81/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0294 - accuracy: 0.9945 - val_loss: 0.6915 - val_accuracy: 0.7978
Epoch 82/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0223 - accuracy: 0.9954 - val_loss: 0.6843 - val_accuracy: 0.8015
Epoch 83/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0260 - accuracy: 0.9945 - val_loss: 0.6777 - val_accuracy: 0.8051
Epoch 84/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0250 - accuracy: 0.9926 - val_loss: 0.6703 - val_accuracy: 0.8051
Epoch 85/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0263 - accuracy: 0.9954 - val_loss: 0.6659 - val_accuracy: 0.8088
Epoch 86/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0189 - accuracy: 0.9963 - val_loss: 0.6651 - val_accuracy: 0.8088
Epoch 87/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0250 - accuracy: 0.9954 - val_loss: 0.6708 - val_accuracy: 0.8199
Epoch 88/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0247 - accuracy: 0.9945 - val_loss: 0.6672 - val_accuracy: 0.8199
Epoch 89/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0237 - accuracy: 0.9936 - val_loss: 0.6719 - val_accuracy: 0.8235
Epoch 90/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0268 - accuracy: 0.9936 - val_loss: 0.6678 - val_accuracy: 0.8235
Epoch 91/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0228 - accuracy: 0.9936 - val_loss: 0.6671 - val_accuracy: 0.8199
Epoch 92/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0154 - accuracy: 1.0000 - val_loss: 0.6638 - val_accuracy: 0.8235
Epoch 93/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0206 - accuracy: 0.9991 - val_loss: 0.6617 - val_accuracy: 0.8199
Epoch 94/100
34/34 [==============================] - 17s 501ms/step - loss: 0.0249 - accuracy: 0.9954 - val_loss: 0.6695 - val_accuracy: 0.8235
Epoch 95/100
34/34 [==============================] - 17s 509ms/step - loss: 0.0282 - accuracy: 0.9945 - val_loss: 0.6846 - val_accuracy: 0.8235
Epoch 96/100
34/34 [==============================] - 17s 513ms/step - loss: 0.0196 - accuracy: 0.9963 - val_loss: 0.6784 - val_accuracy: 0.8272
Epoch 97/100
34/34 [==============================] - 18s 523ms/step - loss: 0.0255 - accuracy: 0.9917 - val_loss: 0.6732 - val_accuracy: 0.8346
Epoch 98/100
34/34 [==============================] - 17s 513ms/step - loss: 0.0278 - accuracy: 0.9899 - val_loss: 0.6697 - val_accuracy: 0.8309
Epoch 99/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0163 - accuracy: 0.9963 - val_loss: 0.6711 - val_accuracy: 0.8309
Epoch 100/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0238 - accuracy: 0.9945 - val_loss: 0.6632 - val_accuracy: 0.8272可视化acc
# 画出训练集准确率曲线图
plt.plot(np.arange(epochs),history.history['accuracy'],c='b',label='train_accuracy')
# 画出验证集准确率曲线图
plt.plot(np.arange(epochs),history.history['val_accuracy'],c='y',label='val_accuracy')
# 图例
plt.legend()
# x坐标描述
plt.xlabel('epochs')
# y坐标描述
plt.ylabel('accuracy')
# 显示图像
plt.show()
# 模型保存
model.save('SE-ResNet50.h5')
模型结构如下所示：
Model: "functional_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 224, 224, 3) 0
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D) (None, 230, 230, 3) 0 input_1[0][0]
__________________________________________________________________________________________________
conv2d (Conv2D) (None, 112, 112, 64) 9472 zero_padding2d[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 112, 112, 64) 256 conv2d[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, 112, 112, 64) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, 114, 114, 64) 0 activation[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 56, 56, 64) 0 zero_padding2d_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 56, 56, 64) 4160 max_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 56, 56, 64) 256 conv2d_2[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, 56, 56, 64) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 56, 56, 64) 36928 activation_1[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 56, 56, 64) 256 conv2d_3[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, 56, 56, 64) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 56, 56, 256) 16640 activation_2[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 56, 56, 256) 1024 conv2d_4[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 256) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice (Tens (None, 1, 1, 256) 0 global_average_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 1, 1, 16) 4112 tf_op_layer_strided_slice[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, 1, 1, 16) 0 conv2d_5[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 1, 1, 256) 4352 activation_3[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 1, 1, 256) 0 conv2d_6[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 56, 56, 256) 16640 max_pooling2d[0][0]
__________________________________________________________________________________________________
multiply (Multiply) (None, 56, 56, 256) 0 batch_normalization_4[0][0]
activation_4[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 56, 56, 256) 1024 conv2d_1[0][0]
__________________________________________________________________________________________________
add (Add) (None, 56, 56, 256) 0 multiply[0][0]
batch_normalization_1[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 56, 56, 256) 0 add[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 56, 56, 64) 16448 activation_5[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 56, 56, 64) 256 conv2d_7[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, 56, 56, 64) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 56, 56, 64) 36928 activation_6[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 56, 56, 64) 256 conv2d_8[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, 56, 56, 64) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 56, 56, 256) 16640 activation_7[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 56, 56, 256) 1024 conv2d_9[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_1 (Glo (None, 256) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_1 (Te (None, 1, 1, 256) 0 global_average_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 1, 1, 16) 4112 tf_op_layer_strided_slice_1[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, 1, 1, 16) 0 conv2d_10[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 1, 1, 256) 4352 activation_8[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 1, 1, 256) 0 conv2d_11[0][0]
__________________________________________________________________________________________________
multiply_1 (Multiply) (None, 56, 56, 256) 0 batch_normalization_7[0][0]
activation_9[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 56, 56, 256) 0 multiply_1[0][0]
activation_5[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, 56, 56, 256) 0 add_1[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 56, 56, 64) 16448 activation_10[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 56, 56, 64) 256 conv2d_12[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, 56, 56, 64) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 56, 56, 64) 36928 activation_11[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 56, 56, 64) 256 conv2d_13[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, 56, 56, 64) 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 56, 56, 256) 16640 activation_12[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 56, 56, 256) 1024 conv2d_14[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_2 (Glo (None, 256) 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_2 (Te (None, 1, 1, 256) 0 global_average_pooling2d_2[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 1, 1, 16) 4112 tf_op_layer_strided_slice_2[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, 1, 1, 16) 0 conv2d_15[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 1, 1, 256) 4352 activation_13[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, 1, 1, 256) 0 conv2d_16[0][0]
__________________________________________________________________________________________________
multiply_2 (Multiply) (None, 56, 56, 256) 0 batch_normalization_10[0][0]
activation_14[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 56, 56, 256) 0 multiply_2[0][0]
activation_10[0][0]
__________________________________________________________________________________________________
activation_15 (Activation) (None, 56, 56, 256) 0 add_2[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, 28, 28, 128) 32896 activation_15[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 28, 28, 128) 512 conv2d_18[0][0]
__________________________________________________________________________________________________
activation_16 (Activation) (None, 28, 28, 128) 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, 28, 28, 128) 147584 activation_16[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 28, 28, 128) 512 conv2d_19[0][0]
__________________________________________________________________________________________________
activation_17 (Activation) (None, 28, 28, 128) 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, 28, 28, 512) 66048 activation_17[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 28, 28, 512) 2048 conv2d_20[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_3 (Glo (None, 512) 0 batch_normalization_14[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_3 (Te (None, 1, 1, 512) 0 global_average_pooling2d_3[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, 1, 1, 32) 16416 tf_op_layer_strided_slice_3[0][0]
__________________________________________________________________________________________________
activation_18 (Activation) (None, 1, 1, 32) 0 conv2d_21[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, 1, 1, 512) 16896 activation_18[0][0]
__________________________________________________________________________________________________
activation_19 (Activation) (None, 1, 1, 512) 0 conv2d_22[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, 28, 28, 512) 131584 activation_15[0][0]
__________________________________________________________________________________________________
multiply_3 (Multiply) (None, 28, 28, 512) 0 batch_normalization_14[0][0]
activation_19[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 28, 28, 512) 2048 conv2d_17[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 28, 28, 512) 0 multiply_3[0][0]
batch_normalization_11[0][0]
__________________________________________________________________________________________________
activation_20 (Activation) (None, 28, 28, 512) 0 add_3[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 28, 28, 128) 65664 activation_20[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 28, 28, 128) 512 conv2d_23[0][0]
__________________________________________________________________________________________________
activation_21 (Activation) (None, 28, 28, 128) 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 28, 28, 128) 147584 activation_21[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 28, 28, 128) 512 conv2d_24[0][0]
__________________________________________________________________________________________________
activation_22 (Activation) (None, 28, 28, 128) 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, 28, 28, 512) 66048 activation_22[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 28, 28, 512) 2048 conv2d_25[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_4 (Glo (None, 512) 0 batch_normalization_17[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_4 (Te (None, 1, 1, 512) 0 global_average_pooling2d_4[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, 1, 1, 32) 16416 tf_op_layer_strided_slice_4[0][0]
__________________________________________________________________________________________________
activation_23 (Activation) (None, 1, 1, 32) 0 conv2d_26[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, 1, 1, 512) 16896 activation_23[0][0]
__________________________________________________________________________________________________
activation_24 (Activation) (None, 1, 1, 512) 0 conv2d_27[0][0]
__________________________________________________________________________________________________
multiply_4 (Multiply) (None, 28, 28, 512) 0 batch_normalization_17[0][0]
activation_24[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 28, 28, 512) 0 multiply_4[0][0]
activation_20[0][0]
__________________________________________________________________________________________________
activation_25 (Activation) (None, 28, 28, 512) 0 add_4[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, 28, 28, 128) 65664 activation_25[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 28, 28, 128) 512 conv2d_28[0][0]
__________________________________________________________________________________________________
activation_26 (Activation) (None, 28, 28, 128) 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, 28, 28, 128) 147584 activation_26[0][0]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 28, 28, 128) 512 conv2d_29[0][0]
__________________________________________________________________________________________________
activation_27 (Activation) (None, 28, 28, 128) 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, 28, 28, 512) 66048 activation_27[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 28, 28, 512) 2048 conv2d_30[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_5 (Glo (None, 512) 0 batch_normalization_20[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_5 (Te (None, 1, 1, 512) 0 global_average_pooling2d_5[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, 1, 1, 32) 16416 tf_op_layer_strided_slice_5[0][0]
__________________________________________________________________________________________________
activation_28 (Activation) (None, 1, 1, 32) 0 conv2d_31[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, 1, 1, 512) 16896 activation_28[0][0]
__________________________________________________________________________________________________
activation_29 (Activation) (None, 1, 1, 512) 0 conv2d_32[0][0]
__________________________________________________________________________________________________
multiply_5 (Multiply) (None, 28, 28, 512) 0 batch_normalization_20[0][0]
activation_29[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 28, 28, 512) 0 multiply_5[0][0]
activation_25[0][0]
__________________________________________________________________________________________________
activation_30 (Activation) (None, 28, 28, 512) 0 add_5[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, 28, 28, 128) 65664 activation_30[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 28, 28, 128) 512 conv2d_33[0][0]
__________________________________________________________________________________________________
activation_31 (Activation) (None, 28, 28, 128) 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, 28, 28, 128) 147584 activation_31[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 28, 28, 128) 512 conv2d_34[0][0]
__________________________________________________________________________________________________
activation_32 (Activation) (None, 28, 28, 128) 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, 28, 28, 512) 66048 activation_32[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 28, 28, 512) 2048 conv2d_35[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_6 (Glo (None, 512) 0 batch_normalization_23[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_6 (Te (None, 1, 1, 512) 0 global_average_pooling2d_6[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 1, 1, 32) 16416 tf_op_layer_strided_slice_6[0][0]
__________________________________________________________________________________________________
activation_33 (Activation) (None, 1, 1, 32) 0 conv2d_36[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, 1, 1, 512) 16896 activation_33[0][0]
__________________________________________________________________________________________________
activation_34 (Activation) (None, 1, 1, 512) 0 conv2d_37[0][0]
__________________________________________________________________________________________________
multiply_6 (Multiply) (None, 28, 28, 512) 0 batch_normalization_23[0][0]
activation_34[0][0]
__________________________________________________________________________________________________
add_6 (Add) (None, 28, 28, 512) 0 multiply_6[0][0]
activation_30[0][0]
__________________________________________________________________________________________________
activation_35 (Activation) (None, 28, 28, 512) 0 add_6[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, 14, 14, 256) 131328 activation_35[0][0]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 14, 14, 256) 1024 conv2d_39[0][0]
__________________________________________________________________________________________________
activation_36 (Activation) (None, 14, 14, 256) 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, 14, 14, 256) 590080 activation_36[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 14, 14, 256) 1024 conv2d_40[0][0]
__________________________________________________________________________________________________
activation_37 (Activation) (None, 14, 14, 256) 0 batch_normalization_26[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, 14, 14, 1024) 263168 activation_37[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 14, 14, 1024) 4096 conv2d_41[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_7 (Glo (None, 1024) 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_7 (Te (None, 1, 1, 1024) 0 global_average_pooling2d_7[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, 1, 1, 64) 65600 tf_op_layer_strided_slice_7[0][0]
__________________________________________________________________________________________________
activation_38 (Activation) (None, 1, 1, 64) 0 conv2d_42[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, 1, 1, 1024) 66560 activation_38[0][0]
__________________________________________________________________________________________________
activation_39 (Activation) (None, 1, 1, 1024) 0 conv2d_43[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, 14, 14, 1024) 525312 activation_35[0][0]
__________________________________________________________________________________________________
multiply_7 (Multiply) (None, 14, 14, 1024) 0 batch_normalization_27[0][0]
activation_39[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 14, 14, 1024) 4096 conv2d_38[0][0]
__________________________________________________________________________________________________
add_7 (Add) (None, 14, 14, 1024) 0 multiply_7[0][0]
batch_normalization_24[0][0]
__________________________________________________________________________________________________
activation_40 (Activation) (None, 14, 14, 1024) 0 add_7[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, 14, 14, 256) 262400 activation_40[0][0]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 256) 1024 conv2d_44[0][0]
__________________________________________________________________________________________________
activation_41 (Activation) (None, 14, 14, 256) 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, 14, 14, 256) 590080 activation_41[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 14, 14, 256) 1024 conv2d_45[0][0]
__________________________________________________________________________________________________
activation_42 (Activation) (None, 14, 14, 256) 0 batch_normalization_29[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, 14, 14, 1024) 263168 activation_42[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 14, 14, 1024) 4096 conv2d_46[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_8 (Glo (None, 1024) 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_8 (Te (None, 1, 1, 1024) 0 global_average_pooling2d_8[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, 1, 1, 64) 65600 tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
activation_43 (Activation) (None, 1, 1, 64) 0 conv2d_47[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, 1, 1, 1024) 66560 activation_43[0][0]
__________________________________________________________________________________________________
activation_44 (Activation) (None, 1, 1, 1024) 0 conv2d_48[0][0]
__________________________________________________________________________________________________
multiply_8 (Multiply) (None, 14, 14, 1024) 0 batch_normalization_30[0][0]
activation_44[0][0]
__________________________________________________________________________________________________
add_8 (Add) (None, 14, 14, 1024) 0 multiply_8[0][0]
activation_40[0][0]
__________________________________________________________________________________________________
activation_45 (Activation) (None, 14, 14, 1024) 0 add_8[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (None, 14, 14, 256) 262400 activation_45[0][0]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 256) 1024 conv2d_49[0][0]
__________________________________________________________________________________________________
activation_46 (Activation) (None, 14, 14, 256) 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (None, 14, 14, 256) 590080 activation_46[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 256) 1024 conv2d_50[0][0]
__________________________________________________________________________________________________
activation_47 (Activation) (None, 14, 14, 256) 0 batch_normalization_32[0][0]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (None, 14, 14, 1024) 263168 activation_47[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 1024) 4096 conv2d_51[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_9 (Glo (None, 1024) 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_9 (Te (None, 1, 1, 1024) 0 global_average_pooling2d_9[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (None, 1, 1, 64) 65600 tf_op_layer_strided_slice_9[0][0]
__________________________________________________________________________________________________
activation_48 (Activation) (None, 1, 1, 64) 0 conv2d_52[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (None, 1, 1, 1024) 66560 activation_48[0][0]
__________________________________________________________________________________________________
activation_49 (Activation) (None, 1, 1, 1024) 0 conv2d_53[0][0]
__________________________________________________________________________________________________
multiply_9 (Multiply) (None, 14, 14, 1024) 0 batch_normalization_33[0][0]
activation_49[0][0]
__________________________________________________________________________________________________
add_9 (Add) (None, 14, 14, 1024) 0 multiply_9[0][0]
activation_45[0][0]
__________________________________________________________________________________________________
activation_50 (Activation) (None, 14, 14, 1024) 0 add_9[0][0]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (None, 14, 14, 256) 262400 activation_50[0][0]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 256) 1024 conv2d_54[0][0]
__________________________________________________________________________________________________
activation_51 (Activation) (None, 14, 14, 256) 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (None, 14, 14, 256) 590080 activation_51[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 256) 1024 conv2d_55[0][0]
__________________________________________________________________________________________________
activation_52 (Activation) (None, 14, 14, 256) 0 batch_normalization_35[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (None, 14, 14, 1024) 263168 activation_52[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 1024) 4096 conv2d_56[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_10 (Gl (None, 1024) 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_10 (T (None, 1, 1, 1024) 0 global_average_pooling2d_10[0][0]
__________________________________________________________________________________________________
conv2d_57 (Conv2D) (None, 1, 1, 64) 65600 tf_op_layer_strided_slice_10[0][0
__________________________________________________________________________________________________
activation_53 (Activation) (None, 1, 1, 64) 0 conv2d_57[0][0]
__________________________________________________________________________________________________
conv2d_58 (Conv2D) (None, 1, 1, 1024) 66560 activation_53[0][0]
__________________________________________________________________________________________________
activation_54 (Activation) (None, 1, 1, 1024) 0 conv2d_58[0][0]
__________________________________________________________________________________________________
multiply_10 (Multiply) (None, 14, 14, 1024) 0 batch_normalization_36[0][0]
activation_54[0][0]
__________________________________________________________________________________________________
add_10 (Add) (None, 14, 14, 1024) 0 multiply_10[0][0]
activation_50[0][0]
__________________________________________________________________________________________________
activation_55 (Activation) (None, 14, 14, 1024) 0 add_10[0][0]
__________________________________________________________________________________________________
conv2d_59 (Conv2D) (None, 14, 14, 256) 262400 activation_55[0][0]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 14, 14, 256) 1024 conv2d_59[0][0]
__________________________________________________________________________________________________
activation_56 (Activation) (None, 14, 14, 256) 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
conv2d_60 (Conv2D) (None, 14, 14, 256) 590080 activation_56[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 14, 14, 256) 1024 conv2d_60[0][0]
__________________________________________________________________________________________________
activation_57 (Activation) (None, 14, 14, 256) 0 batch_normalization_38[0][0]
__________________________________________________________________________________________________
conv2d_61 (Conv2D) (None, 14, 14, 1024) 263168 activation_57[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 14, 14, 1024) 4096 conv2d_61[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_11 (Gl (None, 1024) 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_11 (T (None, 1, 1, 1024) 0 global_average_pooling2d_11[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D) (None, 1, 1, 64) 65600 tf_op_layer_strided_slice_11[0][0
__________________________________________________________________________________________________
activation_58 (Activation) (None, 1, 1, 64) 0 conv2d_62[0][0]
__________________________________________________________________________________________________
conv2d_63 (Conv2D) (None, 1, 1, 1024) 66560 activation_58[0][0]
__________________________________________________________________________________________________
activation_59 (Activation) (None, 1, 1, 1024) 0 conv2d_63[0][0]
__________________________________________________________________________________________________
multiply_11 (Multiply) (None, 14, 14, 1024) 0 batch_normalization_39[0][0]
activation_59[0][0]
__________________________________________________________________________________________________
add_11 (Add) (None, 14, 14, 1024) 0 multiply_11[0][0]
activation_55[0][0]
__________________________________________________________________________________________________
activation_60 (Activation) (None, 14, 14, 1024) 0 add_11[0][0]
__________________________________________________________________________________________________
conv2d_64 (Conv2D) (None, 14, 14, 256) 262400 activation_60[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 14, 14, 256) 1024 conv2d_64[0][0]
__________________________________________________________________________________________________
activation_61 (Activation) (None, 14, 14, 256) 0 batch_normalization_40[0][0]
__________________________________________________________________________________________________
conv2d_65 (Conv2D) (None, 14, 14, 256) 590080 activation_61[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 14, 14, 256) 1024 conv2d_65[0][0]
__________________________________________________________________________________________________
activation_62 (Activation) (None, 14, 14, 256) 0 batch_normalization_41[0][0]
__________________________________________________________________________________________________
conv2d_66 (Conv2D) (None, 14, 14, 1024) 263168 activation_62[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 14, 14, 1024) 4096 conv2d_66[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_12 (Gl (None, 1024) 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_12 (T (None, 1, 1, 1024) 0 global_average_pooling2d_12[0][0]
__________________________________________________________________________________________________
conv2d_67 (Conv2D) (None, 1, 1, 64) 65600 tf_op_layer_strided_slice_12[0][0
__________________________________________________________________________________________________
activation_63 (Activation) (None, 1, 1, 64) 0 conv2d_67[0][0]
__________________________________________________________________________________________________
conv2d_68 (Conv2D) (None, 1, 1, 1024) 66560 activation_63[0][0]
__________________________________________________________________________________________________
activation_64 (Activation) (None, 1, 1, 1024) 0 conv2d_68[0][0]
__________________________________________________________________________________________________
multiply_12 (Multiply) (None, 14, 14, 1024) 0 batch_normalization_42[0][0]
activation_64[0][0]
__________________________________________________________________________________________________
add_12 (Add) (None, 14, 14, 1024) 0 multiply_12[0][0]
activation_60[0][0]
__________________________________________________________________________________________________
activation_65 (Activation) (None, 14, 14, 1024) 0 add_12[0][0]
__________________________________________________________________________________________________
conv2d_70 (Conv2D) (None, 7, 7, 512) 524800 activation_65[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 7, 7, 512) 2048 conv2d_70[0][0]
__________________________________________________________________________________________________
activation_66 (Activation) (None, 7, 7, 512) 0 batch_normalization_44[0][0]
__________________________________________________________________________________________________
conv2d_71 (Conv2D) (None, 7, 7, 512) 2359808 activation_66[0][0]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 7, 7, 512) 2048 conv2d_71[0][0]
__________________________________________________________________________________________________
activation_67 (Activation) (None, 7, 7, 512) 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
conv2d_72 (Conv2D) (None, 7, 7, 2048) 1050624 activation_67[0][0]
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 7, 7, 2048) 8192 conv2d_72[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_13 (Gl (None, 2048) 0 batch_normalization_46[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_13 (T (None, 1, 1, 2048) 0 global_average_pooling2d_13[0][0]
__________________________________________________________________________________________________
conv2d_73 (Conv2D) (None, 1, 1, 128) 262272 tf_op_layer_strided_slice_13[0][0
__________________________________________________________________________________________________
activation_68 (Activation) (None, 1, 1, 128) 0 conv2d_73[0][0]
__________________________________________________________________________________________________
conv2d_74 (Conv2D) (None, 1, 1, 2048) 264192 activation_68[0][0]
__________________________________________________________________________________________________
activation_69 (Activation) (None, 1, 1, 2048) 0 conv2d_74[0][0]
__________________________________________________________________________________________________
conv2d_69 (Conv2D) (None, 7, 7, 2048) 2099200 activation_65[0][0]
__________________________________________________________________________________________________
multiply_13 (Multiply) (None, 7, 7, 2048) 0 batch_normalization_46[0][0]
activation_69[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 7, 7, 2048) 8192 conv2d_69[0][0]
__________________________________________________________________________________________________
add_13 (Add) (None, 7, 7, 2048) 0 multiply_13[0][0]
batch_normalization_43[0][0]
__________________________________________________________________________________________________
activation_70 (Activation) (None, 7, 7, 2048) 0 add_13[0][0]
__________________________________________________________________________________________________
conv2d_75 (Conv2D) (None, 7, 7, 512) 1049088 activation_70[0][0]
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 7, 7, 512) 2048 conv2d_75[0][0]
__________________________________________________________________________________________________
activation_71 (Activation) (None, 7, 7, 512) 0 batch_normalization_47[0][0]
__________________________________________________________________________________________________
conv2d_76 (Conv2D) (None, 7, 7, 512) 2359808 activation_71[0][0]
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 7, 7, 512) 2048 conv2d_76[0][0]
__________________________________________________________________________________________________
activation_72 (Activation) (None, 7, 7, 512) 0 batch_normalization_48[0][0]
__________________________________________________________________________________________________
conv2d_77 (Conv2D) (None, 7, 7, 2048) 1050624 activation_72[0][0]
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, 7, 7, 2048) 8192 conv2d_77[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_14 (Gl (None, 2048) 0 batch_normalization_49[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_14 (T (None, 1, 1, 2048) 0 global_average_pooling2d_14[0][0]
__________________________________________________________________________________________________
conv2d_78 (Conv2D) (None, 1, 1, 128) 262272 tf_op_layer_strided_slice_14[0][0
__________________________________________________________________________________________________
activation_73 (Activation) (None, 1, 1, 128) 0 conv2d_78[0][0]
__________________________________________________________________________________________________
conv2d_79 (Conv2D) (None, 1, 1, 2048) 264192 activation_73[0][0]
__________________________________________________________________________________________________
activation_74 (Activation) (None, 1, 1, 2048) 0 conv2d_79[0][0]
__________________________________________________________________________________________________
multiply_14 (Multiply) (None, 7, 7, 2048) 0 batch_normalization_49[0][0]
activation_74[0][0]
__________________________________________________________________________________________________
add_14 (Add) (None, 7, 7, 2048) 0 multiply_14[0][0]
activation_70[0][0]
__________________________________________________________________________________________________
activation_75 (Activation) (None, 7, 7, 2048) 0 add_14[0][0]
__________________________________________________________________________________________________
conv2d_80 (Conv2D) (None, 7, 7, 512) 1049088 activation_75[0][0]
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, 7, 7, 512) 2048 conv2d_80[0][0]
__________________________________________________________________________________________________
activation_76 (Activation) (None, 7, 7, 512) 0 batch_normalization_50[0][0]
__________________________________________________________________________________________________
conv2d_81 (Conv2D) (None, 7, 7, 512) 2359808 activation_76[0][0]
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, 7, 7, 512) 2048 conv2d_81[0][0]
__________________________________________________________________________________________________
activation_77 (Activation) (None, 7, 7, 512) 0 batch_normalization_51[0][0]
__________________________________________________________________________________________________
conv2d_82 (Conv2D) (None, 7, 7, 2048) 1050624 activation_77[0][0]
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, 7, 7, 2048) 8192 conv2d_82[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_15 (Gl (None, 2048) 0 batch_normalization_52[0][0]
__________________________________________________________________________________________________
tf_op_layer_strided_slice_15 (T (None, 1, 1, 2048) 0 global_average_pooling2d_15[0][0]
__________________________________________________________________________________________________
conv2d_83 (Conv2D) (None, 1, 1, 128) 262272 tf_op_layer_strided_slice_15[0][0
__________________________________________________________________________________________________
activation_78 (Activation) (None, 1, 1, 128) 0 conv2d_83[0][0]
__________________________________________________________________________________________________
conv2d_84 (Conv2D) (None, 1, 1, 2048) 264192 activation_78[0][0]
__________________________________________________________________________________________________
activation_79 (Activation) (None, 1, 1, 2048) 0 conv2d_84[0][0]
__________________________________________________________________________________________________
multiply_15 (Multiply) (None, 7, 7, 2048) 0 batch_normalization_52[0][0]
activation_79[0][0]
__________________________________________________________________________________________________
add_15 (Add) (None, 7, 7, 2048) 0 multiply_15[0][0]
activation_75[0][0]
__________________________________________________________________________________________________
activation_80 (Activation) (None, 7, 7, 2048) 0 add_15[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_16 (Gl (None, 2048) 0 activation_80[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 17) 34833 global_average_pooling2d_16[0][0]
==================================================================================================
Total params: 26,153,537
Trainable params: 26,100,417
Non-trainable params: 53,120
References
Jie Hu, Li Shen, Samuel Albanie, Gang Sun, & Enhua Wu (2018). Squeeze-and-Excitation Networks computer vision and pattern recognition.Tensorflow2 图像处理中注意力机制的解析与代码详解
