Network in Network算法的简介(论文介绍)
Abstract
We propose a novel deep network structure called "Network In Network" (NIN) to enhance model discriminability for local patches within the receptive field. The conventional convolutional layer uses linear filters followed by a nonlinear activation function to scan the input. Instead, we build micro neural networks with more complex structures to abstract the data within the receptive field. We instantiate the micro neural network with a multilayer perceptron, which is a potent function approximator. The feature maps are obtained by sliding the micro networks over the input in a similar manner as CNN; they are then fed into the next layer. Deep NIN can be implemented by stacking mutiple of the above described structure. With enhanced local modeling via the micro network, we are able to utilize global average pooling over feature maps in the classification layer, which is easier to interpret and less prone to overfitting than traditional fully connected layers. We demonstrated the state-of-the-art classification performances with NIN on CIFAR-10 and CIFAR-100, and reasonable performances on SVHN and MNIST datasets.
摘要
我们提出了一种新的深层网络结构，称为“网络中的网络”（NIN），以增强接受域内局部补丁的模型识别能力。传统的卷积层使用线性滤波器和非线性激活函数扫描输入。相反，我们建立了具有更复杂结构的微神经网络来提取接收域内的数据。我们用一个多层感知器来例示微神经网络，这是一个有效的函数逼近器。这些特征图是通过将微型网络滑动到输入端，以类似于CNN的方式获得的；然后将它们输入到下一层。深度NIN可以通过叠加上述结构的多个来实现。通过微网络增强的局部建模，我们能够利用分类层中特征图的全局平均池，这比传统的完全连接层更容易解释，更不容易过度拟合。我们在cifar-10和cifar-100上演示了NIN的最新分类性能，在svhn和mnist数据集上演示了合理的性能。
Conclusions
We proposed a novel deep network called “Network In Network” (NIN) for classification tasks. This new structure consists of mlpconv layers which use multilayer perceptrons to convolve the input and a global average pooling layer as a replacement for the fully connected layers in conventional CNN. Mlpconv layers model the local patches better, and global average pooling acts as a structural regularizer that prevents overfitting globally. With these two components of NIN we demonstrated state-of-the-art performance on CIFAR-10, CIFAR-100 and SVHN datasets. Through visualization of the feature maps, we demonstrated that feature maps from the last mlpconv layer of NIN were confidence maps of the categories, and this motivates the possibility of performing object detection via NIN.
结论
我们提出了一种新的深度网络，称为“网络中的网络”（NIN），用于分类任务。这种新结构由多层感知器卷积输入的mlpconv层和一个全局平均池层组成，以取代传统CNN中的全连接层。Mlpconv层更好地模拟局部小块，而全局平均池充当结构调整器，防止全局过度拟合。通过NIN的这两个组成部分，我们在CIFAR-10、CIFAR-100和SVHN数据集上展示了最先进的性能。通过对特征图的可视化，我们证明了NIN最后一个mlpconv 层的特征图是类别的置信图，这激发了通过NIN进行目标检测的可能性。
Min Lin, QiangChen and ShuichengYan. Network in Network. ICLR, 2014 新加坡国立大学颜水成教授
https://arxiv.org/abs/1312.4400
1、Visualization of NIN
该论文中，还对最后一个mlpconv层特征图的可视化。仅显示特征图中的前10％激活。与特征图对应的类别是：1 飞机，2 汽车，3 鸟，4 猫，5 鹿，6 狗，7 青蛙，8 马，9 船，10 卡车。对应于输入图像的GT的特征图被突出显示。
Network in Network算法的架构详解
1、NIN的引出
传统卷积操作上的一个想法：把卷积操作用一个神经网络去替换，会有什么样的效果呢？
2、NIN的特点
在网络中构建微型网络Mlpconv，它对conv特征进行了组合，提高了卷积的有效性。
对单个像素，1x1卷积等效于该像素点在所有特征上进行一次全连接计算。
Mlpconv中的全连接可以通过1×1 卷积快速实现。
