ResNet算法的简介
来自微软研究院何恺明等 ，荣获ILSVRC2015的分类任务第一名、CVPR 2016 best paper 。ResNet使得训练深度达数百甚至数千层的网络成为可能，而且性能仍然优异，是深度学习算法中，一个里程碑式的网络。
Abstract
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [41] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers.
The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1 , where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
摘要
更深层次的神经网络更难训练。我们提出了一个残差学习框架来简化网络的训练，这些网络比以前使用的网络要深入得多。我们显式地将层重新表示为参考层输入的学习剩余函数，而不是学习未引用的函数。我们提供了全面的经验证据表明，这些剩余网络更容易优化，并可以从大幅增加的深度获得精度。在ImageNet数据集上，我们评估了高达152层的residual网络—比VGG网络[41]深8倍，但仍然具有较低的复杂性。这些残差网的集合在ImageNet测试集上的误差达到3.57%，该结果在ILSVRC 2015年分类任务中获得第一名。我们还对CIFAR-10进行了100层和1000层的分析。
在许多视觉识别任务中，表征的深度是至关重要的。仅仅由于我们的深度表示，我们获得了28%的相对改进的COCO对象检测数据集。深度残差网是我们参加ILSVRC & COCO 2015竞赛的基础，并在ImageNet检测、ImageNet定位、COCO检测、COCO分割等方面获得第一名。
1、比赛结果-ResNets @ ILSVRC & COCO 2015 Competitions
1st places in all five main tracks 五大tracks的第一名，并且大都远远超出第二名！
ImageNet Classification: “Ultra-deep” 152-layer nets
ImageNet分类问题：“超深”152层网
ImageNet Detection: 16% better than 2nd
ImageNet检测问题：比第二名高16%
ImageNet Localization: 27% better than 2nd
ImageNet定位问题：比第二名好27%
COCO Detection: 11% better than 2nd
COCO检测问题：比第二名好11%
COCO Segmentation: 12% better than 2nd
COCO分割问题：比第二名好12%
2、ResNet的深度革命
ResNet采用了很深的152层的网络，准确度脱颖而出，甚至比人的识别率还要高，比排行第二的GoogleNet网络准确度超出很多！
论文
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun（2015）: Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs] (December 2015).
Kaiming He, XiangyuZhang, ShaoqingRen, & Jian Sun.
“Deep Residual Learning for Image Recognition”. CVPR 2016（best paper award）.
https://arxiv.org/abs/1512.03385
1、残差模块
增加了”短路”连接（shortcut connection)或称为跳跃连接（skip connection)
学习残差映射（residual mapping）而不是直接学习期望映射
瓶颈残差块，网络较深（大于50层）时使用后面这种（bottleneck)来提高效率
2、网络架构
plain network：基于VGG19的架构把网络增加到34层
Residual Network：plain network基础上增加残差模块
深度变化：34、50、101、152
3、实验结果
单个模型：top-5错误率为4.49%
ensemble：top-5错误率为3.57%
ResNet算法的架构详解
DL之ResNet：ResNet算法的架构详解
ResNet：方块对应【3*3】的卷积层，其特征在于引入了横跨层的快捷结构。
