1.FasterNet简介
论文发表时间：2023.05.21
github地址：https://github.com/JierunChen/FasterNet.
paper地址：https://arxiv.org/abs/2303.03667
摘要：为了设计快速的神经网络，许多研究都专注于减少浮点运算量（FLOPs）。然而，我们观察到这种FLOPs的减少并不一定导致延迟的相似程度的减少。这主要是因为浮点操作每秒（FLOPS）效率低下。为了实现更快的网络，我们重新审视了流行的操作符，并证明了低FLOPS主要是由于操作符的频繁内存访问，特别是深度卷积。因此，我们提出了一种新颖的部分卷积（PConv），通过同时减少冗余的计算和内存访问来更高效地提取空间特征。在我们的PConv基础上，我们进一步提出了FasterNet，这是一系列新的神经网络，比其他网络在各种视觉任务上实现了更高的运行速度，同时对准确性没有妥协。例如，在ImageNet-1k上，我们小巧的FasterNet-T0在GPU、CPU和ARM处理器上分别比MobileViT-XXS快2.8倍、3.3倍和2.4倍，同时准确率更高2.9%。我们的大型FasterNet-L在GPU上实现了令人印象深刻的83.5%的Top-1准确率，与新兴的Swin-B不相上下，同时在GPU上推理吞吐量比Swin-B高出36%，在CPU上节省了37%的计算时间。
论文主要亮点如下：
• 我们指出在实现更快的神经网络时，实现更高的FLOPS比仅仅减少FLOPS更重要。
• 我们引入了一种简单而快速有效的操作符，称为PConv，它有很高的潜力可以取代现有的首选选择DWConv。
• 我们引入了FasterNet，在各种设备上都表现出非常快的运行速度，如GPU、CPU和ARM处理器。
• 我们对各种任务进行了广泛的实验证明了我们的PConv和FasterNet的高速和有效性。
1.1 FasterNet网络结构
1.2 性能对比
2.使用FasterBlock替换C2f中的Bottleneck
替换位置与替换后网络结构示意
C2f中替换的位置
替换后的YOLOv8网络结构如下：
定义C2f_Faster类
在ultralytics/nn/modules/block.py中添加如下代码块，并定义C2f_Faster类：
from timm.models.layers import DropPath class Partial_conv3(nn.Module): def __init__(self, dim, n_div=4, forward='split_cat'): super().__init__() self.dim_conv3 = dim // n_div self.dim_untouched = dim - self.dim_conv3 self.partial_conv3 = nn.Conv2d(self.dim_conv3, self.dim_conv3, 3, 1, 1, bias=False) if forward == 'slicing': self.forward = self.forward_slicing elif forward == 'split_cat': self.forward = self.forward_split_cat else: raise NotImplementedError def forward_slicing(self, x): # only for inference x = x.clone() # !!! Keep the original input intact for the residual connection later x[:, :self.dim_conv3, :, :] = self.partial_conv3(x[:, :self.dim_conv3, :, :]) return x def forward_split_cat(self, x): # for training/inference x1, x2 = torch.split(x, [self.dim_conv3, self.dim_untouched], dim=1) x1 = self.partial_conv3(x1) x = torch.cat((x1, x2), 1) return x class Faster_Block(nn.Module): def __init__(self, inc, dim, n_div=4, mlp_ratio=2, drop_path=0.1, layer_scale_init_value=0.0, pconv_fw_type='split_cat' ): super().__init__() self.dim = dim self.mlp_ratio = mlp_ratio self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity() self.n_div = n_div mlp_hidden_dim = int(dim * mlp_ratio) mlp_layer = [ Conv(dim, mlp_hidden_dim, 1), nn.Conv2d(mlp_hidden_dim, dim, 1, bias=False) ] self.mlp = nn.Sequential(*mlp_layer) self.spatial_mixing = Partial_conv3( dim, n_div, pconv_fw_type ) self.adjust_channel = None if inc != dim: self.adjust_channel = Conv(inc, dim, 1) if layer_scale_init_value > 0: self.layer_scale = nn.Parameter(layer_scale_init_value * torch.ones((dim)), requires_grad=True) self.forward = self.forward_layer_scale else: self.forward = self.forward def forward(self, x): if self.adjust_channel is not None: x = self.adjust_channel(x) shortcut = x x = self.spatial_mixing(x) x = shortcut + self.drop_path(self.mlp(x)) return x def forward_layer_scale(self, x): shortcut = x x = self.spatial_mixing(x) x = shortcut + self.drop_path( self.layer_scale.unsqueeze(-1).unsqueeze(-1) * self.mlp(x)) return x class C2f_Faster(C2f): def __init__(self, c1, c2, n=1, shortcut=False, g=1, e=0.5): super().__init__(c1, c2, n, shortcut, g, e) self.m = nn.ModuleList(Faster_Block(self.c, self.c) for _ in range(n))
在不同文件导入新建的C2f类
在ultralytics/nn/modules/block.py顶部，all中添加刚才创建的类的名称：C2f_Faster,如下图所示：
同样需要在ultralytics/nn/modules/__init__.py文件，相应位置导入刚出创建的C2f_Faster类。如下图：
还需要在ultralytics/nn/tasks.py中导入创建的C2f_Faster类，如下图：
在parse_model解析函数中添加C2f_Faster类
在ultralytics/nn/tasks.py的parse_model解析网络结构的函数中，加入C2f_Faster类，如下图：
创建新的配置文件c2f_Faster_yolov8.yaml
在ultralytics/cfg/models/v8目录下新建c2f_Faster_yolov8.yaml配置文件，内容如下:
# Ultralytics YOLO 🚀, AGPL-3.0 license # YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect # Parameters nc: 80 # number of classes scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n' # [depth, width, max_channels] n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs # YOLOv8.0n backbone backbone: # [from, repeats, module, args] - [-1, 1, Conv, [64, 3, 2]] # 0-P1/2 - [-1, 1, Conv, [128, 3, 2]] # 1-P2/4 - [-1, 3, C2f_Faster, [128, True]] - [-1, 1, Conv, [256, 3, 2]] # 3-P3/8 - [-1, 6, C2f_Faster, [256, True]] - [-1, 1, Conv, [512, 3, 2]] # 5-P4/16 - [-1, 6, C2f_Faster, [512, True]] - [-1, 1, Conv, [1024, 3, 2]] # 7-P5/32 - [-1, 3, C2f_Faster, [1024, True]] - [-1, 1, SPPF, [1024, 5]] # 9 # YOLOv8.0n head head: - [-1, 1, nn.Upsample, [None, 2, 'nearest']] - [[-1, 6], 1, Concat, [1]] # cat backbone P4 - [-1, 3, C2f_Faster, [512]] # 12 - [-1, 1, nn.Upsample, [None, 2, 'nearest']] - [[-1, 4], 1, Concat, [1]] # cat backbone P3 - [-1, 3, C2f_Faster, [256]] # 15 (P3/8-small) - [-1, 1, Conv, [256, 3, 2]] - [[-1, 12], 1, Concat, [1]] # cat head P4 - [-1, 3, C2f_Faster, [512]] # 18 (P4/16-medium) - [-1, 1, Conv, [512, 3, 2]] - [[-1, 9], 1, Concat, [1]] # cat head P5 - [-1, 3, C2f_Faster, [1024]] # 21 (P5/32-large) - [[15, 18, 21], 1, Detect, [nc]] # Detect(P3, P4, P5)
新的c2f_Faster_yolov8.yaml配置文件与原yolov8.yaml文件的对比如下：
加载配置文件并训练
加载c2f_Faster_yolov8.yaml配置文件，并运行train.py训练代码:
#coding:utf-8 from ultralytics import YOLO if __name__ == '__main__': model = YOLO('ultralytics/cfg/models/v8/c2f_Faster_yolov8.yaml') model.load('yolov8n.pt') # loading pretrain weights model.train(data='datasets/TomatoData/data.yaml', epochs=250, batch=4)
注意观察，打印出的网络结构是否正常修改，如下图所示：
模型推理
模型训练完成后，我们使用训练好的模型对图片进行检测：
#coding:utf-8 from ultralytics import YOLO import cv2 # 所需加载的模型目录 # path = 'models/best2.pt' path = 'runs/detect/train2/weights/best.pt' # 需要检测的图片地址 img_path = "TestFiles/Riped tomato_31.jpeg" # 加载预训练模型 # conf 0.25 object confidence threshold for detection # iou 0.7 intersection over union (IoU) threshold for NMS model = YOLO(path, task='detect') # 检测图片 results = model(img_path) res = results[0].plot() # res = cv2.resize(res,dsize=None,fx=2,fy=2,interpolation=cv2.INTER_LINEAR) cv2.imshow("YOLOv8 Detection", res) cv2.waitKey(0)
