轻量级网络总结

1. SqueezeNet

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size

考虑到卷积层的参数量为$C_{in}\ast C_{out}\ast K\ast K$，显然，为了减少参数量，可以从输入输出通道数和卷积核尺寸两个角度出发：

• 使用更多的1×1卷积
• 减小3×3卷积的输入核输出通道数

根据以上两个基本原则，SqueezeNet中设计了一个Fire module，squeeze中只使用了1×1卷积，以较少的参数量为代价降低通道数，然后在expand中使用1×1和3×3扩展通道数。

class Fire(nn.Module):

    def __init__(self, inplanes, squeeze_planes,
                 expand1x1_planes, expand3x3_planes):
        super(Fire, self).__init__()
        self.inplanes = inplanes
        self.squeeze = nn.Conv2d(inplanes, squeeze_planes, kernel_size=1)
        self.squeeze_activation = nn.ReLU(inplace=True)
        self.expand1x1 = nn.Conv2d(squeeze_planes, expand1x1_planes,
                                   kernel_size=1)
        self.expand1x1_activation = nn.ReLU(inplace=True)
        self.expand3x3 = nn.Conv2d(squeeze_planes, expand3x3_planes,
                                   kernel_size=3, padding=1)
        self.expand3x3_activation = nn.ReLU(inplace=True)

    def forward(self, x):
        x = self.squeeze_activation(self.squeeze(x))
        return torch.cat(