[DLHacks]Squeeze-and-Excitation Networks

Squeeze-and-Excitation Networks M2

• Title: Squeeze-and-Excitation Networks • Auther: Jie Hu, Li Shen, Gang Sun • ILSVRC 2017 image classification • CVPR 2018 Oral

• https://arxiv.org/search?searchtype=author&query=Hu, J
• https://arxiv.org/search?searchtype=author&query=Shen, L
• https://arxiv.org/search?searchtype=author&query=Sun, G

• CV CNN • • • Inception architectures [ICML’15, S.Ioffe et al] • • • [CVPR’16, S.Bell et al] [NIPS’16, M.Jaderverg et al] • •

Squeeze-and-Excitation Networks SE block • • Squeeze • • Excitation 2

Squeeze: Global Information Embedding • • • filter global average pooling →

Excitation: Adaptive Recalibration • •2 • fc • fc – ReLU – fc – Sigmoid • (bottleneck) channel : C -> C/r -> C (r=16) U Rescaling

224 x 224 image • • Forward (GFLOPS) backward(GPU) (ms) inference(cpu) ResNet-50 ~3.86 190 164 SE-ResNet-50 ~3.87 209 167 • ResNet-50 , 10%↑ (+ 2.5 million parameters) • SE Block • 4%↑, top1 accuracy 0.1%

• ImageNet Classification • • •

• Pytorch • https://github.com/moskomule/senet.pytorch • v0.4 • Cifar10 • Google Colaboratory

• https://github.com/moskomule/senet.pytorch

SE block • • • • •