ResNet-RS

Introduced by Bello et al. in Revisiting ResNets: Improved Training and Scaling Strategies

ResNet-RS is a family of ResNet architectures that are 1.7x faster than EfficientNets on TPUs, while achieving similar accuracies on ImageNet. The authors propose two new scaling strategies: (1) scale model depth in regimes where overfitting can occur (width scaling is preferable otherwise); (2) increase image resolution more slowly than previously recommended.

Additional improvements include the use of a cosine learning rate schedule, label smoothing, stochastic depth, RandAugment, decreased weight decay, squeeze-and-excitation and the use of the ResNet-D architecture.

Source: Revisiting ResNets: Improved Training and Scaling Strategies

Read Paper See Code

Papers

Paper	Code	Results	Date	Stars

Tasks

Task	Papers	Share
Action Classification	2	33.33%
Video Recognition	1	16.67%
Document Image Classification	1	16.67%
Image Classification	1	16.67%
Video Classification	1	16.67%

Usage Over Time

This feature is experimental; we are continuously improving our matching algorithm.

Components

Component	Type	Add Remove
Cosine Annealing	Learning Rate Schedules
Label Smoothing	Regularization
ResNet-D	Convolutional Neural Networks
Squeeze-and-Excitation Block	Image Model Blocks
Stochastic Depth	Regularization
Weight Decay	Regularization

Categories

Add Remove

Convolutional Neural Networks