Training a Vision Transformer from scratch in less than 24 hours with 1 GPU

1 code implementation • 9 Nov 2022 • Saghar Irandoust, Thibaut Durand, Yunduz Rakhmangulova, Wenjie Zi, Hossein Hajimirsadeghi

We introduce some algorithmic improvements to enable training a ViT model from scratch with limited hardware (1 GPU) and time (24 hours) resources.