no code implementations • 18 Feb 2021 • Bradley T. Baker, Aashis Khanal, Vince D. Calhoun, Barak Pearlmutter, Sergey M. Plis
We introduce an innovative, communication-friendly approach for training distributed DNNs, which capitalizes on the outer-product structure of the gradient as revealed by the mechanics of auto-differentiation.