1 code implementation • 14 Feb 2022 • Mark Tuddenham, Adam Prügel-Bennett, Jonathan Hare
The optimisation of neural networks can be sped up by orthogonalising the gradients before the optimisation step, ensuring the diversification of the learned representations.
no code implementations • 29 Sep 2021 • Mark Tuddenham, Adam Prugel-Bennett, Jonathon Hare
The optimisation of neural networks can be sped up by orthogonalising the gradients before the optimisation step, ensuring the diversification of the learned representations.
no code implementations • 28 Nov 2020 • Mark Tuddenham, Adam Prügel-Bennett, Jonathan Hare
Classification problems using deep learning have been shown to have a high-curvature subspace in the loss landscape equal in dimension to the number of classes.