Search Results for author:
Found 3 papers, 2 papers with code
Survey Descent: A Multipoint Generalization of Gradient Descent for Nonsmooth Optimization
For strongly convex objectives that are smooth, the classical theory of gradient descent ensures linear convergence relative to the number of gradient evaluations.
Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path
The analytically-tractable MSE loss offers more mathematical opportunities than the hard-to-analyze CE loss, inspiring us to leverage MSE loss towards the theoretical investigation of NC.
Prevalence of Neural Collapse during the terminal phase of deep learning training
Modern practice for training classification deepnets involves a Terminal Phase of Training (TPT), which begins at the epoch where training error first vanishes; During TPT, the training error stays effectively zero while training loss is pushed towards zero.
