Search Results for author:
Found 6 papers, 2 papers with code
A Margin-based Multiclass Generalization Bound via Geometric Complexity
We derive a new upper bound on the generalization error which scales with the margin-normalized geometric complexity of the network and which holds for a broad family of data distributions and model classes.
The Impact of Geometric Complexity on Neural Collapse in Transfer Learning
Many of the recent remarkable advances in computer vision and language models can be attributed to the success of transfer learning via the pre-training of large foundation models.
Unified Functional Hashing in Automatic Machine Learning
In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence caching technique, which we also present.
Why neural networks find simple solutions: the many regularizers of geometric complexity
Using a combination of theoretical arguments and empirical results, we show that many common training heuristics such as parameter norm regularization, spectral norm regularization, flatness regularization, implicit gradient regularization, noise regularization and the choice of parameter initialization all act to control geometric complexity, providing a unifying framework in which to characterize the behavior of deep learning models.
The Geometric Occam's Razor Implicit in Deep Learning
We argue that over-parameterized neural networks trained with stochastic gradient descent are subject to a Geometric Occam's Razor; that is, these networks are implicitly regularized by the geometric model complexity.
COT-GAN: Generating Sequential Data via Causal Optimal Transport
We introduce COT-GAN, an adversarial algorithm to train implicit generative models optimized for producing sequential data.
