Search Results for author:
Found 12 papers, 6 papers with code
Spectral regularization for adversarially-robust representation learning
To this end, we propose a new spectral regularizer for representation learning that encourages black-box adversarial robustness in downstream classification tasks.
Asymptotic theory of in-context learning by linear attention
Transformers have a remarkable ability to learn and execute tasks based on examples provided within the input itself, without explicit prior training.
Scaling and renormalization in high-dimensional regression
This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models using the basic tools of random matrix theory and free probability.
Long Sequence Hopfield Memory
Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions.
Neural networks learn to magnify areas near decision boundaries
This holds in deep networks trained on high-dimensional image classification tasks, and even in self-supervised representation learning.
Contrasting random and learned features in deep Bayesian linear regression
Moreover, we show that the leading-order correction to the kernel-limit learning curve cannot distinguish between random feature models and deep networks in which all layers are trained.
On neural network kernels and the storage capacity problem
In this short note, we reify the connection between work on the storage capacity problem in wide two-layer treelike neural networks and the rapidly-growing body of literature on kernel limits of wide neural networks.
Parallel locomotor control strategies in mice and flies
Our understanding of the neural basis of locomotor behavior can be informed by careful quantification of animal movement.
Depth induces scale-averaging in overparameterized linear Bayesian neural networks
Inference in deep Bayesian neural networks is only fully understood in the infinite-width limit, where the posterior flexibility afforded by increased depth washes out and the posterior predictive collapses to a shallow Gaussian process.
Asymptotics of representation learning in finite Bayesian neural networks
However, our theoretical understanding of how the learned hidden layer representations of finite networks differ from the fixed representations of infinite networks remains incomplete.
Exact marginal prior distributions of finite Bayesian neural networks
For deep linear networks, the prior has a simple expression in terms of the Meijer $G$-function.
Activation function dependence of the storage capacity of treelike neural networks
Though a wide variety of nonlinear activation functions have been proposed for use in artificial neural networks, a detailed understanding of their role in determining the expressive power of a network has not emerged.
