Search Results for author:
Found 8 papers, 0 papers with code
Interplay between depth of neural networks and locality of target functions
It has been recognized that heavily overparameterized deep neural networks (DNNs) exhibit surprisingly good generalization performance in various machine-learning tasks.
Logarithmic landscape and power-law escape rate of SGD
Stochastic gradient descent (SGD) undergoes complicated multiplicative noise for the mean-square loss.
Power-law escape rate of SGD
Stochastic gradient descent (SGD) undergoes complicated multiplicative noise for the mean-square loss.
Metastability associated with many-body explosion of eigenmode expansion coefficients
Metastable states in stochastic systems are often characterized by the presence of small eigenvalues in the generator of the stochastic dynamics.
Statistical Mechanics Disordered Systems and Neural Networks
Strength of Minibatch Noise in SGD
The noise in stochastic gradient descent (SGD), caused by minibatch sampling, is poorly understood despite its practical importance in deep learning.
Rigorous Bounds on the Heating Rate in Thue-Morse Quasiperiodically and Randomly Driven Quantum Many-Body Systems
The nonequilibrium quantum dynamics of closed many-body systems is a rich yet challenging field.
Statistical Mechanics Quantum Physics
Improved generalization by noise enhancement
Recent studies have demonstrated that noise in stochastic gradient descent (SGD) is closely related to generalization: A larger SGD noise, if not too large, results in better generalization.
Is deeper better? It depends on locality of relevant features
It is shown that the NTK does not correctly capture the depth dependence of the generalization performance, which indicates the importance of the feature learning rather than the lazy learning.
