Search Results for author:
Found 5 papers, 3 papers with code
SGEM: stochastic gradient with energy and momentum
In this paper, we propose SGEM, Stochastic Gradient with Energy and Momentum, to solve a large class of general non-convex stochastic optimization problems, based on the AEGD method that originated in the work [AEGD: Adaptive Gradient Descent with Energy.
An Adaptive Gradient Method with Energy and Momentum
We introduce a novel algorithm for gradient-based optimization of stochastic objective functions.
A global convergence theory for deep ReLU implicit networks via over-parameterization
Implicit deep learning has received increasing attention recently due to the fact that it generalizes the recursive prediction rules of many commonly used neural network architectures.
SGDEM: stochastic gradient descent with energy and momentum
In this paper, we propose SGDEM, Stochastic Gradient Descent with Energy and Momentum to solve a large class of general nonconvex stochastic optimization problems, based on the AEGD method that originated in the work [AEGD: Adaptive Gradient Descent with Energy.
AEGD: Adaptive Gradient Descent with Energy
We propose AEGD, a new algorithm for first-order gradient-based optimization of non-convex objective functions, based on a dynamically updated energy variable.
