Search Results for author:
Found 15 papers, 8 papers with code
General Cyclical Training of Neural Networks
This paper describes the principle of "General Cyclical Training" in machine learning, where training starts and ends with "easy training" and the "hard training" happens during the middle epochs.
Cyclical Focal Loss
In this paper, we introduce a novel cyclical focal loss and demonstrate that it is a more universal loss function than cross-entropy softmax loss or focal loss.
FROST: Faster and more Robust One-shot Semi-supervised Training
Specifically, we show that by combining semi-supervised learning with a one-stage, single network version of self-training, our FROST methodology trains faster and is more robust to choices for the labeled samples and changes in hyper-parameters.
Building One-Shot Semi-supervised (BOSS) Learning up to Fully Supervised Performance
Reaching the performance of fully supervised learning with unlabeled data and only labeling one sample per class might be ideal for deep learning applications.
Empirical Perspectives on One-Shot Semi-supervised Learning
One of the greatest obstacles in the adoption of deep neural networks for new applications is that training the network typically requires a large number of manually labeled training samples.
A Useful Taxonomy for Adversarial Robustness of Neural Networks
In addition, there are several papers in the literature of adversarial defenses that claim there is a cost for adversarial robustness, or a trade-off between robustness and accuracy but, under this proposed taxonomy, we hypothesis that this is not universal.
A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
Although deep learning has produced dazzling successes for applications of image, speech, and video processing in the past few years, most trainings are with suboptimal hyper-parameters, requiring unnecessarily long training times.
Super-Convergence: Very Fast Training of Residual Networks Using Large Learning Rates
In this paper, we show a phenomenon, which we named ``super-convergence'', where residual networks can be trained using an order of magnitude fewer iterations than is used with standard training methods.
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
One of the key elements of super-convergence is training with one learning rate cycle and a large maximum learning rate.
Best Practices for Applying Deep Learning to Novel Applications
This report is targeted to groups who are subject matter experts in their application but deep learning novices.
Exploring loss function topology with cyclical learning rates
We present observations and discussion of previously unreported phenomena discovered while training residual networks.
Deep Convolutional Neural Network Design Patterns
Recent research in the deep learning field has produced a plethora of new architectures.
Gradual DropIn of Layers to Train Very Deep Neural Networks
In particular, an untrainable deep network starts as a trainable shallow network and newly added layers are slowly, organically added during training, thereby increasing the network's depth.
Cyclical Learning Rates for Training Neural Networks
This paper describes a new method for setting the learning rate, named cyclical learning rates, which practically eliminates the need to experimentally find the best values and schedule for the global learning rates.
How to find real-world applications for compressive sensing
The potential of compressive sensing (CS) has spurred great interest in the research community and is a fast growing area of research.
