Search Results for author:
Found 11 papers, 5 papers with code
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
Building on the recent progress in open-source LLMs, our proposed prompting novelty, and some brute-force scaling, we construct OpenMathInstruct-1, a math instruction tuning dataset with 1. 8M problem-solution pairs.
Ranked #1 on
Math Word Problem Solving
on MAWPS
(using extra training data)
Confidence-based Ensembles of End-to-End Speech Recognition Models
Second, we demonstrate that it is possible to combine base and adapted models to achieve strong results on both original and target data.
Powerful and Extensible WFST Framework for RNN-Transducer Losses
This paper presents a framework based on Weighted Finite-State Transducers (WFST) to simplify the development of modifications for RNN-Transducer (RNN-T) loss.
Understanding the Role of Momentum in Stochastic Gradient Methods
The use of momentum in stochastic gradient methods has become a widespread practice in machine learning.
OpenSeq2Seq: Extensible Toolkit for Distributed and Mixed Precision Training of Sequence-to-Sequence Models
We present OpenSeq2Seq {--} an open-source toolkit for training sequence-to-sequence models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq
We present OpenSeq2Seq - a TensorFlow-based toolkit for training sequence-to-sequence models that features distributed and mixed-precision training.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Novel Prediction Techniques Based on Clusterwise Linear Regression
In this paper we propose two novel approaches on how to solve this problem.
Convergence Analysis of Gradient Descent Algorithms with Proportional Updates
The basic idea of both of these algorithms is to make each step of the gradient descent proportional to the current weight norm and independent of the gradient magnitude.
Large Batch Training of Convolutional Networks with Layer-wise Adaptive Rate Scaling
Using LARS, we scaled AlexNet and ResNet-50 to a batch size of 16K.
Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification
However, it is not clear if these algorithms could replace BN in practical, large-scale applications.
Large Batch Training of Convolutional Networks
Using LARS, we scaled Alexnet up to a batch size of 8K, and Resnet-50 to a batch size of 32K without loss in accuracy.
