Search Results for author:
Found 6 papers, 2 papers with code
Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order
Pretrained language models underpin several AI applications, but their high computational cost for training limits accessibility.
Myths and Legends in High-Performance Computing
While some laws end, new directions are emerging, such as algorithmic scaling or novel architecture research.
Outliers Dimensions that Disrupt Transformers Are Driven by Frequency
While Transformer-based language models are generally very robust to pruning, there is the recently discovered outlier phenomenon: disabling only 48 out of 110M parameters in BERT-base drops its performance by nearly 30% on MNLI.
MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems
Scientific communities are increasingly adopting machine learning and deep learning models in their applications to accelerate scientific insights.
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics.
Scaling Distributed Deep Learning Workloads beyond the Memory Capacity with KARMA
An alternative solution is to use out-of-core methods instead of, or in addition to, data parallelism.
