no code implementations • 6 Feb 2024 • Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, Sanmi Koyejo
With sufficient alignment, both downstream cross-entropy and BLEU score improve monotonically with more pretraining data.