no code implementations • 16 Nov 2023 • Chung-Ching Chang, William W. Cohen, Yun-Hsuan Sung
We propose a theoretical framework for formulating language model decoder algorithms with dynamic programming and information theory.
no code implementations • 15 Nov 2023 • Cicero Nogueira dos santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David Uthus
We demonstrate that MoWE performs significantly better than the T5 family of models with similar number of FLOPs in a variety of NLP tasks.
no code implementations • 13 Nov 2023 • Abdullatif Köksal, Renat Aksitov, Chung-Ching Chang
For open book QA as a case study, we demonstrate that models finetuned with our counterfactual datasets improve text grounding, leading to better open book QA performance, with up to an 8. 0% increase in F1 score.
2 code implementations • 2 Jun 2023 • Chung-Ching Chang, David Reitter, Renat Aksitov, Yun-Hsuan Sung
One common approach to mitigate hallucinations is to provide source/grounding documents and the model is trained to produce predictions that bind to and are attributable to the provided source.
no code implementations • 11 Feb 2023 • Renat Aksitov, Chung-Ching Chang, David Reitter, Siamak Shakeri, YunHsuan Sung
One common solution to this is augmenting LLMs with a retrieval system and making sure that the generated output is attributable to the retrieved information.
no code implementations • 25 May 2022 • Dheeraj Rajagopal, Siamak Shakeri, Cicero Nogueira dos santos, Eduard Hovy, Chung-Ching Chang
Abstractive summarization systems based on pretrained language models often generate coherent but factually inconsistent sentences.
2 code implementations • 20 Jan 2022 • Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, Yaguang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, Quoc Le
We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding.
Ranked #114 on Code Generation on HumanEval