no code implementations • 13 May 2024 • Priyanshu Gupta, Shashank Kirtania, Ananya Singha, Sumit Gulwani, Arjun Radhakrishna, Sherry Shi, Gustavo Soares
Our results demonstrate a notable improvement, with METARELECTION outperforming GPT-4 by 16. 82% (IAC), 31. 33% (COT), and 15. 42% (REACT), underscoring the potential of METAREFLECTION as a viable method for enhancing the efficiency of LLMs.
no code implementations • 26 Oct 2023 • Anirudh Khatry, Sumit Gulwani, Priyanshu Gupta, Vu Le, Ananya Singha, Mukul Singh, Gust Verbruggen
Target similarity tuning (TST) is a method of selecting relevant examples in natural language (NL) to code generation through large language models (LLMs) to improve performance.
no code implementations • 16 Oct 2023 • Ananya Singha, José Cambronero, Sumit Gulwani, Vu Le, Chris Parnin
Inspired by prior work, we generate a collection of self-supervised structural tasks (e. g. navigate to a cell and row; transpose the table) and evaluate the performance differences when using 8 formats.