1 code implementation • 25 Apr 2024 • Harman Singh, Nitish Gupta, Shikhar Bharadwaj, Dinesh Tewari, Partha Talukdar
To facilitate research on multilingual LLM evaluation, we release IndicGenBench - the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set 29 of Indic languages covering 13 scripts and 4 language families.
no code implementations • 23 May 2023 • Harman Singh, Poorva Garg, Mohit Gupta, Kevin Shah, Ashish Goswami, Satyam Modi, Arnab Kumar Mondal, Dinesh Khandelwal, Dinesh Garg, Parag Singla
We are interested in image manipulation via natural language text -- a task that is useful for multiple AI applications but requires complex reasoning over multi-modal spaces.
no code implementations • 23 May 2023 • Harman Singh, Pengchuan Zhang, Qifan Wang, Mengjiao Wang, Wenhan Xiong, Jingfei Du, Yu Chen
Along with this, we propose novel negative mining techniques in the scene graph space for improving attribute binding and relation understanding.
Ranked #1 on Image Retrieval on CREPE (Compositional REPresentation Evaluation) (Recall@1 (HN-Comp, UC) metric)
1 code implementation • ACL 2022 • Soumya Sanyal, Harman Singh, Xiang Ren
Recent works show that such models can also produce the reasoning steps (i. e., the proof graph) that emulate the model's logical reasoning process.
no code implementations • 28 Apr 2021 • Sergey Knyazev, Karishma Chhugani, Varuni Sarwal, Ram Ayyala, Harman Singh, Smruthi Karthikeyan, Dhrithi Deshpande, Zoia Comarova, Angela Lu, Yuri Porozov, Aiping Wu, Malak Abedalthagafi, Shivashankar Nagaraj, Adam Smith, Pavel Skums, Jason Ladner, Tommy Tsan-Yuk Lam, Nicholas Wu, Alex Zelikovsky, Rob Knight, Keith Crandall, Serghei Mangul
More than any other infectious disease epidemic, the COVID-19 pandemic has been characterized by the generation of large volumes of viral genomic data at an incredible pace due to recent advances in high-throughput sequencing technologies, the rapid global spread of SARS-CoV-2, and its persistent threat to public health.