A Literature Review and Framework for Human Evaluation of Generative Large Language Models in Healthcare

no code implementations • 4 May 2024 • Thomas Yu CHow Tam, Sonish Sivarajkumar, Sumit Kapoor, Alisa V Stolyar, Katelyn Polanska, Karleigh R McCarthy, Hunter Osterhoudt, Xizhi Wu, Shyam Visweswaran, Sunyang Fu, Piyush Mathur, Giovanni E. Cacciamani, Cong Sun, Yifan Peng, Yanshan Wang

This review provides a comprehensive overview of the human evaluation approaches used in diverse healthcare applications. This analysis examines the human evaluation of LLMs across various medical specialties, addressing factors such as evaluation dimensions, sample types, and sizes, the selection and recruitment of evaluators, frameworks and metrics, the evaluation process, and statistical analysis of the results.

Multimodal Machine Learning for Automated ICD Coding

no code implementations • 31 Oct 2018 • Keyang Xu, Mike Lam, Jingzhi Pang, Xin Gao, Charlotte Band, Piyush Mathur, Frank Papay, Ashish K. Khanna, Jacek B. Cywinski, Kamal Maheshwari, Pengtao Xie, Eric Xing

This study presents a multimodal machine learning model to predict ICD-10 diagnostic codes.

BIG-bench Machine Learning