no code implementations • 21 Mar 2023 • Ethan Che, Hongseok Namkoong
Standard bandit algorithms that assume continual reallocation of measurement effort are challenging to implement due to delayed feedback and infrastructural/organizational difficulties.
1 code implementation • 3 Mar 2023 • Tiffany Tianhui Cai, Hongseok Namkoong, Steve Yadlowsky
In order to do this, we define a hypothetical distribution on $X$ consisting of values common in both training and target, over which it is easy to compare $Y \mid X$ and thus predictive performance.
no code implementations • 3 Feb 2023 • Brian Hsu, Xiaotong Chen, Ying Han, Hongseok Namkoong, Kinjal Basu
We demonstrate our framework with a case study on predictive parity.
no code implementations • 13 Dec 2022 • Hongseok Namkoong, Yuanzhe Ma, Peter W. Glynn
The performance of decision policies and prediction models often deteriorates when applied to environments different from the ones seen during training.
5 code implementations • 10 Mar 2022 • Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, Ludwig Schmidt
The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set, discarding the remainder.
Ranked #1 on Image Classification on ImageNet V2 (using extra training data)
no code implementations • NeurIPS 2021 • Mike Li, Hongseok Namkoong, Shangzhou Xia
The performance of ML models degrades when the training population is different from that seen under operation.
3 code implementations • CVPR 2022 • Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo-Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, Ludwig Schmidt
Compared to standard fine-tuning, WiSE-FT provides large accuracy improvements under distribution shift, while preserving high accuracy on the target distribution.
Ranked #12 on Image Classification on ObjectNet (using extra training data)
no code implementations • 29 Nov 2020 • Hongseok Namkoong, Samuel Daulton, Eytan Bakshy
We propose a novel imitation-learning-based algorithm that distills a TS policy into an explicit policy representation by performing posterior inference and optimization offline.
1 code implementation • 28 Jul 2020 • John Duchi, Tatsunori Hashimoto, Hongseok Namkoong
While modern large-scale datasets often consist of heterogeneous subpopulations -- for example, multiple demographic groups or multiple text corpora -- the standard practice of minimizing average loss fails to guarantee uniformly low losses across all subpopulations.
1 code implementation • 5 Jul 2020 • Sookyo Jeong, Hongseok Namkoong
Study populations are typically sampled from limited points in space and time, and marginalized groups are underrepresented.
1 code implementation • NeurIPS 2020 • Hongseok Namkoong, Ramtin Keramati, Steve Yadlowsky, Emma Brunskill
We assess robustness of OPE methods under unobserved confounding by developing worst-case bounds on the performance of an evaluation policy.
no code implementations • 2 Dec 2018 • Matthew O'Kelly, Aman Sinha, Justin Norden, Hongseok Namkoong
Modern treatments for Type 1 diabetes (T1D) use devices known as artificial pancreata (APs), which combine an insulin pump with a continuous glucose monitor (CGM) operating in a closed-loop manner to control blood glucose levels.
1 code implementation • NeurIPS 2018 • Matthew O'Kelly, Aman Sinha, Hongseok Namkoong, John Duchi, Russ Tedrake
While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing.
no code implementations • 20 Oct 2018 • John Duchi, Hongseok Namkoong
A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts, or unmodeled temporal effects.
1 code implementation • ICML 2018 • Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, Percy Liang
Machine learning models (e. g., speech recognizers) are usually trained to minimize average loss, which results in representation disparity---minority groups (e. g., non-native speakers) contribute less to the training objective and thus tend to suffer higher loss.
2 code implementations • NeurIPS 2018 • Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John Duchi, Vittorio Murino, Silvio Savarese
Only using training data from a single source distribution, we propose an iterative procedure that augments the dataset with examples from a fictitious target domain that is "hard" under the current model.
1 code implementation • ICLR 2018 • Aman Sinha, Hongseok Namkoong, Riccardo Volpi, John Duchi
Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms.
no code implementations • ICML 2017 • Hongseok Namkoong, Aman Sinha, Steve Yadlowsky, John C. Duchi
Standard forms of coordinate and stochastic gradient methods do not adapt to structure in data; their good behavior under random sampling is predicated on uniformity in data.
no code implementations • NeurIPS 2016 • Hongseok Namkoong, John C. Duchi
We develop efficient solution methods for a robust empirical risk minimization problem designed to give calibrated confidence intervals on performance and provide optimal tradeoffs between bias and variance.
no code implementations • 11 Oct 2016 • John Duchi, Peter Glynn, Hongseok Namkoong
We study statistical inference and distributionally robust solution methods for stochastic optimization problems, focusing on confidence intervals for optimal values and solutions that achieve exact coverage asymptotically.
1 code implementation • NeurIPS 2017 • John Duchi, Hongseok Namkoong
We develop an approach to risk minimization and stochastic optimization that provides a convex surrogate for variance, allowing near-optimal and computationally efficient trading between approximation and estimation error.