no code implementations • 2 Mar 2024 • Heling Zhang, Lillian J. Ratliff, Roy Dong
Our approach finds the open-loop control law that minimizes the worst-case loss, given that the noise induced by this control lies in its $(1 - p)$-confidence set for a predetermined $p$.
no code implementations • 19 Dec 2023 • Avinandan Bose, Mihaela Curmei, Daniel L. Jiang, Jamie Morgenstern, Sarah Dean, Lillian J. Ratliff, Maryam Fazel
(ii) Suboptimal Local Solutions: The total loss (sum of loss functions across all users and all services) landscape is not convex even if the individual losses on a single service are convex, making it likely for the learning dynamics to get stuck in local minima.
1 code implementation • 25 Oct 2023 • Arnab Maiti, Ross Boczar, Kevin Jamieson, Lillian J. Ratliff
We design a near-optimal algorithm whose sample complexity matches the lower bound, up to log factors.
1 code implementation • 22 Jun 2023 • Arnab Maiti, Kevin Jamieson, Lillian J. Ratliff
If the row player uses the EXP3 strategy, an algorithm known for obtaining $\sqrt{T}$ regret against an arbitrary sequence of rewards, it is immediate that the row player also achieves $\sqrt{T}$ regret relative to the Nash equilibrium in this game setting.
no code implementations • 4 May 2023 • Boling Yang, Liyuan Zheng, Lillian J. Ratliff, Byron Boots, Joshua R. Smith
Autocurricular training is an important sub-area of multi-agent reinforcement learning~(MARL) that allows multiple agents to learn emergent skills in an unsupervised co-evolving scheme.
no code implementations • 1 May 2023 • Benjamin J. Chasnov, Lillian J. Ratliff, Samuel A. Burden
Our algorithms enable the machine to select the outcome of the co-adaptive interaction from a constellation of game-theoretic equilibria in action and policy spaces.
no code implementations • 19 Mar 2023 • Arnab Maiti, Kevin Jamieson, Lillian J. Ratliff
We study the sample complexity of identifying an approximate equilibrium for two-player zero-sum $n\times 2$ matrix games.
2 code implementations • 6 Jun 2022 • Sarah Dean, Mihaela Curmei, Lillian J. Ratliff, Jamie Morgenstern, Maryam Fazel
Numerous online services are data-driven: the behavior of users affects the system's parameters, and the system's parameters affect the users' experience of the service, which in turn affects the way users may interact with the system.
no code implementations • 5 May 2022 • Sarah H. Q. Li, Lillian J. Ratliff, Peeyush Kumar
Inspired by applications such as supply chain management, epidemics, and social networks, we formulate a stochastic game model that addresses three key features common across these domains: 1) network-structured player interactions, 2) pair-wise mixed cooperation and competition among players, and 3) limited global information toward individual decision-making.
no code implementations • 8 Apr 2022 • Mitas Ray, Dmitriy Drusvyatskiy, Maryam Fazel, Lillian J. Ratliff
This paper studies the problem of expected loss minimization given a data distribution that is dependent on the decision-maker's action and evolves dynamically in time according to a geometric decay process.
no code implementations • 10 Jan 2022 • Adhyyan Narang, Evan Faulkner, Dmitriy Drusvyatskiy, Maryam Fazel, Lillian J. Ratliff
We show that under mild assumptions, the performatively stable equilibria can be found efficiently by a variety of algorithms, including repeated retraining and the repeated (stochastic) gradient method.
1 code implementation • 25 Sep 2021 • Liyuan Zheng, Tanner Fiez, Zane Alumbaugh, Benjamin Chasnov, Lillian J. Ratliff
The hierarchical interaction between the actor and critic in actor-critic based reinforcement learning algorithms naturally lends itself to a game-theoretic interpretation.
no code implementations • 30 Jun 2021 • Roy Dong, Heling Zhang, Lillian J. Ratliff
As data-driven methods are deployed in real-world settings, the processes that generate the observed data will often react to the decisions of the learner.
no code implementations • 16 Jun 2021 • Chinmay Maheshwari, Chih-Yuan Chiu, Eric Mazumdar, S. Shankar Sastry, Lillian J. Ratliff
Min-max optimization is emerging as a key framework for analyzing problems of robustness to strategically and adversarially generated data.
1 code implementation • ICLR 2022 • Tanner Fiez, Chi Jin, Praneeth Netrapalli, Lillian J. Ratliff
This paper considers minimax optimization $\min_x \max_y f(x, y)$ in the challenging setting where $f$ can be both nonconvex in $x$ and nonconcave in $y$.
no code implementations • 23 Dec 2020 • Mitas Ray, Omid Sadeghi, Lillian J. Ratliff, Maryam Fazel
We study the problem of online resource allocation, where multiple customers arrive sequentially and the seller must irrevocably allocate resources to each incoming customer while also facing a procurement cost for the total allocation.
1 code implementation • 20 Mar 2020 • Liyuan Zheng, Yuanyuan Shi, Lillian J. Ratliff, Baosen Zhang
This paper focuses on finding reinforcement learning policies for control systems with hard state and action constraints.
no code implementations • 26 Jan 2020 • Liyuan Zheng, Lillian J. Ratliff
Constrained Markov Decision Processes are a class of stochastic decision problems in which the decision maker must select a policy that satisfies auxiliary cost constraints.
no code implementations • 8 Jul 2019 • Eric Mazumdar, Lillian J. Ratliff, Michael. I. Jordan, S. Shankar Sastry
In such games the state and action spaces are continuous and global Nash equilibria can be found be solving coupled Ricatti equations.
1 code implementation • 4 Jun 2019 • Tanner Fiez, Benjamin Chasnov, Lillian J. Ratliff
Using this insight, we develop a gradient-based update for the leader while the follower employs a best response strategy for which each stable critical point is guaranteed to be a Stackelberg equilibrium in zero-sum games.
no code implementations • 30 May 2019 • Benjamin Chasnov, Lillian J. Ratliff, Eric Mazumdar, Samuel A. Burden
Considering a class of gradient-based multi-agent learning algorithms in non-cooperative settings, we provide local convergence guarantees to a neighborhood of a stable local Nash equilibrium.
no code implementations • 29 Apr 2019 • Tyler Westenbroek, Roy Dong, Lillian J. Ratliff, S. Shankar Sastry
Recent work has explored mechanisms to ensure that the data sources share high quality data with a single data aggregator, addressing the issue of moral hazard.
no code implementations • 6 Jul 2018 • Tanner Fiez, Shreyas Sekar, Liyuan Zheng, Lillian J. Ratliff
The design of personalized incentives or recommendations to improve user engagement is gaining prominence as digital platform providers continually emerge.
no code implementations • 16 Apr 2018 • Eric Mazumdar, Lillian J. Ratliff, S. Shankar Sastry
We formulate a general framework for competitive gradient-based learning that encompasses a wide breadth of multi-agent learning algorithms, and analyze the limiting behavior of competitive gradient-based learning algorithms using dynamical systems theory.
no code implementations • 11 Mar 2018 • Tanner Fiez, Shreyas Sekar, Lillian J. Ratliff
We analyze these algorithms under two types of smoothed reward feedback at the end of each epoch: a reward that is the discount-average of the discounted rewards within an epoch, and a reward that is the time-average of the rewards within an epoch.
no code implementations • 29 Mar 2017 • Lillian J. Ratliff, Eric Mazumdar
We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive.