no code implementations • ICML 2020 • Jayadev Acharya, Kallista Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun
The original definition of LDP assumes that all the elements in the data domain are equally sensitive.
no code implementations • 2 Apr 2024 • Joy Qiping Yang, Salman Salamatian, Ziteng Sun, Ananda Theertha Suresh, Ahmad Beirami
The goal of language model alignment is to alter $p$ to a new distribution $\phi$ that results in a higher expected reward while keeping $\phi$ close to $p.$ A popular alignment method is the KL-constrained reinforcement learning (RL), which chooses a distribution $\phi_\Delta$ that maximizes $E_{\phi_{\Delta}} r(y)$ subject to a relative entropy constraint $KL(\phi_\Delta || p) \leq \Delta.$ Another simple alignment method is best-of-$N$, where $N$ samples are drawn from $p$ and one with highest reward is selected.
no code implementations • 15 Mar 2024 • Ziteng Sun, Jae Hun Ro, Ahmad Beirami, Ananda Theertha Suresh
To the best of our knowledge, our work is the first to establish improvement over speculative decoding through a better draft verification algorithm.
no code implementations • NeurIPS 2023 • Ziteng Sun, Ananda Theertha Suresh, Jae Hun Ro, Ahmad Beirami, Himanshu Jain, Felix Yu
We show that the optimal draft selection algorithm (transport plan) can be computed via linear programming, whose best-known runtime is exponential in $k$.
no code implementations • 19 Jul 2023 • Ziteng Sun, Ananda Theertha Suresh, Aditya Krishna Menon
Training machine learning models with differential privacy (DP) has received increasing interest in recent years.
no code implementations • 1 Mar 2023 • Travis Dick, Alex Kulesza, Ziteng Sun, Ananda Theertha Suresh
We propose a new definition of instance optimality for differentially private estimation algorithms.
no code implementations • 14 Feb 2023 • Clément L. Canonne, Ziteng Sun, Ananda Theertha Suresh
We study the problem of discrete distribution estimation in KL divergence and provide concentration bounds for the Laplace estimator.
no code implementations • 7 Nov 2022 • Jayadev Acharya, YuHan Liu, Ziteng Sun
Perhaps surprisingly, we show that in suitable parameter regimes, having $m$ samples per user is equivalent to having $m$ times more users, each with only one sample.
no code implementations • 14 Mar 2022 • Jayadev Acharya, Clément L. Canonne, Ziteng Sun, Himanshu Tyagi
Without sparsity assumptions, it has been established that interactivity cannot improve the minimax rates of estimation under these information constraints.
no code implementations • 9 Mar 2022 • Ananda Theertha Suresh, Ziteng Sun, Jae Hun Ro, Felix Yu
We show that applying the proposed protocol as sub-routine in distributed optimization algorithms leads to better convergence rates.
no code implementations • NeurIPS 2021 • Jayadev Acharya, Clement Canonne, YuHan Liu, Ziteng Sun, Himanshu Tyagi
We obtain tight minimax rates for the problem of distributed estimation of discrete distributions under communication constraints, where $n$ users observing $m $ samples each can broadcast only $\ell$ bits.
no code implementations • 21 Apr 2021 • Jayadev Acharya, Ziteng Sun, Huanyu Zhang
We consider both the "centralized setting" and the "distributed setting with information constraints" including communication and local privacy (LDP) constraints.
no code implementations • NeurIPS 2021 • Daniel Levy, Ziteng Sun, Kareem Amin, Satyen Kale, Alex Kulesza, Mehryar Mohri, Ananda Theertha Suresh
We show that for high-dimensional mean estimation, empirical risk minimization with smooth losses, stochastic convex optimization, and learning hypothesis classes with finite metric entropy, the privacy cost decreases as $O(1/\sqrt{m})$ as users provide more samples.
no code implementations • 30 Oct 2020 • Jayadev Acharya, Peter Kairouz, YuHan Liu, Ziteng Sun
We consider the problem of estimating sparse discrete distributions under local differential privacy (LDP) and communication constraints.
no code implementations • 21 Jul 2020 • Jayadev Acharya, Clément L. Canonne, Yu-Han Liu, Ziteng Sun, Himanshu Tyagi
We study the role of interactivity in distributed statistical inference under information constraints, e. g., communication constraints and local differential privacy.
no code implementations • 14 Apr 2020 • Jayadev Acharya, Ziteng Sun, Huanyu Zhang
The technical component of our paper relates coupling between distributions to the sample complexity of estimation under differential privacy.
8 code implementations • 10 Dec 2019 • Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
no code implementations • 18 Nov 2019 • Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, H. Brendan McMahan
This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task.
no code implementations • NeurIPS 2019 • Jayadev Acharya, Sourbh Bhadane, Piotr Indyk, Ziteng Sun
We consider the task of estimating the entropy of $k$-ary distributions from samples in the streaming model, where space is limited.
no code implementations • 31 Oct 2019 • Jayadev Acharya, Keith Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun
Local differential privacy (LDP) is a strong notion of privacy for individual users that often comes at the expense of a significant drop in utility.
no code implementations • 20 Jul 2019 • Jayadev Acharya, Clément L. Canonne, Yanjun Han, Ziteng Sun, Himanshu Tyagi
We study goodness-of-fit of discrete distributions in the distributed setting, where samples are divided between multiple users who can only release a limited amount of information about their samples due to various information constraints.
no code implementations • 28 May 2019 • Jayadev Acharya, Ziteng Sun
We consider the problems of distribution estimation and heavy hitter (frequency) estimation under privacy and communication constraints.
1 code implementation • ICML 2018 • Jayadev Acharya, Gautam Kamath, Ziteng Sun, Huanyu Zhang
We develop differentially private methods for estimating various distributional properties.
3 code implementations • 13 Feb 2018 • Jayadev Acharya, Ziteng Sun, Huanyu Zhang
All previously known sample optimal algorithms require linear (in $k$) communication from each user in the high privacy regime $(\varepsilon=O(1))$, and run in time that grows as $n\cdot k$, which can be prohibitive for large domain size $k$.
no code implementations • NeurIPS 2018 • Jayadev Acharya, Ziteng Sun, Huanyu Zhang
We propose a general framework to establish lower bounds on the sample complexity of statistical tasks under differential privacy.