no code implementations • ICML 2020 • Fabian Latorre, Paul Rolland, Shaul Nadav Hallak, Volkan Cevher
We demonstrate two new important properties of the path-norm regularizer for shallow neural networks.
no code implementations • ICML 2020 • Paul Rolland, Armin Eftekhari, Ali Kavis, Volkan Cevher
A well-known first-order method for sampling from log-concave probability distributions is the Unadjusted Langevin Algorithm (ULA).
1 code implementation • 7 May 2024 • Elias Abad Rocamora, Yongtao Wu, Fanghui Liu, Grigorios G. Chrysos, Volkan Cevher
Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels.
no code implementations • 3 May 2024 • Luca Viano, Stratis Skoulakis, Volkan Cevher
We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs to sample from the environment.
no code implementations • 29 Apr 2024 • Fanghui Liu, Leello Dadi, Volkan Cevher
Based on this result, we derive the improved result of metric entropy for $\epsilon$-covering up to $\mathcal{O}(\epsilon^{-\frac{2d}{d+2}})$ ($d$ is the input dimension and the depending constant is at most polynomial order of $d$) via the convex hull technique, which demonstrates the separation with kernel methods with $\Omega(\epsilon^{-d})$ to learn the target function in a Barron space.
no code implementations • 19 Mar 2024 • Yongtao Wu, Fanghui Liu, Carl-Johann Simon-Gabriel, Grigorios G Chrysos, Volkan Cevher
Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data.
no code implementations • 14 Mar 2024 • Yihang Chen, Fanghui Liu, Yiping Lu, Grigorios G. Chrysos, Volkan Cevher
To derive the generalization bounds under this setting, our analysis necessitates a shift from the conventional time-invariant Gram matrix employed in the lazy training regime to a time-variant, distribution-dependent version.
no code implementations • 27 Feb 2024 • Vyas Raina, Samson Tan, Volkan Cevher, Aditya Rawal, Sheng Zha, George Karypis
Deep learning-based Natural Language Processing (NLP) models are vulnerable to adversarial attacks, where small perturbations can cause a model to misclassify.
no code implementations • 24 Feb 2024 • Adrian Müller, Pragnya Alatur, Volkan Cevher, Giorgia Ramponi, Niao He
As Efroni et al. (2020) pointed out, it is an open question whether primal-dual algorithms can provably achieve sublinear regret if we do not allow error cancellations.
no code implementations • 14 Feb 2024 • Yixin Cheng, Markos Georgopoulos, Volkan Cevher, Grigorios G. Chrysos
We contend that the prior context--the information preceding the attack query--plays a pivotal role in enabling potent Jailbreaking attacks.
no code implementations • 31 Jan 2024 • Yixin Cheng, Grigorios G. Chrysos, Markos Georgopoulos, Volkan Cevher
On the other hand, Polynomial Networks is a class of models that does not require activation functions, but have yet to perform on par with modern architectures.
1 code implementation • 21 Jan 2024 • Elias Abad Rocamora, Fanghui Liu, Grigorios G. Chrysos, Pablo M. Olmos, Volkan Cevher
Our regularization term can be theoretically linked to curvature of the loss function and is computationally cheaper than previous methods by avoiding Double Backpropagation.
no code implementations • 17 Jan 2024 • Kaan Ozkara, Can Karakus, Parameswaran Raman, Mingyi Hong, Shoham Sabach, Branislav Kveton, Volkan Cevher
Since Adam was introduced, several novel adaptive optimizers for deep learning have been proposed.
no code implementations • 5 Jan 2024 • Ruichen Jiang, Parameswaran Raman, Shoham Sabach, Aryan Mokhtari, Mingyi Hong, Volkan Cevher
In this paper, we introduce a novel subspace cubic regularized Newton method that achieves a dimension-independent global convergence rate of ${O}\left(\frac{1}{mk}+\frac{1}{k^2}\right)$ for solving convex optimization problems.
1 code implementation • NeurIPS 2023 • Thomas Pethick, Wanyun Xie, Volkan Cevher
This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training.
no code implementations • 18 Oct 2023 • Junaid Ali, Matthaeus Kleindessner, Florian Wenzel, Kailash Budhathoki, Volkan Cevher, Chris Russell
We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP), that are used for labeling tasks.
1 code implementation • 17 Aug 2023 • Ali Ramezani-Kebrya, Kimon Antonakopoulos, Igor Krawczuk, Justin Deschenaux, Volkan Cevher
We consider monotone variational inequality (VI) problems in multi-GPU settings where multiple processors/workers/clients have access to local stochastic dual vectors.
no code implementations • 19 Jun 2023 • Alexander Robey, Fabian Latorre, George J. Pappas, Hamed Hassani, Volkan Cevher
One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially chosen perturbations of data.
no code implementations • 8 Jun 2023 • Ali Ramezani-Kebrya, Fanghui Liu, Thomas Pethick, Grigorios Chrysos, Volkan Cevher
This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance.
no code implementations • 30 May 2023 • Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Francesco Locatello, Volkan Cevher
This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification while obtaining (nearly) zero-training error under the lazy training regime.
no code implementations • 25 Apr 2023 • Fanghui Liu, Luca Viano, Volkan Cevher
In online reinforcement learning (RL), instead of employing standard structural assumptions on Markov decision processes (MDPs), using a certain coverage condition (original from offline RL) is enough to ensure sample-efficient guarantees (Xie et al. 2023).
no code implementations • CVPR 2023 • Grigorios G Chrysos, Bohan Wang, Jiankang Deng, Volkan Cevher
We introduce a class of PNs, which are able to reach the performance of ResNet across a range of six benchmarks.
1 code implementation • ICLR 2022 • Thomas Pethick, Puya Latafat, Panagiotis Patrinos, Olivier Fercoq, Volkan Cevher
This paper introduces a new extragradient-type algorithm for a class of nonconvex-nonconcave minimax problems.
1 code implementation • 17 Feb 2023 • Thomas Pethick, Grigorios G. Chrysos, Volkan Cevher
Despite progress in adversarial training (AT), there is a substantial gap between the top-performing and worst-performing classes in many datasets.
1 code implementation • 17 Feb 2023 • Thomas Pethick, Olivier Fercoq, Puya Latafat, Panagiotis Patrinos, Volkan Cevher
This paper introduces a family of stochastic extragradient-type algorithms for a class of nonconvex-nonconcave problems characterized by the weak Minty variational inequality (MVI).
no code implementations • 10 Jan 2023 • Volkan Cevher, Georgios Piliouras, Ryann Sim, Stratis Skoulakis
In this paper we present a first-order method that admits near-optimal convergence rates for convex/concave min-max problems while requiring a simple and intuitive analysis.
no code implementations • 3 Nov 2022 • Kimon Antonakopoulos, Ali Kavis, Volkan Cevher
This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions.
no code implementations • 3 Nov 2022 • Ali Kavis, Stratis Skoulakis, Kimon Antonakopoulos, Leello Tadesse Dadi, Volkan Cevher
We propose an adaptive variance-reduction method, called AdaSpider, for minimization of $L$-smooth, non-convex functions with a finite-sum structure.
1 code implementation • 29 Sep 2022 • Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, Pascal Frossard
This work introduces DiGress, a discrete denoising diffusion model for generating graphs with categorical node and edge attributes.
2 code implementations • 22 Sep 2022 • Luca Viano, Angeliki Kamoutsi, Gergely Neu, Igor Krawczuk, Volkan Cevher
Thanks to PPM, we avoid nested policy evaluation and cost updates for online IL appearing in the prior literature.
1 code implementation • 22 Sep 2022 • Paul Rolland, Luca Viano, Norman Schuerhoff, Boris Nikolov, Volkan Cevher
While Reinforcement Learning (RL) aims to train an agent from a reward function in a given environment, Inverse Reinforcement Learning (IRL) seeks to recover the reward function from observing an expert's behavior.
no code implementations • 16 Sep 2022 • Yongtao Wu, Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher
Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds.
1 code implementation • 15 Sep 2022 • Elias Abad Rocamora, Mehmet Fatih Sahin, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher
Polynomial Networks (PNs) have demonstrated promising performance on face and image recognition recently.
no code implementations • 15 Sep 2022 • Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher
In particular, when initialized with LeCun initialization, depth helps robustness with the lazy training regime.
no code implementations • 15 Sep 2022 • Fanghui Liu, Luca Viano, Volkan Cevher
To be specific, we focus on the value based algorithm with the $\epsilon$-greedy exploration via deep (and two-layer) neural networks endowed by Besov (and Barron) function spaces, respectively, which aims at approximating an $\alpha$-smooth Q-function in a $d$-dimensional feature space.
no code implementations • 15 Sep 2022 • Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher
To this end, we derive the lower (and upper) bounds of the minimum eigenvalue of the Neural Tangent Kernel (NTK) under the (in)finite-width regime using a certain search space including mixed activation functions, fully connected, and residual neural networks.
no code implementations • 14 Jun 2022 • Yongtao Wu, Grigorios G Chrysos, Volkan Cevher
Our models can encourage the systematic design of other efficient architectures on the complex field.
no code implementations • 13 Jun 2022 • Yu-Guan Hsieh, Kimon Antonakopoulos, Volkan Cevher, Panayotis Mertikopoulos
We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algorithm, it is possible to achieve significantly lower regret relative to fully adversarial environments.
no code implementations • 8 Jun 2022 • Panayotis Mertikopoulos, Ya-Ping Hsieh, Volkan Cevher
We develop a flexible stochastic approximation framework for analyzing the long-run behavior of learning in games (both continuous and finite).
no code implementations • ICLR 2022 • Ali Kavis, Kfir Yehuda Levy, Volkan Cevher
We present our analysis in a modular way and obtain a complementary $\mathcal O (1 / T)$ convergence rate in the deterministic setting.
no code implementations • 8 Mar 2022 • Paul Rolland, Volkan Cevher, Matthäus Kleindessner, Chris Russel, Bernhard Schölkopf, Dominik Janzing, Francesco Locatello
This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models.
no code implementations • ICLR 2022 • Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos, Julien Mairal, Volkan Cevher
Inspired by such studies, we conduct a spectral analysis of the Neural Tangent Kernel (NTK) of PNNs.
1 code implementation • 26 Feb 2022 • Gideon Dresdner, Maria-Luiza Vladarean, Gunnar Rätsch, Francesco Locatello, Volkan Cevher, Alp Yurtsever
We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms.
no code implementations • ICLR 2022 • Zhenyu Zhu, Fabian Latorre, Grigorios G Chrysos, Volkan Cevher
While the class of Polynomial Nets demonstrates comparable performance to neural networks (NN), it currently has neither theoretical generalization characterization nor robustness guarantees.
no code implementations • 19 Jan 2022 • Ahmet Alacaoglu, Volkan Cevher, Stephen J. Wright
We prove complexity bounds for the primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD), which has been shown to obtain good practical performance for solving convex-concave min-max problems with bilinear coupling.
no code implementations • NeurIPS 2021 • Fabian Latorre, Leello Tadesse Dadi, Paul Rolland, Volkan Cevher
We demonstrate this by deriving an upper bound on the Rademacher Complexity that depends on two key quantities: (i) the intrinsic dimension, which is a measure of isotropy, and (ii) the largest eigenvalue of the second moment (covariance) matrix of the distribution.
no code implementations • NeurIPS 2021 • Kimon Antonakopoulos, Thomas Pethick, Ali Kavis, Panayotis Mertikopoulos, Volkan Cevher
Our first result is that the algorithm achieves the optimal rates of convergence for cocoercive problems when the profile of the randomness is known to the optimizer: $\mathcal{O}(1/\sqrt{T})$ for absolute noise profiles, and $\mathcal{O}(1/T)$ for relative ones.
no code implementations • NeurIPS 2021 • Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.
no code implementations • NeurIPS 2021 • Kfir Levy, Ali Kavis, Volkan Cevher
In this work we propose $\rm{STORM}^{+}$, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.
no code implementations • NeurIPS 2021 • ChaeHwan Song, Ali Ramezani-Kebrya, Thomas Pethick, Armin Eftekhari, Volkan Cevher
Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex training.
no code implementations • 1 Nov 2021 • Kfir Y. Levy, Ali Kavis, Volkan Cevher
In this work we propose STORM+, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.
no code implementations • NeurIPS 2021 • Maria-Luiza Vladarean, Yura Malitsky, Volkan Cevher
We consider the problem of finding a saddle point for the convex-concave objective $\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)$, where $f$ is a convex function with locally Lipschitz gradient and $g$ is convex and possibly non-smooth.
no code implementations • 13 Oct 2021 • Fanghui Liu, Johan A. K. Suykens, Volkan Cevher
We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime.
no code implementations • 29 Sep 2021 • Thomas Sanchez, Igor Krawczuk, Volkan Cevher
Deep learning approaches have shown great promise in accelerating magnetic resonance imaging (MRI), by reconstructing high quality images from highly undersampled data.
no code implementations • 29 Sep 2021 • Paul Rolland, Ali Ramezani-Kebrya, ChaeHwan Song, Fabian Latorre, Volkan Cevher
Despite the non-convex landscape, first-order methods can be shown to reach global minima when training overparameterized neural networks, where the number of parameters far exceed the number of training data.
no code implementations • 29 Sep 2021 • Thomas Pethick, Grigorios Chrysos, Volkan Cevher
In this work, we identify that the focus on the average accuracy metric can create vulnerabilities to the "weakest" class.
no code implementations • 29 Sep 2021 • Yihang Chen, Grigorios Chrysos, Volkan Cevher
Domain generalization deals with the difference in the distribution between the training and testing datasets, i. e., the domain shift problem, by extracting domain-invariant features.
no code implementations • 29 Sep 2021 • Ahmet Alacaoglu, Luca Viano, Niao He, Volkan Cevher
Our sample complexities also match the best-known results for global convergence of policy gradient and two time-scale actor-critic algorithms in the single agent setting.
1 code implementation • 17 Sep 2021 • Aleksandr Timofeev, Grigorios G. Chrysos, Volkan Cevher
The results demonstrate how the proposed method can be used in imbalanced datasets, while it can be fully run on a single GPU.
no code implementations • 1 Jan 2021 • Igor Krawczuk, Pedro Abranches, Andreas Loukas, Volkan Cevher
We study the fundamental problem of graph generation.
1 code implementation • 3 Nov 2020 • Zhaodong Sun, Thomas Sanchez, Fabian Latorre, Volkan Cevher
When the noise level is small, it does not considerably reduce the overfitting problem.
no code implementations • 23 Oct 2020 • Thomas Sanchez, Igor Krawczuk, Zhaodong Sun, Volkan Cevher
We propose an adaptive sampling method for the linear model, driven by the uncertainty estimation with a generative adversarial network (GAN) model.
no code implementations • 13 Oct 2020 • Nadav Hallak, Panayotis Mertikopoulos, Volkan Cevher
In this setting, the minimization of external regret is beyond reach for first-order methods, so we focus on a local regret measure defined via a proximal-gradient mapping.
no code implementations • ICML 2020 • Ahmet Alacaoglu, Olivier Fercoq, Volkan Cevher
We introduce a randomly extrapolated primal-dual coordinate descent method that adapts to sparsity of the data matrix and the favorable structures of the objective function.
no code implementations • ICML 2020 • Maria-Luiza Vladarean, Ahmet Alacaoglu, Ya-Ping Hsieh, Volkan Cevher
We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints.
1 code implementation • NeurIPS 2021 • Luca Viano, Yu-Ting Huang, Parameswaran Kamalaruban, Adrian Weller, Volkan Cevher
We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner.
no code implementations • 2 Jul 2020 • Fabian Latorre, Paul Rolland, Nadav Hallak, Volkan Cevher
We demonstrate two new important properties of the 1-path-norm of shallow neural networks.
no code implementations • 1 Jul 2020 • Martin Troussard, Emmanuel Pignat, Parameswaran Kamalaruban, Sylvain Calinon, Volkan Cevher
This paper proposes an inverse reinforcement learning (IRL) framework to accelerate learning when the learner-teacher \textit{interaction} is \textit{limited} during training.
no code implementations • 23 Jun 2020 • Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla
However, the applicability of potential-based reward shaping is limited in settings where (i) the state space is very large, and it is challenging to compute an appropriate potential function, (ii) the feedback signals are noisy, and even with shaped rewards the agent could be trapped in local optima, and (iii) changing the rewards alone is not sufficient, and effective shaping requires changing the dynamics.
no code implementations • NeurIPS 2020 • Panayotis Mertikopoulos, Nadav Hallak, Ali Kavis, Volkan Cevher
This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's convergence properties in non-convex problems.
no code implementations • 16 Jun 2020 • Ya-Ping Hsieh, Panayotis Mertikopoulos, Volkan Cevher
Compared to ordinary function minimization problems, min-max optimization algorithms encounter far greater challenges because of the existence of periodic cycles and similar phenomena.
no code implementations • 11 Jun 2020 • Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.
no code implementations • ICLR 2020 • Fabian Latorre, Paul Rolland, Volkan Cevher
We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bounds on the Lipschitz constant of neural networks.
no code implementations • ICML 2020 • Ahmet Alacaoglu, Yura Malitsky, Panayotis Mertikopoulos, Volkan Cevher
In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.).
1 code implementation • 17 Feb 2020 • Deyi Liu, Volkan Cevher, Quoc Tran-Dinh
We demonstrate how to scalably solve a class of constrained self-concordant minimization problems using linear minimization oracles (LMO) over the constraint set.
1 code implementation • 14 Feb 2020 • Parameswaran Kamalaruban, Yu-Ting Huang, Ya-Ping Hsieh, Paul Rolland, Cheng Shi, Volkan Cevher
We introduce a sampling perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents.
no code implementations • 1 Dec 2019 • Donghwan Lee, Niao He, Parameswaran Kamalaruban, Volkan Cevher
This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and cooperate.
Distributed Optimization Multi-agent Reinforcement Learning +2
no code implementations • NeurIPS 2019 • Ali Kavis, Kfir. Y. Levy, Francis Bach, Volkan Cevher
To the best of our knowledge, this is the first adaptive, unified algorithm that achieves the optimal rates in the constrained setting.
no code implementations • 9 Oct 2019 • Armin Eftekhari, ChaeHwan Song, Volkan Cevher
A recent line of work has shown that an overparametrized neural network can perfectly fit the training data, an otherwise often intractable nonconvex optimization problem.
no code implementations • 25 Sep 2019 • Thomas Sanchez, Igor Krawczuk, Zhaodong Sun, Volkan Cevher
This work proposes a closed-loop, uncertainty-driven adaptive sampling frame- work (CLUDAS) for accelerating magnetic resonance imaging (MRI) via deep Bayesian inversion.
no code implementations • NeurIPS 2019 • Fabian Latorre Gómez, Armin Eftekhari, Volkan Cevher
We focus on the special case where such constraint arises from the specification that a variable should lie in the range of a neural network.
no code implementations • 28 May 2019 • Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla
We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher.
no code implementations • 15 Mar 2019 • Chen Liu, Ryota Tomioka, Volkan Cevher
This work studies the robustness certification problem of neural network models, which aims to find certified adversary-free regions as large as possible around data points.
no code implementations • 9 Feb 2019 • Lijun Ding, Alp Yurtsever, Volkan Cevher, Joel A. Tropp, Madeleine Udell
This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form.
1 code implementation • 1 Feb 2019 • Thomas Sanchez, Baran Gözcü, Ruud B. van Heeswijk, Armin Eftekhari, Efe Ilıcak, Tolga Çukur, Volkan Cevher
Compressed sensing applied to magnetic resonance imaging (MRI) allows to reduce the scanning time by enabling images to be reconstructed from highly undersampled data.
1 code implementation • NeurIPS 2019 • Francesco Locatello, Alp Yurtsever, Olivier Fercoq, Volkan Cevher
A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints.
no code implementations • 2 Jan 2019 • Jonathan Scarlett, Volkan Cevher
Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning.
no code implementations • 11 Dec 2018 • Paul Rolland, Ali Kavis, Alex Immer, Adish Singla, Volkan Cevher
We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests.
no code implementations • 8 Nov 2018 • Teresa Yeo, Parameswaran Kamalaruban, Adish Singla, Arpit Merchant, Thibault Asselborn, Louis Faucon, Pierre Dillenbourg, Volkan Cevher
We consider the machine teaching problem in a classroom-like setting wherein the teacher has to deliver the same examples to a diverse group of students.
no code implementations • 5 Nov 2018 • Junhong Lin, Volkan Cevher
We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space.
no code implementations • NeurIPS 2018 • Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, Volkan Cevher
In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we require the function value to remain as high as possible even after this perturbation.
no code implementations • ICLR 2019 • Ya-Ping Hsieh, Chen Liu, Volkan Cevher
We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective.
no code implementations • 12 Sep 2018 • Ali Ramezani-Kebrya, Kimon Antonakopoulos, Volkan Cevher, Ashish Khisti, Ben Liang
While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods.
no code implementations • NeurIPS 2018 • Kfir. Y. Levy, Alp Yurtsever, Volkan Cevher
We present a novel method for convex unconstrained optimization that, without any modifications, ensures: (i) accelerated convergence rate for smooth objectives, (ii) standard convergence rate in the general (non-smooth) setting, and (iii) standard convergence rate in the stochastic optimization setting.
no code implementations • ICML 2018 • Ehsan Asadi Kangarshahi, Ya-Ping Hsieh, Mehmet Fatih Sahin, Volkan Cevher
We propose a simple algorithmic framework that simultaneously achieves the best rates for honest regret as well as adversarial regret, and in addition resolves the open problem of removing the logarithmic terms in convergence to the value of the game.
no code implementations • ICML 2018 • Junhong Lin, Volkan Cevher
We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS).
no code implementations • 3 May 2018 • Baran Gözcü, Rabeeh Karimi Mahabadi, Yen-Huan Li, Efe Ilıcak, Tolga Çukur, Jonathan Scarlett, Volkan Cevher
In the area of magnetic resonance imaging (MRI), an extensive range of non-linear reconstruction algorithms have been proposed that can be used with general Fourier subsampling patterns.
no code implementations • ICML 2018 • Junhong Lin, Volkan Cevher
We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space.
no code implementations • NeurIPS 2018 • Ya-Ping Hsieh, Ali Kavis, Paul Rolland, Volkan Cevher
We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design.
no code implementations • 26 Feb 2018 • Ya-Ping Hsieh, Volkan Cevher
Information concentration of probability measures have important implications in learning theory.
1 code implementation • 20 Feb 2018 • Paul Rolland, Jonathan Scarlett, Ilija Bogunovic, Volkan Cevher
In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables.
no code implementations • 20 Feb 2018 • Ilija Bogunovic, Junyao Zhao, Volkan Cevher
In this work, we present a new algorithm Oblivious-Greedy and prove the first constant-factor approximation guarantees for a wider class of non-submodular objectives.
no code implementations • 22 Jan 2018 • Junhong Lin, Volkan Cevher
We then extend our results to spectral-regularization algorithms (SRA), including kernel ridge regression (KRR), kernel principal component analysis, and gradient methods.
no code implementations • 20 Jan 2018 • Junhong Lin, Alessandro Rudi, Lorenzo Rosasco, Volkan Cevher
In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space.
no code implementations • NeurIPS 2017 • Ahmet Alacaoglu, Quoc Tran-Dinh, Olivier Fercoq, Volkan Cevher
We propose a new randomized coordinate descent method for a convex optimization template with broad applications.
no code implementations • NeurIPS 2017 • Slobodan Mitrović, Ilija Bogunovic, Ashkan Norouzi-Fard, Jakub Tarnawski, Volkan Cevher
We study the classical problem of maximizing a monotone submodular function subject to a cardinality constraint k, with two additional twists: (i) elements arrive in a streaming fashion, and (ii) m items from the algorithm's memory are removed after the stream is finished.
no code implementations • NeurIPS 2017 • Jonathan Scarlett, Volkan Cevher
In this paper, we study the pooled data problem of identifying the labels associated with a large collection of items, based on a sequence of pooled tests revealing the counts of each label within the pool.
no code implementations • 17 Oct 2017 • Marwa El Halabi, Francis Bach, Volkan Cevher
We consider the homogeneous and the non-homogeneous convex relaxations for combinatorial penalty functions defined on support sets.
no code implementations • NeurIPS 2017 • Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher
Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates.
no code implementations • ICML 2017 • Ilija Bogunovic, Slobodan Mitrović, Jonathan Scarlett, Volkan Cevher
We study the problem of maximizing a monotone submodular function subject to a cardinality constraint $k$, with the added twist that a number of items $\tau$ from the returned set may be removed.
no code implementations • 31 May 2017 • Jonathan Scarlett, Ilijia Bogunovic, Volkan Cevher
For the isotropic squared-exponential kernel in $d$ dimensions, we find that an average simple regret of $\epsilon$ requires $T = \Omega\big(\frac{1}{\epsilon^2} (\log\frac{1}{\epsilon})^{d/2}\big)$, and the average cumulative regret is at least $\Omega\big( \sqrt{T(\log T)^{d/2}} \big)$, thus matching existing upper bounds up to the replacement of $d/2$ by $2d+O(1)$ in both cases.
no code implementations • 7 Mar 2017 • Dmytro Perekrestenko, Volkan Cevher, Martin Jaggi
Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems.
1 code implementation • 22 Feb 2017 • Alp Yurtsever, Madeleine Udell, Joel A. Tropp, Volkan Cevher
This paper concerns a fundamental class of convex matrix optimization problems.
no code implementations • NeurIPS 2016 • Ilija Bogunovic, Jonathan Scarlett, Andreas Krause, Volkan Cevher
We present a new algorithm, truncated variance reduction (TruVaR), that treats Bayesian optimization (BO) and level-set estimation (LSE) with Gaussian processes in a unified fashion.
no code implementations • 31 Aug 2016 • Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher
This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch.
no code implementations • 8 Jul 2016 • Jonathan Scarlett, Volkan Cevher
We consider the problem of estimating the underlying graph associated with a Markov random field, with the added twist that the decoding algorithm can iteratively choose which subsets of nodes to sample based on the previous samples, resulting in an active learning setting.
no code implementations • 21 Mar 2016 • Anastasios Kyrillidis, Bubacarr Bah, Rouzbeh Hasheminezhad, Quoc Tran-Dinh, Luca Baldassarre, Volkan Cevher
Our experimental findings on synthetic and real applications support our claims for faster recovery in the convex setting -- as opposed to using dense sensing matrices, while showing a competitive recovery performance.
no code implementations • 5 Mar 2016 • Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher
First, it allows handling non-smooth objectives via proximal operators; this avoids lifting the problem dimension in order to accommodate non-smooth components in optimization.
no code implementations • 11 Feb 2016 • Jonathan Scarlett, Volkan Cevher
We adopt an \emph{approximate recovery} criterion that allows for a number of missed edges or incorrectly-included edges, in contrast with the widely-studied exact recovery problem.
no code implementations • 2 Feb 2016 • Jonathan Scarlett, Volkan Cevher
In this paper, we study the information-theoretic limits of community detection in the symmetric two-community stochastic block model, with intra-community and inter-community edge probabilities $\frac{a}{n}$ and $\frac{b}{n}$ respectively.
no code implementations • 1 Feb 2016 • Yen-Huan Li, Volkan Cevher
The standard approach to compressive sampling considers recovering an unknown deterministic signal with certain known structure, and designing the sub-sampling pattern and recovery algorithm based on the known structure.
no code implementations • 25 Jan 2016 • Ilija Bogunovic, Jonathan Scarlett, Volkan Cevher
We illustrate the performance of the algorithms on both synthetic and real data, and we find the gradual forgetting of TV-GP-UCB to perform favorably compared to the sharp resetting of R-GP-UCB.
no code implementations • NeurIPS 2015 • David E. Carlson, Edo Collins, Ya-Ping Hsieh, Lawrence Carin, Volkan Cevher
These challenges include, but are not limited to, the non-convexity of learning objectives and estimating the quantities needed for optimization algorithms, such as gradients.
no code implementations • NeurIPS 2015 • Alp Yurtsever, Quoc Tran Dinh, Volkan Cevher
We propose a new primal-dual algorithmic framework for a prototypical constrained convex optimization template.
no code implementations • 21 Oct 2015 • Luca Baldassarre, Yen-Huan Li, Jonathan Scarlett, Baran Gözcü, Ilija Bogunovic, Volkan Cevher
In this paper, we instead take a principled learning-based approach in which a \emph{fixed} index set is chosen based on a set of training signals $\mathbf{x}_1,\dotsc,\mathbf{x}_m$.
no code implementations • 20 Jul 2015 • Anastasios Kyrillidis, Luca Baldassarre, Marwa El-Halabi, Quoc Tran-Dinh, Volkan Cevher
For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations.
no code implementations • 20 Jul 2015 • Volkan Cevher, Sina Jafarpour, Anastasios Kyrillidis
We describe two nonconventional algorithms for linear regression, called GAME and CLASH.
no code implementations • 14 Jul 2015 • Quoc Tran-Dinh, Volkan Cevher
We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems.
2 code implementations • 11 Mar 2015 • Joao F. C. Mota, Nikos Deligiannis, Aswin C. Sankaranarayanan, Volkan Cevher, Miguel R. D. Rodrigues
We propose and analyze an online algorithm for reconstructing a sequence of signals from a limited number of linear measurements.
no code implementations • 4 Feb 2015 • Quoc Tran-Dinh, Yen-Huan Li, Volkan Cevher
The self-concordant-like property of a smooth convex function is a new analytical structure that generalizes the self-concordant notion.
no code implementations • 29 Jan 2015 • Jonathan Scarlett, Volkan Cevher
In several cases, our bounds not only provide matching scaling laws in the necessary and sufficient number of measurements, but also sharp thresholds with matching constant factors.
no code implementations • NeurIPS 2014 • John J. Bruer, Joel A. Tropp, Volkan Cevher, Stephen Becker
This paper proposes a tradeoff between sample complexity and computation time that applies to statistical estimators based on convex optimization.
no code implementations • NeurIPS 2014 • Quoc Tran-Dinh, Volkan Cevher
We introduce a model-based excessive gap technique to analyze first-order primal- dual methods for constrained convex minimization.
no code implementations • 7 Nov 2014 • Marwa El Halabi, Volkan Cevher
This paper describes a simple framework for structured sparse recovery based on convex optimization.
no code implementations • 4 Nov 2014 • Volkan Cevher, Stephen Becker, Mark Schmidt
This article reviews recent advances in convex optimization algorithms for Big Data, which aim to reduce the computational, storage, and communications bottlenecks.
no code implementations • 20 Jun 2014 • Quoc Tran-Dinh, Volkan Cevher
Our main analysis technique provides a fresh perspective on Nesterov's excessive gap technique in a structured fashion and unifies it with smoothing and primal-dual methods.
1 code implementation • 4 Jun 2014 • Aleksandr Aravkin, Stephen Becker, Volkan Cevher, Peder Olsen
We introduce a new convex formulation for stable principal component pursuit (SPCP) to decompose noisy signals into low-rank and sparse representations.
no code implementations • 13 May 2014 • Anastasios Kyrillidis, Rabeeh Karimi Mahabadi, Quoc Tran-Dinh, Volkan Cevher
We consider the class of convex minimization problems, composed of a self-concordant function, such as the $\log\det$ metric, a convex data fidelity term $h(\cdot)$ and, a regularizing -- possibly non-smooth -- function $g(\cdot)$.
no code implementations • NeurIPS 2013 • Josip Djolonga, Andreas Krause, Volkan Cevher
Many applications in machine learning require optimizing unknown functions defined over a high-dimensional space from noisy samples that are expensive to obtain.
no code implementations • 7 Nov 2013 • Quoc Tran Dinh, Anastasios Kyrillidis, Volkan Cevher
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets.
no code implementations • 7 Oct 2013 • Hemant Tyagi, Volkan Cevher
We consider the problem of learning multi-ridge functions of the form f(x) = g(Ax) from point evaluations of f. We assume that the function f is defined on an l_2-ball in R^d, g is twice continuously differentiable almost everywhere, and A \in R^{k \times d} is a rank k matrix, where k << d. We propose a randomized, polynomial-complexity sampling scheme for estimating such functions.
no code implementations • 13 Aug 2013 • Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher
We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator.
no code implementations • 12 Jul 2013 • Bubacarr Bah, Ali Sadeghian, Volkan Cevher
We propose a dimensionality reducing matrix design based on training data with constraints on its Frobenius norm and number of rows.
no code implementations • 13 Mar 2013 • Luca Baldassarre, Nirav Bhan, Volkan Cevher, Anastasios Kyrillidis, Siddhartha Satpathi
Group-based sparsity models are proven instrumental in linear regression problems for recovering signals from much fewer measurements than standard compressive sensing.
no code implementations • NeurIPS 2012 • Tyagi Hemant, Volkan Cevher
We consider the problem of actively learning \textit{multi-index} functions of the form $f(\vecx) = g(\matA\vecx)= \sum_{i=1}^k g_i(\veca_i^T\vecx)$ from point evaluations of $f$.
no code implementations • NeurIPS 2009 • Volkan Cevher
By using order statistics, we show that N-sample iid realizations of generalized Pareto, Student’s t, log-normal, Frechet, and log-logistic distributions are compressible, i. e., they have a constant expected decay rate, which is independent of N. In contrast, we show that generalized Gaussian distribution with shape parameter q is compressible only in restricted cases since the expected decay rate of its N-sample iid realizations decreases with N as 1/[q log(N/q)].
no code implementations • NeurIPS 2008 • Volkan Cevher, Marco F. Duarte, Chinmay Hegde, Richard Baraniuk
Compressive Sensing (CS) combines sampling and compression into a single sub-Nyquist linear measurement process for sparse and compressible signals.