no code implementations • 29 Mar 2024 • Ieva Petrulionyte, Julien Mairal, Michael Arbel
In this paper, we introduce a new functional point of view on bilevel optimization problems for machine learning, where the inner objective is minimized over a function space.
1 code implementation • 21 Feb 2024 • Michael Arbel, Alexandre Zouaoui
Replicability in machine learning (ML) research is increasingly concerning due to the utilization of complex non-deterministic algorithms and the dependence on numerous hyper-parameter choices, such as model architecture and training datasets.
no code implementations • 17 Feb 2024 • Juliette Marrie, Michael Arbel, Julien Mairal, Diane Larlus
Large pretrained visual models exhibit remarkable generalization across diverse recognition tasks.
no code implementations • CVPR 2023 • Juliette Marrie, Michael Arbel, Diane Larlus, Julien Mairal
Data augmentation is known to improve the generalization capabilities of neural networks, provided that the set of transformations is chosen with care, a selection often performed manually.
no code implementations • NeurIPS 2023 • Michael Arbel, Romain Menegaux, Pierre Wolinski
This work studies the global convergence and implicit bias of Gauss Newton's (GN) when optimizing over-parameterized one-hidden layer networks in the mean-field regime.
1 code implementation • 26 Oct 2022 • Pierre Glaser, Michael Arbel, Samo Hromadka, Arnaud Doucet, Arthur Gretton
We introduce two synthetic likelihood methods for Simulation-Based Inference (SBI), to conduct either amortized or targeted inference from experimental observations when a high-fidelity simulator is available.
2 code implementations • 31 Jan 2022 • Alexander G. D. G. Matthews, Michael Arbel, Danilo J. Rezende, Arnaud Doucet
We propose Continual Repeated Annealed Flow Transport Monte Carlo (CRAFT), a method that combines a sequential Monte Carlo (SMC) sampler (itself a generalization of Annealed Importance Sampling) with variational inference using normalizing flows.
no code implementations • ICLR 2022 • Michael Arbel, Julien Mairal
We study a class of algorithms for solving bilevel optimization problems in both stochastic and deterministic settings when the inner-level objective is strongly convex.
no code implementations • 4 Nov 2021 • Ted Moskovitz, Michael Arbel, Jack Parker-Holder, Aldo Pacchiano
Much of the recent success of deep reinforcement learning has been driven by regularized policy optimization (RPO) algorithms with strong performance across multiple domains.
1 code implementation • NeurIPS 2021 • Pierre Glaser, Michael Arbel, Arthur Gretton
We study the gradient flow for a relaxed approximation to the Kullback-Leibler (KL) divergence between a moving source and a fixed target distribution.
3 code implementations • 15 Feb 2021 • Michael Arbel, Alexander G. D. G. Matthews, Arnaud Doucet
Annealed Importance Sampling (AIS) and its Sequential Monte Carlo (SMC) extensions are state-of-the-art methods for estimating normalizing constants of probability distributions.
2 code implementations • NeurIPS 2021 • Ted Moskovitz, Jack Parker-Holder, Aldo Pacchiano, Michael Arbel, Michael I. Jordan
In recent years, deep off-policy actor-critic algorithms have become a dominant approach to reinforcement learning for continuous control.
1 code implementation • 19 Jan 2021 • Louis Thiry, Michael Arbel, Eugene Belilovsky, Edouard Oyallon
A recent line of work showed that various forms of convolutional kernel methods can be competitive with standard supervised deep convolutional networks on datasets like CIFAR-10, obtaining accuracies in the range of 87-90% while being more amenable to theoretical analysis.
no code implementations • ICLR 2021 • Louis Thiry, Michael Arbel, Eugene Belilovsky, Edouard Oyallon
A recent line of work showed that various forms of convolutional kernel methods can be competitive with standard supervised deep convolutional networks on datasets like CIFAR-10, obtaining accuracies in the range of 87-90% while being more amenable to theoretical analysis.
1 code implementation • ICLR 2021 • Ted Moskovitz, Michael Arbel, Ferenc Huszar, Arthur Gretton
A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL).
no code implementations • 14 Jul 2020 • Samuel Cohen, Michael Arbel, Marc Peter Deisenroth
Barycentric averaging is a principled way of summarizing populations of measures.
no code implementations • NeurIPS 2020 • Anna Korba, Adil Salim, Michael Arbel, Giulia Luise, Arthur Gretton
We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution $\pi\propto e^{-V}$ on $\mathbb{R}^d$.
no code implementations • CVPR 2020 • Tolga Birdal, Michael Arbel, Umut Şimşekli, Leonidas Guibas
We introduce a new paradigm, $\textit{measure synchronization}$, for synchronizing graphs with measure-valued edges.
1 code implementation • ICLR 2021 • Michael Arbel, Liang Zhou, Arthur Gretton
We show that both training stages are well-defined: the energy is learned by maximising a generalized likelihood, and the resulting energy-based loss provides informative gradients for learning the base.
1 code implementation • ICLR 2020 • Michael Arbel, Arthur Gretton, Wuchen Li, Guido Montufar
Many machine learning problems can be expressed as the optimization of some cost functional over a parametric family of probability distributions.
1 code implementation • NeurIPS 2019 • Michael Arbel, Anna Korba, Adil Salim, Arthur Gretton
We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties.
1 code implementation • NeurIPS 2018 • Michael Arbel, Danica J. Sutherland, Mikołaj Bińkowski, Arthur Gretton
We propose a principled method for gradient-based regularization of the critic of GAN-like models trained by adversarially optimizing the kernel of a Maximum Mean Discrepancy (MMD).
Ranked #128 on Image Generation on CIFAR-10
7 code implementations • ICLR 2018 • Mikołaj Bińkowski, Danica J. Sutherland, Michael Arbel, Arthur Gretton
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs.
1 code implementation • 15 Nov 2017 • Michael Arbel, Arthur Gretton
A nonparametric family of conditional distributions is introduced, which generalizes conditional exponential families using functional parameters in a suitable RKHS.
1 code implementation • 23 May 2017 • Danica J. Sutherland, Heiko Strathmann, Michael Arbel, Arthur Gretton
We propose a fast method with statistical guarantees for learning an exponential family density model where the natural parameter is in a reproducing kernel Hilbert space, and may be infinite-dimensional.