no code implementations • 3 Mar 2024 • Amal Rannen-Triki, Jorg Bornschein, Razvan Pascanu, Marcus Hutter, Andras György, Alexandre Galashov, Yee Whye Teh, Michalis K. Titsias
We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation.
no code implementations • 14 Jun 2023 • Michalis K. Titsias, Alexandre Galashov, Amal Rannen-Triki, Razvan Pascanu, Yee Whye Teh, Jorg Bornschein
Non-stationarity over the linear predictor weights is modelled using a parameter drift transition density, parametrized by a coefficient that quantifies forgetting.
no code implementations • 20 Feb 2022 • Sotirios Nikoloutsopoulos, Iordanis Koutsopoulos, Michalis K. Titsias
At the final update, each client computes the joint gradient over both client-specific and common weights and returns the gradient of common parameters to the server.
1 code implementation • 19 Feb 2022 • Jiaxin Shi, Yuhao Zhou, Jessica Hwang, Michalis K. Titsias, Lester Mackey
Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems.
1 code implementation • pproximateinference AABI Symposium 2022 • Michalis K. Titsias, Jiaxin Shi
We introduce a variance reduction technique for score function estimators that makes use of double control variates.
1 code implementation • NeurIPS 2021 • Marcel Hirt, Michalis K. Titsias, Petros Dellaportas
Hamiltonian Monte Carlo (HMC) is a popular Markov Chain Monte Carlo (MCMC) algorithm to sample from an unnormalized probability distribution.
no code implementations • 6 Oct 2020 • Michalis K. Titsias, Jakub Sygnowski, Yutian Chen
We introduce a framework for online changepoint detection and simultaneous model learning which is applicable to highly parametrized models, such as deep neural networks.
no code implementations • 5 Oct 2020 • Francisco J. R. Ruiz, Michalis K. Titsias, Taylan Cemgil, Arnaud Doucet
The variational auto-encoder (VAE) is a deep latent variable model that has two neural networks in an autoencoder-like architecture; one of them parameterizes the model's likelihood.
no code implementations • 7 Sep 2020 • Michalis K. Titsias, Francisco J. R. Ruiz, Sotirios Nikoloutsopoulos, Alexandre Galashov
We formulate meta learning using information theoretic concepts; namely, mutual information and the information bottleneck.
1 code implementation • NeurIPS 2019 • Michalis K. Titsias, Petros Dellaportas
We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets.
1 code implementation • pproximateinference AABI Symposium 2019 • Jiaxin Shi, Michalis K. Titsias, andriy mnih
We introduce a new interpretation of sparse variational approximations for Gaussian processes using inducing points, which can lead to more scalable algorithms than previous methods.
2 code implementations • 9 Oct 2019 • Adji B. Dieng, Francisco J. R. Ruiz, David M. Blei, Michalis K. Titsias
Generative adversarial networks (GANs) are a powerful approach to unsupervised learning.
Ranked #2 on Image Generation on Stacked MNIST
2 code implementations • 10 May 2019 • Francisco J. R. Ruiz, Michalis K. Titsias
We develop a method to combine Markov chain Monte Carlo (MCMC) and variational inference (VI), leveraging the advantages of both inference approaches.
1 code implementation • ICLR 2020 • Michalis K. Titsias, Jonathan Schwarz, Alexander G. de G. Matthews, Razvan Pascanu, Yee Whye Teh
We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network.
no code implementations • 30 Sep 2018 • Michalis K. Titsias, Sotirios Nikoloutsopoulos
The resulting method is flexible and it can be easily incorporated to any standard off-policy and on-policy algorithms, such as those based on temporal differences and policy gradients.
1 code implementation • 6 Aug 2018 • Michalis K. Titsias, Francisco J. R. Ruiz
We develop unbiased implicit variational inference (UIVI), a method that expands the applicability of variational inference by defining an expressive variational family.
no code implementations • 6 Jul 2018 • Aristeidis Panos, Petros Dellaportas, Michalis K. Titsias
We introduce fully scalable Gaussian processes, an implementation scheme that tackles the problem of treating a high number of training instances together with high dimensional input data.
1 code implementation • ICML 2018 • Francisco J. R. Ruiz, Michalis K. Titsias, Adji B. Dieng, David M. Blei
It maximizes a lower bound on the marginal likelihood of the data.
no code implementations • 4 Aug 2017 • Michalis K. Titsias
We introduce a new algorithm for approximate inference that combines reparametrization, Markov chain Monte Carlo and variational methods.
no code implementations • 24 Mar 2017 • Kaspar Märtens, Michalis K. Titsias, Christopher Yau
Bayesian inference for factorial hidden Markov models is challenging due to the exponentially sized latent variable space.
no code implementations • ICML 2017 • Tammo Rukat, Chris C. Holmes, Michalis K. Titsias, Christopher Yau
Boolean matrix factorisation aims to decompose a binary data matrix into an approximate Boolean product of two low rank, binary matrices: one containing meaningful patterns, the other quantifying how the observations can be expressed as a combination of these patterns.
1 code implementation • 30 Oct 2016 • Michalis K. Titsias, Omiros Papaspiliopoulos
We introduce a new family of MCMC samplers that combine auxiliary variables, Gibbs sampling and Taylor expansions of the target density.
no code implementations • NeurIPS 2016 • Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei
The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective.
no code implementations • NeurIPS 2016 • Michalis K. Titsias
The softmax representation of probabilities for categorical variables plays a prominent role in modern machine learning with numerous applications in areas such as large scale classification, neural language modeling and recommendation systems.
no code implementations • 3 Mar 2016 • Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei
Instead of taking samples from the variational distribution, we use importance sampling to take samples from an overdispersed distribution in the same exponential family as the variational approximation.
no code implementations • NeurIPS 2015 • Rémi Bardenet, Michalis K. Titsias
DPPs possess desirable properties, such as exact sampling or analyticity of the moments, but learning the parameters of kernel $K$ through likelihood-based inference is not straightforward.
no code implementations • 4 Mar 2015 • Michalis K. Titsias
We introduce local expectation gradients which is a general purpose stochastic variational inference algorithm for constructing stochastic gradients through sampling from the variational distribution.
no code implementations • 8 Sep 2014 • Andreas C. Damianou, Michalis K. Titsias, Neil D. Lawrence
The Gaussian process latent variable model (GP-LVM) provides a flexible approach for non-linear dimensionality reduction that has been widely applied.
no code implementations • 5 Nov 2013 • Michalis K. Titsias, Christopher Yau, Christopher C. Holmes
Hidden Markov models (HMMs) are one of the most widely used statistical methods for analyzing sequence data.
no code implementations • NeurIPS 2011 • Michalis K. Titsias, Miguel Lázaro-Gredilla
We introduce a variational Bayesian inference algorithm which can be widely applied to sparse linear models.
no code implementations • NeurIPS 2011 • Andreas Damianou, Michalis K. Titsias, Neil D. Lawrence
Our work builds on recent variational approximations for Gaussian process latent variable models to allow for nonlinear dimensionality reduction simultaneously with learning a dynamical prior in the latent space.
no code implementations • NeurIPS 2008 • Neil D. Lawrence, Magnus Rattray, Michalis K. Titsias
We describe an efficient Markov chain Monte Carlo algorithm for sampling from the posterior process of the GP model.
no code implementations • NeurIPS 2007 • Michalis K. Titsias
This model can play the role of the prior in an nonparametric Bayesian learning scenario where both the latent features and the number of their occurrences are unknown.