2 code implementations • 27 Feb 2024 • Ruby Sedgwick, John P. Goertz, Molly M. Stevens, Ruth Misener, Mark van der Wilk
With the rise in engineered biomolecular devices, there is an increased need for tailor-made biological sequences.
no code implementations • 15 Feb 2024 • Sebastian W. Ober, Artem Artemev, Marcel Wagenländer, Rudolfs Grobins, Mark van der Wilk
To address this, we make recommendations for comparing GP approximations based on a specification of what a user should expect from a method.
no code implementations • 13 Feb 2024 • Jose Pablo Folch, Calvin Tsay, Robert M Lee, Behrang Shafei, Weronika Ormaniec, Andreas Krause, Mark van der Wilk, Ruth Misener, Mojmír Mutný
Bayesian optimization is a methodology to optimize black-box functions.
1 code implementation • 22 Dec 2023 • Shahin Honarvar, Mark van der Wilk, Alastair Donaldson
Thus, from a single question template, it is possible to ask an LLM a $\textit{neighbourhood}$ of very similar programming questions, and assess the correctness of the result returned for each question.
Ranked #1 on Code Generation on Turbulence
no code implementations • 1 Dec 2023 • Jose Pablo Folch, James Odgers, Shiqiang Zhang, Robert M Lee, Behrang Shafei, David Walz, Calvin Tsay, Mark van der Wilk, Ruth Misener
There has been a surge in interest in data-driven experimental design with applications to chemical engineering and drug manufacturing.
no code implementations • 24 Nov 2023 • Seth Nabarro, Mark van der Wilk, Andrew J Davison
We propose an approach to do learning in Gaussian factor graphs.
no code implementations • 25 Jul 2023 • Jacob Green, Cecilia Cabrera Diaz, Maximilian A. H. Jakobs, Andrea Dimitracopoulos, Mark van der Wilk, Ryan D. Greenhalgh
However, it remains unclear for practitioners which method or approach is most suitable, as different papers benchmark on different datasets and methods, leading to varying conclusions that are not easily compared.
1 code implementation • 6 Jun 2023 • Alexander Immer, Tycho F. A. van der Ouderaa, Mark van der Wilk, Gunnar Rätsch, Bernhard Schölkopf
Recent works show that Bayesian model selection with Laplace approximations can allow to optimize such hyperparameters just like standard neural network parameters using gradients and on the training data.
no code implementations • 5 Jun 2023 • Anish Dhir, Mark van der Wilk
With only observational data on two variables, and without other assumptions, it is not possible to infer which one causes the other.
1 code implementation • 11 Apr 2023 • Harry Jake Cunningham, Daniel Augusto de Souza, So Takao, Mark van der Wilk, Marc Peter Deisenroth
For large datasets, sparse GPs reduce these demands by conditioning on a small set of inducing variables designed to summarise the data.
1 code implementation • 11 Nov 2022 • Jose Pablo Folch, Robert M Lee, Behrang Shafei, David Walz, Calvin Tsay, Mark van der Wilk, Ruth Misener
Bayesian Optimization is a useful tool for experiment design.
1 code implementation • 14 Oct 2022 • Alexander Terenin, David R. Burt, Artem Artemev, Seth Flaxman, Mark van der Wilk, Carl Edward Rasmussen, Hong Ge
For low-dimensional tasks such as geospatial modeling, we propose an automated method for computing inducing points satisfying these conditions.
1 code implementation • 28 Jun 2022 • Artem Artemev, Tilman Roeder, Mark van der Wilk
We believe that further focus on removing memory constraints at a compiler level will widen the range of machine learning methods that can be developed in the future.
no code implementations • 14 Apr 2022 • Tycho F. A. van der Ouderaa, David W. Romero, Mark van der Wilk
Equivariances provide useful inductive biases in neural network modeling, with the translation equivariance of convolutional neural networks being a canonical example.
no code implementations • 25 Feb 2022 • Tycho F. A. van der Ouderaa, Mark van der Wilk
Assumptions about invariances or symmetries in data can significantly increase the predictive power of statistical models.
1 code implementation • 22 Feb 2022 • Alexander Immer, Tycho F. A. van der Ouderaa, Gunnar Rätsch, Vincent Fortuin, Mark van der Wilk
We develop a convenient gradient-based method for selecting the data augmentation without validation data during training of a deep neural network.
2 code implementations • 31 Jan 2022 • Jose Pablo Folch, Shiqiang Zhang, Robert M Lee, Behrang Shafei, David Walz, Calvin Tsay, Mark van der Wilk, Ruth Misener
Bayesian Optimization is a very effective tool for optimizing expensive black-box functions.
no code implementations • pproximateinference AABI Symposium 2022 • Mark van der Wilk, Artem Artemev, James Hensman
The need for matrix decompositions (inverses) is often named as a major impediment to scaling Gaussian process (GP) models, even in efficient approximations.
no code implementations • pproximateinference AABI Symposium 2022 • Sebastian Popescu, Ben Glocker, Mark van der Wilk
We propose a new variational lower bound for performing inference in sparse Student's T Processes that does not require computationally intensive operations such as matrix inversions or log determinants of matrices.
no code implementations • NeurIPS Workshop ICBINB 2021 • David R. Burt, Artem Artemev, Mark van der Wilk
We suggest a method that adaptively selects the amount of computation to use when estimating the log marginal likelihood so that the bias of the objective function is guaranteed to be small.
no code implementations • 20 Jul 2021 • Nikolaos Mourdoukoutas, Marco Federici, Georges Pantalos, Mark van der Wilk, Vincent Fortuin
We propose a novel Bayesian neural network architecture that can learn invariances from data alone by inferring a posterior distribution over different weight-sharing schemes.
1 code implementation • 14 Jun 2021 • Pola Schwöbel, Martin Jørgensen, Sebastian W. Ober, Mark van der Wilk
Computing the marginal likelihood is hard for neural networks, but success with tractable approaches that compute the marginal likelihood for the last layer only raises the question of whether this convenient approach might be employed for learning invariances.
no code implementations • 10 Jun 2021 • Seth Nabarro, Stoil Ganev, Adrià Garriga-Alonso, Vincent Fortuin, Mark van der Wilk, Laurence Aitchison
Here, we provide several approaches to developing principled Bayesian neural networks incorporating data augmentation.
1 code implementation • 14 May 2021 • Vincent Fortuin, Adrià Garriga-Alonso, Mark van der Wilk, Laurence Aitchison
Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance.
no code implementations • NeurIPS 2021 • Vincent Dutordoir, James Hensman, Mark van der Wilk, Carl Henrik Ek, Zoubin Ghahramani, Nicolas Durrande
This results in models that can either be seen as neural networks with improved uncertainty prediction or deep Gaussian processes with increased prediction accuracy.
1 code implementation • 12 Apr 2021 • Vincent Dutordoir, Hugh Salimbeni, Eric Hambro, John McLeod, Felix Leibfried, Artem Artemev, Mark van der Wilk, James Hensman, Marc P. Deisenroth, ST John
GPflux is compatible with and built on top of the Keras deep learning eco-system.
no code implementations • 24 Feb 2021 • Sebastian W. Ober, Carl E. Rasmussen, Mark van der Wilk
Through careful experimentation on the UCI, CIFAR-10, and the UTKFace datasets, we find that the overfitting from overparameterized maximum marginal likelihood, in which the model is "somewhat Bayesian", can in certain scenarios be worse than that from not being Bayesian at all.
no code implementations • 16 Feb 2021 • Artem Artemev, David R. Burt, Mark van der Wilk
We propose a lower bound on the log marginal likelihood of Gaussian process regression models that can be computed without matrix factorisation of the full kernel matrix.
1 code implementation • NeurIPS Workshop ICBINB 2020 • Vincent Fortuin, Adrià Garriga-Alonso, Sebastian W. Ober, Florian Wenzel, Gunnar Rätsch, Richard E. Turner, Mark van der Wilk, Laurence Aitchison
Isotropic Gaussian priors are the de facto standard for modern Bayesian neural network inference.
no code implementations • pproximateinference AABI Symposium 2021 • Adrià Garriga-Alonso, Mark van der Wilk
Infinite width limits of deep neural networks often have tractable forms.
no code implementations • 20 Nov 2020 • Ruby Sedgwick, John Goertz, Molly Stevens, Ruth Misener, Mark van der Wilk
There is a growing trend in molecular and synthetic biology of using mechanistic (non machine learning) models to design biomolecular networks.
2 code implementations • pproximateinference AABI Symposium 2021 • David R. Burt, Sebastian W. Ober, Adrià Garriga-Alonso, Mark van der Wilk
Then, we propose (featurized) Bayesian linear regression as a benchmark for `function-space' inference methods that directly measures approximation quality.
no code implementations • NeurIPS 2020 • Clare Lyle, Lisa Schut, Binxin Ru, Yarin Gal, Mark van der Wilk
This provides two major insights: first, that a measure of a model's training speed can be used to estimate its marginal likelihood.
no code implementations • 28 Sep 2020 • Binxin Ru, Clare Lyle, Lisa Schut, Mark van der Wilk, Yarin Gal
Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS).
1 code implementation • 1 Aug 2020 • David R. Burt, Carl Edward Rasmussen, Mark van der Wilk
Gaussian processes are distributions over functions that are versatile and mathematically convenient priors in Bayesian modelling.
no code implementations • 23 Jun 2020 • David R. Burt, Carl Edward Rasmussen, Mark van der Wilk
We present a construction of features for any stationary prior kernel that allow for computation of an unbiased estimator to the ELBO using $T$ Monte Carlo samples in $\mathcal{O}(\tilde{N}T+M^2T)$ and in $\mathcal{O}(\tilde{N}T+MT)$ with an additional approximation.
1 code implementation • NeurIPS 2020 • Miguel Monteiro, Loïc le Folgoc, Daniel Coelho de Castro, Nick Pawlowski, Bernardo Marques, Konstantinos Kamnitsas, Mark van der Wilk, Ben Glocker
In image segmentation, there is often more than one plausible solution for a given input.
2 code implementations • NeurIPS 2021 • Binxin Ru, Clare Lyle, Lisa Schut, Miroslav Fil, Mark van der Wilk, Yarin Gal
Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS).
no code implementations • 1 May 2020 • Clare Lyle, Mark van der Wilk, Marta Kwiatkowska, Yarin Gal, Benjamin Bloem-Reddy
Many real world data analysis problems exhibit invariant structure, and models that take advantage of this structure have shown impressive empirical performance, particularly in deep learning.
no code implementations • 7 Apr 2020 • Lewis Smith, Lisa Schut, Yarin Gal, Mark van der Wilk
'Capsule' models try to explicitly represent the poses of objects, enforcing a linear relationship between an object's pose and that of its constituent parts.
1 code implementation • 2 Mar 2020 • Mark van der Wilk, Vincent Dutordoir, ST John, Artem Artemev, Vincent Adam, James Hensman
One obstacle to the use of Gaussian processes (GPs) in large-scale problems, and as a component in deep learning system, is the need for bespoke derivations and implementations for small variations in the model or inference.
no code implementations • pproximateinference AABI Symposium 2019 • Mark van der Wilk, ST John, Artem Artemev, James Hensman
We present a variational approximation for a wide range of GP models that does not require a matrix inverse to be performed at each optimisation step.
1 code implementation • NeurIPS 2019 • Creighton Heaukulani, Mark van der Wilk
We implement gradient-based variational inference routines for Wishart and inverse Wishart processes, which we apply as Bayesian models for the dynamic, heteroskedastic covariance matrix of a multivariate time series.
1 code implementation • 13 Jun 2019 • Alessandro Davide Ialongo, Mark van der Wilk, James Hensman, Carl Edward Rasmussen
As we demonstrate in our experiments, the factorisation between latent system states and transition function can lead to a miscalibrated posterior and to learning unnecessarily large noise terms.
1 code implementation • 8 Mar 2019 • David R. Burt, Carl E. Rasmussen, Mark van der Wilk
Excellent variational approximations to Gaussian process posteriors have been developed which avoid the $\mathcal{O}\left(N^3\right)$ scaling with dataset size $N$.
no code implementations • 15 Feb 2019 • Vincent Dutordoir, Mark van der Wilk, Artem Artemev, James Hensman
We also demonstrate that our fully Bayesian approach improves on dropout-based Bayesian deep learning methods in terms of uncertainty and marginal likelihood estimates.
no code implementations • 14 Dec 2018 • Alessandro Davide Ialongo, Mark van der Wilk, James Hensman, Carl Edward Rasmussen
We focus on variational inference in dynamical systems where the discrete time transition function (or evolution rule) is modelled by a Gaussian process.
no code implementations • 10 Dec 2018 • Alessandro Davide Ialongo, Mark van der Wilk, Carl Edward Rasmussen
We examine an analytic variational inference scheme for the Gaussian Process State Space Model (GPSSM) - a probabilistic model for system identification and time-series modelling.
1 code implementation • NeurIPS 2019 • Dustin Tran, Michael W. Dusenberry, Mark van der Wilk, Danijar Hafner
We describe Bayesian Layers, a module designed for fast experimentation with neural network uncertainty.
no code implementations • NeurIPS 2018 • Mark van der Wilk, Matthias Bauer, ST John, James Hensman
Generalising well in supervised learning tasks relies on correctly extrapolating the training data to a large region of the input space.
4 code implementations • NeurIPS 2017 • Mark van der Wilk, Carl Edward Rasmussen, James Hensman
We present a practical way of introducing convolutional structure into Gaussian processes, making them more suited to high-dimensional inputs like images.
1 code implementation • 27 Oct 2016 • Alexander G. de G. Matthews, Mark van der Wilk, Tom Nickson, Keisuke Fujii, Alexis Boukouvalas, Pablo León-Villagrá, Zoubin Ghahramani, James Hensman
GPflow is a Gaussian process library that uses TensorFlow for its core computations and Python for its front end.
no code implementations • NeurIPS 2016 • Matthias Bauer, Mark van der Wilk, Carl Edward Rasmussen
Good sparse approximations are essential for practical inference in Gaussian Processes as the computational cost of exact methods is prohibitive for large datasets.
1 code implementation • NeurIPS 2014 • Yarin Gal, Mark van der Wilk, Carl E. Rasmussen
We show that GP performance improves with increasing amounts of data in regression (on flight data with 2 million records) and latent variable modelling (on MNIST).
no code implementations • 6 Feb 2014 • Yarin Gal, Mark van der Wilk
In this tutorial we explain the inference procedures developed for the sparse Gaussian process (GP) regression and Gaussian process latent variable model (GPLVM).