no code implementations • 13 Feb 2024 • Daniel D. Johnson, Daniel Tarlow, David Duvenaud, Chris J. Maddison
Identifying how much a model ${\widehat{p}}_{\theta}(Y|X)$ knows about the stochastic real-world process $p(Y|X)$ it was trained on is important to ensure it avoids producing incorrect or "hallucinated" answers or taking unsafe actions.
1 code implementation • 1 Mar 2023 • Daniel D. Johnson, Daniel Tarlow, Christian Walder
Large language models show impressive results at predicting structured text such as code, but also commonly introduce errors and hallucinations in their output.
1 code implementation • 15 Aug 2022 • David Bieber, Kensen Shi, Petros Maniatis, Charles Sutton, Vincent Hellendoorn, Daniel Johnson, Daniel Tarlow
Graph representations of programs are commonly a central element of machine learning for code research.
no code implementations • 9 Aug 2022 • Binghong Chen, Daniel Tarlow, Kevin Swersky, Martin Maas, Pablo Heiber, Ashish Naik, Milad Hashemi, Parthasarathy Ranganathan
To automatically learn these hints from the dataset, we propose a novel discrete variational auto-encoder, where each discrete latent variable represents a different learned category of code-edit that increases performance.
1 code implementation • 26 Jun 2022 • Disha Shrivastava, Hugo Larochelle, Daniel Tarlow
With the success of large language models (LLMs) of code and their use as code assistants (e. g. Codex used in GitHub Copilot), techniques for introducing domain-specific knowledge in the prompt design process become important.
1 code implementation • 7 Mar 2022 • David Bieber, Rishab Goel, Daniel Zheng, Hugo Larochelle, Daniel Tarlow
This presents an interesting machine learning challenge: can we predict runtime errors in a "static" setting, where program execution is not possible?
1 code implementation • NeurIPS 2021 • Zimin Chen, Vincent Hellendoorn, Pascal Lamblin, Petros Maniatis, Pierre-Antoine Manzagol, Daniel Tarlow, Subhodeep Moitra
Machine learning for understanding and editing source code has recently attracted significant interest, with many developments in new models, new code representations, and new tasks. This proliferation can appear disparate and disconnected, making each approach seemingly unique and incompatible, thus obscuring the core machine learning challenges and contributions. In this work, we demonstrate that the landscape can be significantly simplified by taking a general approach of mapping a graph to a sequence of tokens and pointers. Our main result is to show that 16 recently published tasks of different shapes can be cast in this form, based on which a single model architecture achieves near or above state-of-the-art results on nearly all tasks, outperforming custom models like code2seq and alternative generic models like Transformers. This unification further enables multi-task learning and a series of cross-cutting experiments about the importance of different modeling choices for code understanding and repair tasks. The full framework, called PLUR, is easily extensible to more tasks, and will be open-sourced (https://github. com/google-research/plur).
1 code implementation • NeurIPS 2021 • Guy Lorberbom, Daniel D. Johnson, Chris J. Maddison, Daniel Tarlow, Tamir Hazan
To perform counterfactual reasoning in Structural Causal Models (SCMs), one needs to know the causal mechanisms, which provide factorizations of conditional distributions into noise sources and deterministic functions mapping realizations of noise to samples.
no code implementations • ICML Workshop INNF 2021 • Daniel D. Johnson, Jacob Austin, Rianne van den Berg, Daniel Tarlow
Denoising diffusion probabilistic models (DDPMs) have shown impressive results on sequence generation by iteratively corrupting each example and then learning to map corrupted versions back to the original.
3 code implementations • NeurIPS 2021 • Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, Rianne van den Berg
Here, we introduce Discrete Denoising Diffusion Probabilistic Models (D3PMs), diffusion-like generative models for discrete data that generalize the multinomial diffusion model of Hoogeboom et al. 2021, by going beyond corruption processes with uniform transition probabilities.
1 code implementation • NeurIPS 2021 • Disha Shrivastava, Hugo Larochelle, Daniel Tarlow
Most learning-based approaches try to find a program that satisfies all examples at once.
no code implementations • 28 May 2021 • Xuechen Li, Chris J. Maddison, Daniel Tarlow
Source code spends most of its time in a broken or incomplete state during software development.
1 code implementation • NeurIPS 2020 • David Bieber, Charles Sutton, Hugo Larochelle, Daniel Tarlow
More practically, we evaluate these models on the task of learning to execute partial programs, as might arise if using the model as a heuristic function in program synthesis.
no code implementations • NeurIPS Workshop CAP 2020 • Disha Shrivastava, Hugo Larochelle, Daniel Tarlow
The ability to adapt to unseen, local contexts is an important challenge that successful models of source code must overcome.
1 code implementation • NeurIPS 2020 • Daniel D. Johnson, Hugo Larochelle, Daniel Tarlow
In practice, edges are used both to represent intrinsic structure (e. g., abstract syntax trees of programs) and more abstract relations that aid reasoning for a downstream task (e. g., results of relevant program analyses).
no code implementations • 2 Jul 2020 • Kian Ahrabian, Daniel Tarlow, Hehuimin Cheng, Jin L. C. Guo
We present a multi-relational temporal Knowledge Graph based on the daily interactions between artifacts in GitHub, one of the largest social coding platforms.
1 code implementation • NeurIPS 2020 • Max B. Paulus, Dami Choi, Daniel Tarlow, Andreas Krause, Chris J. Maddison
The Gumbel-Max trick is the basis of many relaxed gradient estimators.
no code implementations • 26 Mar 2020 • Disha Shrivastava, Hugo Larochelle, Daniel Tarlow
The ability to adapt to unseen, local contexts is an important challenge that successful models of source code must overcome.
no code implementations • 4 Nov 2019 • Daniel Tarlow, Subhodeep Moitra, Andrew Rice, Zimin Chen, Pierre-Antoine Manzagol, Charles Sutton, Edward Aftandilian
A diff specifies how to modify the code's abstract syntax tree, represented in the neural network as a sequence of tokens and of pointers to code locations.
no code implementations • 27 Jun 2019 • Matej Balog, Bart van Merriënboer, Subhodeep Moitra, Yujia Li, Daniel Tarlow
Graph neural networks have become increasingly popular in recent years due to their ability to naturally encode relational input data and their ability to scale to large graphs by operating on a sparse representation of graph adjacency matrices.
no code implementations • ICLR 2020 • Zhan Shi, Kevin Swersky, Daniel Tarlow, Parthasarathy Ranganathan, Milad Hashemi
In this work, we propose a new approach to use GNNs to learn fused representations of general source code and its execution.
no code implementations • NeurIPS 2020 • Guy Lorberbom, Chris J. Maddison, Nicolas Heess, Tamir Hazan, Daniel Tarlow
A main benefit of DirPG algorithms is that they allow the insertion of domain knowledge in the form of upper bounds on return-to-go at training time, like is used in heuristic search, while still directly computing a policy gradient.
no code implementations • 4 Apr 2019 • Rui Zhao, David Bieber, Kevin Swersky, Daniel Tarlow
In this work, we instead treat source code as a dynamic object and tackle the problem of modeling the edits that software developers make to source code files.
1 code implementation • ICLR 2018 • Renjie Liao, Marc Brockschmidt, Daniel Tarlow, Alexander L. Gaunt, Raquel Urtasun, Richard Zemel
We present graph partition neural networks (GPNN), an extension of graph neural networks (GNNs) able to handle extremely large graphs.
1 code implementation • ICLR 2018 • Alexander L. Gaunt, Matthew A. Johnson, Maik Riechert, Daniel Tarlow, Ryota Tomioka, Dimitrios Vytiniotis, Sam Webster
Through an implementation on multi-core CPUs, we show that AMP training converges to the same accuracy as conventional synchronous training algorithms in a similar number of epochs, but utilizes the available hardware more efficiently even for small minibatch sizes, resulting in significantly shorter overall training times.
no code implementations • 10 Feb 2017 • Kirthevasan Kandasamy, Yoram Bachrach, Ryota Tomioka, Daniel Tarlow, David Carter
We study reinforcement learning of chatbots with recurrent neural network architectures when the rewards are noisy and expensive to obtain.
no code implementations • 2 Dec 2016 • Alexander L. Gaunt, Marc Brockschmidt, Rishabh Singh, Nate Kushman, Pushmeet Kohli, Jonathan Taylor, Daniel Tarlow
A TerpreT model is composed of a specification of a program representation and an interpreter that describes how programs map inputs to outputs.
3 code implementations • 7 Nov 2016 • Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, Daniel Tarlow
We develop a first line of attack for solving programming competition-style problems from input-output examples using deep learning.
no code implementations • ICML 2017 • Alexander L. Gaunt, Marc Brockschmidt, Nate Kushman, Daniel Tarlow
We develop a framework for combining differentiable programming languages with neural networks.
1 code implementation • 7 Nov 2016 • John K. Feser, Marc Brockschmidt, Alexander L. Gaunt, Daniel Tarlow
Recent work on differentiable interpreters relaxes the discrete space of programs into a continuous space so that search over programs can be performed using gradient-based optimization.
no code implementations • 15 Aug 2016 • Alexander L. Gaunt, Marc Brockschmidt, Rishabh Singh, Nate Kushman, Pushmeet Kohli, Jonathan Taylor, Daniel Tarlow
TerpreT is similar to a probabilistic programming language: a model is composed of a specification of a program representation (declarations of random variables) and an interpreter describing how programs map inputs to outputs (a model connecting unknowns to observations).
no code implementations • CVPR 2016 • David Joseph Tan, Thomas Cashman, Jonathan Taylor, Andrew Fitzgibbon, Daniel Tarlow, Sameh Khamis, Shahram Izadi, Jamie Shotton
We present a fast, practical method for personalizing a hand shape basis to an individual user's detailed hand shape using only a small set of depth images.
13 code implementations • 17 Nov 2015 • Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard Zemel
Graph-structured data appears frequently in domains including chemistry, natural language semantics, social networks, and knowledge bases.
Ranked #1 on Graph Classification on IPC-grounded
no code implementations • 10 Dec 2014 • Faruk Ahmed, Daniel Tarlow, Dhruv Batra
The result is that we can use loss-aware prediction methodology to improve performance of the highly tuned pipeline system.
no code implementations • NeurIPS 2014 • S. M. Ali Eslami, Daniel Tarlow, Pushmeet Kohli, John Winn
Much of research in machine learning has centered around the search for inference algorithms that are both general-purpose and efficient.
no code implementations • NeurIPS 2014 • Chris J. Maddison, Daniel Tarlow, Tom Minka
The problem of drawing samples from a discrete distribution can be converted into a discrete optimization problem.
no code implementations • 27 Oct 2014 • Varun Jampani, S. M. Ali Eslami, Daniel Tarlow, Pushmeet Kohli, John Winn
Generative models provide a powerful framework for probabilistic reasoning.
no code implementations • CVPR 2014 • Vittal Premachandran, Daniel Tarlow, Dhruv Batra
When building vision systems that predict structured objects such as image segmentations or human poses, a crucial concern is performance under task-specific evaluation measures (e. g. Jaccard Index or Average Precision).
no code implementations • 2 Jan 2014 • Chris J. Maddison, Daniel Tarlow
We study the problem of building generative models of natural source code (NSC); that is, source code written and understood by humans.
no code implementations • 19 Dec 2013 • Robert Nishihara, Thomas Minka, Daniel Tarlow
Probabilistic models often have parameters that can be translated, scaled, permuted, or otherwise transformed without changing the model.
no code implementations • NeurIPS 2013 • Nicolas Heess, Daniel Tarlow, John Winn
Expectation Propagation (EP) is a popular approximate posterior inference algorithm that often provides a fast and accurate alternative to sampling-based methods.
no code implementations • 26 Sep 2013 • Elad Mezuman, Daniel Tarlow, Amir Globerson, Yair Weiss
In this work, we study the LP relaxations that result from enforcing additional consistency constraints between the HOP and the rest of the model.
no code implementations • CVPR 2013 • Yujia Li, Daniel Tarlow, Richard Zemel
In this work, we study the learning of a general class of pattern-like high order potential, which we call Compositional High Order Pattern Potentials (CHOPPs).
no code implementations • NeurIPS 2012 • Kevin Swersky, Ilya Sutskever, Daniel Tarlow, Richard S. Zemel, Ruslan R. Salakhutdinov, Ryan P. Adams
The Restricted Boltzmann Machine (RBM) is a popular density model that is also good for extracting features.