no code implementations • 28 Mar 2024 • Johannes Müller, Semih Çaycı, Guido Montúfar
Kakade's natural policy gradient method has been studied extensively in the last years showing linear convergence with and without regularization.
no code implementations • 18 Mar 2024 • Marie-Charlotte Brandenburg, Georg Loho, Guido Montúfar
The parameter space of ReLU neural networks is contained as a semialgebraic set inside the parameter space of tropical rational functions.
no code implementations • 11 Mar 2024 • Kedar Karhadkar, Erin George, Michael Murray, Guido Montúfar, Deanna Needell
The problem of benign overfitting asks whether it is possible for a model to perfectly fit noisy training data and still generalize well.
no code implementations • 31 May 2023 • Kedar Karhadkar, Michael Murray, Hanna Tseran, Guido Montúfar
We study the loss landscape of both shallow and deep, mildly overparameterized ReLU neural networks on a generic finite input dataset for the squared error loss.
no code implementations • 12 Apr 2023 • Kathlén Kohn, Guido Montúfar, Vahid Shahverdi, Matthew Trager
We study the geometry of linear networks with one-dimensional convolutional layers.
1 code implementation • 6 Mar 2023 • Pierre Bréchet, Katerina Papagiannouli, Jing An, Guido Montúfar
We consider a deep matrix factorization model of covariance matrices trained with the Bures-Wasserstein distance.
1 code implementation • 17 Jan 2023 • Hanna Tseran, Guido Montúfar
We study the gradients of a maxout network with respect to inputs and parameters and obtain bounds for the moments depending on the architecture and the parameter distribution.
no code implementations • 3 Nov 2022 • Johannes Müller, Guido Montúfar
We study the convergence of several natural policy gradient (NPG) methods in infinite-horizon discounted Markov decision processes with regular policy parametrizations.
1 code implementation • 21 Oct 2022 • Kedar Karhadkar, Pradeep Kr. Banerjee, Guido Montúfar
On the other hand, adding edges to the message-passing graph can lead to increasingly similar node representations and a problem known as oversmoothing.
no code implementations • 29 Sep 2022 • Laura Escobar, Patricio Gallardo, Javier González-Anaya, José L. González, Guido Montúfar, Alejandro H. Morales
We investigate the combinatorics of max-pooling layers, which are functions that downsample input arrays by taking the maximum over shifted windows of input coordinates, and which are commonly used in convolutional neural networks.
1 code implementation • 6 Aug 2022 • Pradeep Kr. Banerjee, Kedar Karhadkar, Yu Guang Wang, Uri Alon, Guido Montúfar
We compare the spectral expansion properties of our algorithm with that of an existing curvature-based non-local rewiring strategy.
1 code implementation • 21 Jun 2022 • Renata Turkeš, Guido Montúfar, Nina Otter
The goal of this work is to identify some types of problems where PH performs well or even better than other methods in data analysis.
1 code implementation • 27 May 2022 • Johannes Müller, Guido Montúfar
Reward optimization in fully observable Markov decision processes is equivalent to a linear program over the polytope of state-action frequencies.
no code implementations • 27 Oct 2021 • Dohyun Kwon, Yeoneung Kim, Guido Montúfar, Insoon Yang
We propose a stable method to train Wasserstein generative adversarial networks.
no code implementations • ICLR 2022 • Hui Jin, Pradeep Kr. Banerjee, Guido Montúfar
We characterize the power-law asymptotics of learning curves for Gaussian process regression (GPR) under the assumption that the eigenspectrum of the prior and the eigenexpansion coefficients of the target function follow a power law.
2 code implementations • ICLR 2022 • Johannes Müller, Guido Montúfar
We then describe the optimization problem as a linear optimization problem in the space of feasible state-action frequencies subject to polynomial constraints that we characterize explicitly.
no code implementations • 3 Aug 2021 • Kathlén Kohn, Thomas Merkh, Guido Montúfar, Matthew Trager
We study the family of functions that are represented by a linear convolutional neural network (LCN).
1 code implementation • NeurIPS 2021 • Hanna Tseran, Guido Montúfar
Learning with neural networks relies on the complexity of the representable functions, but more importantly, the particular assignment of typical parameters to functions of different complexity.
1 code implementation • NeurIPS 2021 • Cristian Bodnar, Fabrizio Frasca, Nina Otter, Yu Guang Wang, Pietro Liò, Guido Montúfar, Michael Bronstein
Nevertheless, these models can be severely constrained by the rigid combinatorial structure of Simplicial Complexes (SCs).
Ranked #1 on Graph Regression on ZINC 100k
no code implementations • 4 May 2021 • Pradeep Kr. Banerjee, Guido Montúfar
We present a unifying picture of PAC-Bayesian and mutual information-based upper bounds on the generalization error of randomized learning algorithms.
no code implementations • 16 Apr 2021 • Guido Montúfar, Yue Ren, Leon Zhang
We present results on the number of linear regions of the functions that can be represented by artificial feedforward neural networks with maxout units.
1 code implementation • ICLR Workshop GTRL 2021 • Cristian Bodnar, Fabrizio Frasca, Yu Guang Wang, Nina Otter, Guido Montúfar, Pietro Liò, Michael Bronstein
The pairwise interaction paradigm of graph machine learning has predominantly governed the modelling of relational systems.
no code implementations • NeurIPS Workshop TDA_and_Beyond 2020 • Guido Montúfar, Nina Otter, Yuguang Wang
Topological data analysis uses tools from topology -- the mathematical area that studies shapes -- to create representations of data.
no code implementations • 18 Jul 2020 • Guido Montúfar, Yu Guang Wang
Learning mappings of data on manifolds is an important topic in contemporary machine learning, with applications in astrophysics, geophysics, statistical physics, medical diagnosis, biochemistry, 3D object analysis.
1 code implementation • 12 Jun 2020 • Hui Jin, Guido Montúfar
\hj{For stochastic gradient descent we obtain the same implicit bias result.}
no code implementations • ICML 2020 • Yonatan Dukler, Quanquan Gu, Guido Montúfar
The success of deep neural networks is in part due to the use of normalization layers.
no code implementations • 22 Oct 2019 • Thomas Merkh, Guido Montúfar
We investigate different types of shallow and deep architectures, and the minimal number of layers and units per layer that are sufficient and necessary in order for the network to be able to approximate any given stochastic mapping from the set of inputs to the set of outputs arbitrarily well.
no code implementations • 9 Oct 2019 • Anton Mallasto, Guido Montúfar, Augusto Gerolin
Generative modelling is often cast as minimizing a similarity measure between a data distribution and a model distribution.
no code implementations • 27 Oct 2018 • Pradeep Kr. Banerjee, Guido Montúfar
We introduce a bottleneck method for learning data representations based on information deficiency, rather than the more traditional information sufficiency.
no code implementations • NeurIPS 2014 • Guido Montúfar, Razvan Pascanu, Kyunghyun Cho, Yoshua Bengio
We study the complexity of functions computable by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and the number of linear regions that they have.