3 code implementations • 11 May 2022 • Daniel Hesslow, Niccoló Zanichelli, Pascal Notin, Iacopo Poli, Debora Marks
In this work we introduce RITA: a suite of autoregressive generative models for protein sequences, with up to 1. 2 billion parameters, trained on over 280 million protein sequences belonging to the UniRef-100 database.
no code implementations • LREC 2022 • Julien Launay, E. L. Tommasone, Baptiste Pannier, François Boniface, Amélie Chatelain, Alessandro Cappelli, Iacopo Poli, Djamé Seddah
We fit a scaling law for compute for the French language, and compare it with its English counterpart.
no code implementations • NeurIPS Workshop ICBINB 2021 • Amélie Chatelain, Amine Djeghri, Daniel Hesslow, Julien Launay, Iacopo Poli
Recent work has identified simple empirical scaling laws for language models, linking compute budget, dataset size, model size, and autoregressive modeling loss.
no code implementations • ICML Workshop AML 2021 • Alessandro Cappelli, Julien Launay, Laurent Meunier, Ruben Ohana, Iacopo Poli
Robustness to adversarial attacks is typically obtained through expensive adversarial training with Projected Gradient Descent.
no code implementations • NeurIPS 2021 • Ruben Ohana, Hamlet J. Medina Ruiz, Julien Launay, Alessandro Cappelli, Iacopo Poli, Liva Ralaivola, Alain Rakotomamonjy
Optical Processing Units (OPUs) -- low-power photonic chips dedicated to large scale random projections -- have been used in previous work to train deep neural networks using Direct Feedback Alignment (DFA), an effective alternative to backpropagation.
no code implementations • 29 Apr 2021 • Daniel Hesslow, Alessandro Cappelli, Igor Carron, Laurent Daudet, Raphaël Lafargue, Kilian Müller, Ruben Ohana, Gustave Pariente, Iacopo Poli
Randomized Numerical Linear Algebra (RandNLA) is a powerful class of methods, widely used in High Performance Computing (HPC).
1 code implementation • 8 Feb 2021 • Daniel Hesslow, Iacopo Poli
The performance of algorithms for neural architecture search strongly depends on the parametrization of the search space.
1 code implementation • 6 Jan 2021 • Alessandro Cappelli, Ruben Ohana, Julien Launay, Laurent Meunier, Iacopo Poli, Florent Krzakala
In the white-box setting, our defense works by obfuscating the parameters of the random projection.
no code implementations • 11 Dec 2020 • Julien Launay, Iacopo Poli, Kilian Müller, Gustave Pariente, Igor Carron, Laurent Daudet, Florent Krzakala, Sylvain Gigan
We present a photonic accelerator for Direct Feedback Alignment, able to compute random projections with trillions of parameters.
1 code implementation • NeurIPS 2020 • Julien Launay, Iacopo Poli, François Boniface, Florent Krzakala
Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea.
1 code implementation • 15 Jun 2020 • Amélie Chatelain, Giuseppe Luca Tommasone, Laurent Daudet, Iacopo Poli
In this work, we focus on the identification of such events given many noisy observables.
no code implementations • 2 Jun 2020 • Julien Launay, Iacopo Poli, Kilian Müller, Igor Carron, Laurent Daudet, Florent Krzakala, Sylvain Gigan
As neural networks grow larger and more complex and data-hungry, training costs are skyrocketing.
2 code implementations • 11 Jun 2019 • Julien Launay, Iacopo Poli, Florent Krzakala
In this work, we focus on direct feedback alignment and present a set of best practices justified by observations of the alignment angles.
1 code implementation • 21 May 2018 • Nicolas Keriven, Damien Garreau, Iacopo Poli
We consider the problem of detecting abrupt changes in the distribution of a multi-dimensional time series, with limited computing power and memory.