Search Results for author: Matthew Farrugia-Roberts

Found 3 papers, 0 papers with code

The Developmental Landscape of In-Context Learning

no code implementations4 Feb 2024 Jesse Hoogland, George Wang, Matthew Farrugia-Roberts, Liam Carroll, Susan Wei, Daniel Murfet

We show that in-context learning emerges in transformers in discrete developmental stages, when they are trained on either language modeling or linear regression tasks.

In-Context Learning Language Modelling +1

Proximity to Losslessly Compressible Parameters

no code implementations5 Jun 2023 Matthew Farrugia-Roberts

In the setting of single-hidden-layer hyperbolic tangent networks, we define the rank of a parameter as the minimum number of hidden units required to implement the same function.

Cannot find the paper you are looking for? You can Submit a new open access paper.