no code implementations • 6 Feb 2024 • Zach Furman, Edmund Lau
The \textit{local learning coefficient} (LLC) is a principled way of quantifying model complexity, originally derived in the context of Bayesian statistics using singular learning theory (SLT).
2 code implementations • 14 Mar 2023 • Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, Jacob Steinhardt
We analyze transformers from the perspective of iterative inference, seeking to understand how model predictions are refined layer by layer.