Sparse Distributed Memory is a Continual Learner

1 code implementation • 20 Mar 2023 • Trenton Bricken, Xander Davies, Deepak Singh, Dmitry Krotov, Gabriel Kreiman

Continual learning is a problem for artificial neural networks that their biological counterparts are adept at solving.

Continual Learning

Attention Approximates Sparse Distributed Memory

1 code implementation • NeurIPS 2021 • Trenton Bricken, Cengiz Pehlevan

While Attention has come to be an important mechanism in deep learning, there remains limited intuition for why it works so well.