CommitBench: A Benchmark for Commit Message Generation

1 code implementation • 8 Mar 2024 • Maximilian Schall, Tamara Czinczoll, Gerard de Melo

Writing commit messages is a tedious daily task for many software developers, and often remains neglected.

NextLevelBERT: Investigating Masked Language Modeling with Higher-Level Representations for Long Documents

1 code implementation • 27 Feb 2024 • Tamara Czinczoll, Christoph Hönes, Maximilian Schall, Gerard de Melo

While (large) language models have significantly improved over the last years, they still struggle to sensibly process long sequences found, e. g., in books, due to the quadratic scaling of the underlying attention mechanism.

Document Classification Language Modelling +4

Scientific and Creative Analogies in Pretrained Language Models

1 code implementation • 28 Nov 2022 • Tamara Czinczoll, Helen Yannakoudakis, Pushkar Mishra, Ekaterina Shutova

This paper examines the encoding of analogy in large-scale pretrained language models, such as BERT and GPT-2.