Search Results for author:
Found 6 papers, 5 papers with code
Towards Modular LLMs by Building and Reusing a Library of LoRAs
The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trained adapters to improve performance for new tasks.
Language Modeling Using Tensor Trains
We propose a novel tensor network language model based on the simplest tensor network (i. e., tensor trains), called `Tensor Train Language Model' (TTLM).
History-Aware Conversational Dense Retrieval
To address the aforementioned issues, we propose a History-Aware Conversational Dense Retrieval (HAConvDR) system, which incorporates two ideas: context-denoised query reformulation and automatic mining of supervision signals based on the actual impact of historical turns.
Multi-Head Adapter Routing for Cross-Task Generalization
We find that routing is most beneficial during multi-task pre-training rather than during few-shot adaptation and propose $\texttt{MHR}$-$\mu$, which discards routing and fine-tunes the average of the pre-trained adapters on each downstream tasks.
A Generalized Language Model in Tensor Space
Theoretically, we prove that such tensor representation is a generalization of the n-gram language model.
A Quantum Many-body Wave Function Inspired Language Modeling Approach
The recently proposed quantum language model (QLM) aimed at a principled approach to modeling term dependency by applying the quantum probability theory.
