A low latency attention module for streaming self-supervised speech representation learning

1 code implementation • 27 Feb 2023 • Jianbo Ma, Siqi Pan, Deepak Chandran, Andrea Fanelli, Richard Cartwright

The SA represents our proposal for an efficient streaming SSRL implementation, while the LLSA solves the latency build-up problem of other streaming attention architectures, such as the masked acausal attention (MAA), guaranteeing a latency equal to one layer even when multiple layers are stacked.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3