Search Results for author:
Found 10 papers, 1 papers with code
SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models
Despite safety guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs to adversarial perturbations and transfer attacks, with average attack success rates of 90% and 10% respectively when evaluated on a dataset of carefully designed harmful questions spanning 12 different toxic categories.
SpeechVerse: A Large-scale Generalizable Audio Language Model
The models are instruction finetuned using continuous latent representations extracted from the speech foundation model to achieve optimal zero-shot performance on a diverse range of speech processing tasks using natural language instructions.
End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation
Conventional speech-to-text translation (ST) systems are trained on single-speaker utterances, and they may not generalize to real-life scenarios where the audio contains conversations by multiple speakers.
Speaker Diarization of Scripted Audiovisual Content
The media localization industry usually requires a verbatim script of the final film or TV production in order to create subtitles or dubbing scripts in a foreign language.
Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction
Speaker diarization (SD) is typically used with an automatic speech recognition (ASR) system to ascribe speaker labels to recognized words.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Device Directedness with Contextual Cues for Spoken Dialog Systems
In this work, we define barge-in verification as a supervised learning task where audio-only information is used to classify user spoken dialogue into true and false barge-ins.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech
Also, most of these models are trained with synthetic mixtures and do not generalize to real conversational data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Representation learning through cross-modal conditional teacher-student training for speech emotion recognition
To improve the efficacy of our approach, we propose a novel estimate of the quality of the emotion predictions, to condition teacher-student training.
Speaker-conversation factorial designs for diarization error analysis
Determining the cause of diarization errors is difficult because speaker voice acoustics and conversation structure co-vary, and the interactions between acoustics, conversational structure, and diarization accuracy are complex.
Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learning
Training deep neural networks for automatic speech recognition (ASR) requires large amounts of transcribed speech.
