Search Results for author:
Found 243 papers, 96 papers with code
TenTrans Large-Scale Multilingual Machine Translation System for WMT21
This paper describes TenTrans large-scale multilingual machine translation system for WMT 2021.
End-to-End Chinese Speaker Identification
End-to-end SI systems, on the other hand, are not limited by individual modules, but suffer from insufficient training data from the existing small-scale datasets.
Variational Graph Autoencoding as Cheap Supervision for AMR Coreference Resolution
Coreference resolution over semantic graphs like AMRs aims to group the graph nodes that represent the same entity.
RAST: Domain-Robust Dialogue Rewriting as Sequence Tagging
The task of dialogue rewriting aims to reconstruct the latest dialogue utterance by copying the missing content from the dialogue context.
Instance-adaptive training with noise-robust losses against noisy labels
In order to alleviate the huge demand for annotated datasets for different tasks, many recent natural language processing datasets have adopted automated pipelines for fast-tracking usable data.
面向人工智能伦理计算的中文道德词典构建方法研究(Construction of a Chinese Moral Dictionary for Artificial Intelligence Ethical Computing)
道德词典资源的建设是人工智能伦理计算的一个研究重点。由于道德行为复杂多样, 现有的英文道德词典分类体系并不完善, 而中文方面目前尚未有相关的词典资源, 理论体系和构建方法仍待探究。针对以上问题, 该文提出了面向人工智能伦理计算的中文道德词典构建任务, 设计了四类标签和四种类型, 得到包含25, 012个词的中文道德词典资源。实验结果表明, 该词典资源不仅能够使机器学会道德知识, 判断词的道德标签和类型, 而且能够为句子级别的道德文本分析提供数据支持。
结合深度学习和语言难度特征的句子可读性计算方法(The method of calculating sentence readability combined with deep learning and language difficulty characteristics)
本文提出了可读性语料库构建的改进方法, 基于该方法, 构建了规模更大的汉语句子可读性语料库。该语料库在句子绝对难度评估任务上的准确率达到0. 7869, 相对前人工作提升了0. 15以上, 证明了改进方法的有效性。将深度学习方法应用于汉语可读性评估, 探究了不同深度学习方法自动捕获难度特征的能力, 并进仛步探究了向深度学习特征中融入不同层面的语难度特征对模型整体性能的影响。实验结果显示, 不同深度学习模型的难度特征捕获能力不尽相同, 语言难度特征可以不同程度地提高深度学习模型的难度表征能力。
From Polarity to Intensity: Mining Morality from Semantic Space
It only needs moral polarity labels, which are more robust and easier to acquire.
Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
Despite the impressive capabilities of Large Language Models (LLMs) on various tasks, they still struggle with scenarios that involves complex reasoning and planning.
Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models
Current state-of-the-art approaches refine decoding by contrasting early-exit distributions from a lower layer with the final layer to exploit information related to factuality within the model forward procedure.
Polarity Calibration for Opinion Summarization
To address this issue and make the summarizer express both sides of opinions, we introduce the concept of polarity calibration, which aims to align the polarity of output summary with that of input text.
Conceptual and Unbiased Reasoning in Language Models
Conceptual reasoning, the ability to reason in abstract and high-level perspectives, is key to generalization in human cognition.
Self-Consistency Boosts Calibration for Math Reasoning
Calibration, which establishes the correlation between accuracy and model confidence, is important for LLM development.
A Knowledge Plug-and-Play Test Bed for Open-domain Dialogue Generation
In this paper, we present a high-quality benchmark named multi-source Wizard of Wikipedia (Ms. WoW) for evaluating multi-source dialogue knowledge selection and response generation.
Can Large Language Models do Analytical Reasoning?
Our analytical reasoning embodies the tasks of letting large language models count how many points each team scores in a quarter in the NBA and NFL games.
Collaborative decoding of critical tokens for boosting factuality of large language models
The most common training pipeline for large language models includes pretraining, finetuning and aligning phases, with their respective resulting models, such as the pretrained model and the finetuned model.
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
For a LLM to be trustworthy, its confidence level should be well-calibrated with its actual performance.
Fine-Grained Self-Endorsement Improves Factuality and Reasoning
Experiments on Biographies show that our method can effectively improve the factuality of generations with simple and intuitive prompts across different scales of LLMs.
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization
Scientific data visualization plays a crucial role in research by enabling the direct display of complex information and assisting researchers in identifying implicit patterns.
SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs
In this paper, we introduce four novel tasks centered around sports data analytics to evaluate the numerical reasoning and information fusion capabilities of LLMs.
Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
We consider the problem of multi-objective alignment of foundation models with human preferences, which is a critical step towards helpful and harmless AI systems.
SPECTRUM: Speaker-Enhanced Pre-Training for Long Dialogue Summarization
Multi-turn dialogues are characterized by their extended length and the presence of turn-taking conversations.
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
The rapid advancement of large language models (LLMs) has led to a new era marked by the development of autonomous applications in real-world scenarios, which drives innovation in creating advanced web agents.
MM-LLMs: Recent Advances in MultiModal Large Language Models
In the past year, MultiModal Large Language Models (MM-LLMs) have undergone substantial advancements, augmenting off-the-shelf LLMs to support MM inputs or outputs via cost-effective training strategies.
Inconsistent dialogue responses and how to recover from them
One critical issue for chat systems is to stay consistent about preferences, opinions, beliefs and facts of itself, which has been shown a difficult problem.
InFoBench: Evaluating Instruction Following Ability in Large Language Models
This paper introduces the Decomposed Requirements Following Ratio (DRFR), a new metric for evaluating Large Language Models' (LLMs) ability to follow instructions.
Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention
This paper introduces a novel approach to enhance the capabilities of Large Language Models (LLMs) in processing and understanding extensive text sequences, a critical aspect in applications requiring deep comprehension and synthesis of large volumes of information.
CLOMO: Counterfactual Logical Modification with Large Language Models
In this study, we delve into the realm of counterfactual reasoning capabilities of large language models (LLMs).
Deep Audio Zooming: Beamwidth-Controllable Neural Beamformer
Audio zooming, a signal processing technique, enables selective focusing and enhancement of sound signals from a specified region, attenuating others.
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning
Recognizing the need for a comprehensive evaluation of LMM chart understanding, we also propose a MultiModal Chart Benchmark (\textbf{MMC-Benchmark}), a comprehensive human-annotated benchmark with nine distinct tasks evaluating reasoning capabilities over charts.
Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
In response to these challenges, we introduces Chain-of-Noting (CoN), a novel approach aimed at improving the robustness of RALMs in facing noisy, irrelevant documents and in handling unknown scenarios.
A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning
In this paper, we take a closer look at the self-verification abilities of LLMs in the context of logical reasoning, focusing on their ability to identify logical fallacies accurately.
TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMs
We construct a hierarchical task tree encompassing 7 major areas covering over 200 categories and over 800 tasks, which covers diverse capabilities such as question answering, reasoning, multiturn dialogue, and text generation, to evaluate LLMs in a comprehensive and in-depth manner.
Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations
We introduce sub-sentence encoder, a contrastively-learned contextual embedding model for fine-grained semantic representation of text.
RIR-SF: Room Impulse Response Based Spatial Feature for Multi-channel Multi-talker ASR
Multi-channel multi-talker automatic speech recognition (ASR) presents ongoing challenges within the speech community, particularly when confronted with significant reverberation effects.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
UniX-Encoder: A Universal $X$-Channel Speech Encoder for Ad-Hoc Microphone Array Speech Processing
2) Multi-Task Capability: Beyond the single-task focus of previous systems, UniX-Encoder acts as a robust upstream model, adeptly extracting features for diverse tasks including ASR and speaker recognition.
On the Dimensionality of Sentence Embeddings
Therefore, we propose a two-step training method for sentence representation learning models, wherein the encoder and the pooler are optimized separately to mitigate the overall performance loss in low-dimension scenarios.
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation
Multimodal machine translation (MMT) simultaneously takes the source sentence and a relevant image as input for translation.
uSee: Unified Speech Enhancement and Editing with Conditional Diffusion Models
Speech enhancement aims to improve the quality of speech signals in terms of quality and intelligibility, and speech editing refers to the process of editing the speech according to specific user needs.
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
In this work, we investigate how the instruction tuning adjusts pre-trained models with a focus on intrinsic changes.
The Trickle-down Impact of Reward (In-)consistency on RLHF
We propose Contrast Instructions -- a benchmarking strategy for the consistency of RM.
Advancing Acoustic Howling Suppression through Recursive Training of Neural Networks
In this paper, we introduce a novel training framework designed to comprehensively address the acoustic howling issue by examining its fundamental formation process.
Neural Network Augmented Kalman Filter for Robust Acoustic Howling Suppression
Acoustic howling suppression (AHS) is a critical challenge in audio communication systems.
Proposition from the Perspective of Chinese Language: A Chinese Proposition Classification Evaluation Benchmark
Existing propositions often rely on logical constants for classification.
Stabilizing RLHF through Advantage Model and Selective Rehearsal
Large Language Models (LLMs) have revolutionized natural language processing, yet aligning these models with human values and preferences using RLHF remains a significant challenge.
Unifying Robustness and Fidelity: A Comprehensive Study of Pretrained Generative Methods for Speech Enhancement in Adverse Conditions
Enhancing speech signal quality in adverse acoustic environments is a persistent challenge in speech processing.
LASER: LLM Agent with State-Space Exploration for Web Navigation
We evaluate our proposed LLM Agent with State-Space ExploRation (LASER) on both the WebShop task and amazon. com.
Unsupervised Multi-document Summarization with Holistic Inference
SRI balances the importance and diversity of a subset of sentences from the source documents and can be calculated in unsupervised and adaptive manners.
Text-Only Domain Adaptation for End-to-End Speech Recognition through Down-Sampling Acoustic Representation
Mapping two modalities, speech and text, into a shared representation space, is a research topic of using text-only data to improve end-to-end automatic speech recognition (ASR) performance in new domains.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Bayes Risk Transducer: Transducer with Controllable Alignment Prediction
While the vanilla transducer does not have a prior preference for any of the valid paths, this work intends to enforce the preferred paths and achieve controllable alignment prediction.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models
Compositional generalization empowers the LLMs to solve problems that are harder than the ones they have seen (i. e., easy-to-hard generalization), which is a critical reasoning capability of human-like intelligence.
Ranked #17 on
Math Word Problem Solving
on MATH
MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning
Our approach uniquely considers the various annotation formats as different "views" and leverages them in training the model.
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
Specifically, the detection technique achieves a recall of ~88% and the mitigation technique successfully mitigates 57. 6% of the correctly detected hallucinations.
Make-A-Voice: Unified Voice Synthesis With Discrete Representation
Various applications of voice synthesis have been developed independently despite the fact that they generate "voice" as output in common.
PIVOINE: Instruction Tuning for Open-world Information Extraction
In particular, we construct INSTRUCTOPENWIKI, a substantial instruction tuning dataset for Open-world IE enriched with a comprehensive corpus, extensive annotations, and diverse instructions.
Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations
It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space.
Open-Domain Event Graph Induction for Mitigating Framing Bias
However, none of the existing methods have explicitly addressed the issue of framing bias that is inherent in news articles.
Hybrid AHS: A Hybrid of Kalman Filter and Deep Learning for Acoustic Howling Suppression
During offline training, a pre-processed signal obtained from the Kalman filter and an ideal microphone signal generated via teacher-forced training strategy are used to train the deep neural network (DNN).
Faithful Question Answering with Monte-Carlo Planning
In this paper, we propose FAME (FAithful question answering with MontE-carlo planning) to answer questions based on faithful reasoning steps.
Deep Learning for Joint Acoustic Echo and Acoustic Howling Suppression in Hybrid Meetings
In particular, the interplay between acoustic echo and acoustic howling in a hybrid meeting makes the joint suppression of them difficult.
Deep AHS: A Deep Learning Approach to Acoustic Howling Suppression
In this paper, we formulate acoustic howling suppression (AHS) as a supervised learning problem and propose a deep learning approach, called Deep AHS, to address it.
Search-Engine-augmented Dialogue Response Generation with Cheaply Supervised Query Production
We propose a dialogue model that can access the vast and dynamic information from any search engine for response generation.
Friend-training: Learning from Models of Different but Related Tasks
Current self-training methods such as standard self-training, co-training, tri-training, and others often focus on improving model performance on a single task, utilizing differences in input features, model architectures, and training processes.
Neural Target Speech Extraction: An Overview
Humans can listen to a target speaker even in challenging acoustic conditions that have noise, reverberation, and interfering speakers.
NeuralKalman: A Learnable Kalman Filter for Acoustic Echo Cancellation
The robustness of the Kalman filter to double talk and its rapid convergence make it a popular approach for addressing acoustic echo cancellation (AEC) challenges.
OASum: Large-Scale Open Domain Aspect-based Summarization
Specifically, zero/few-shot and fine-tuning results show that the model pre-trained on our corpus demonstrates a strong aspect or query-focused generation ability compared with the backbone model.
TriNet: stabilizing self-supervised learning from complete or slow collapse on ASR
Self-supervised learning (SSL) models confront challenges of abrupt informational collapse or slow dimensional collapse.
ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
However, there has been limited research on the zero-shot KBC settings, where we need to deal with unseen entities and relations that emerge in a constantly growing knowledge base.
Deep Neural Mel-Subband Beamformer for In-car Speech Separation
While current deep learning (DL)-based beamforming techniques have been proved effective in speech separation, they are often designed to process narrow-band (NB) frequencies independently which results in higher computational costs and inference times, making them unsuitable for real-world use.
Efficient Zero-shot Event Extraction with Context-Definition Alignment
We argue that using the static embedding of the event type name might not be enough because a single word could be ambiguous, and we need a sentence to define the type semantics accurately.
Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing
Pretrained natural language processing (NLP) models have achieved high overall performance, but they still make systematic errors.
Toward Unifying Text Segmentation and Long Document Summarization
The problem is only exacerbated by a lack of segmentation in transcripts of audio/video recordings.
Ranked #5 on
Text Summarization
on Pubmed
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
In this paper, we develop a novel semi-parametric language model architecture, Knowledge-in-Context (KiC), which empowers a parametric text-to-text language model with a knowledge-rich external memory.
Ranked #5 on
Question Answering
on StoryCloze
Salience Allocation as Guidance for Abstractive Summarization
Recent literature adds extractive summaries as guidance for abstractive summarization models to provide hints of salient content and achieves better performance.
Ranked #7 on
Abstractive Text Summarization
on CNN / Daily Mail
MetaLogic: Logical Reasoning Explanations with Fine-Grained Structure
To this end, we propose a comprehensive logical reasoning explanation form.
Learning a Grammar Inducer from Massive Uncurated Instructional Videos
While previous work focuses on building systems for inducing grammars on text that are well-aligned with video content, we investigate the scenario, in which text and video are only in loose correspondence.
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination
Large-scale pretrained language models have made significant advances in solving downstream language understanding tasks.
Ranked #2 on
Visual Commonsense Tests
on ViComTe-color
Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence tasks
Besides predicting the target sequence, a side product of CTC is to predict the alignment, which is the most probable input-long sequence that specifies a hard aligning relationship between the input and target units.
Cross-Lingual Speaker Identification Using Distant Supervision
Speaker identification, determining which character said each utterance in literary text, benefits many downstream tasks.
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks
Notably, our proposed $\text{Zemi}_\text{LARGE}$ outperforms T0-3B by 16% on all seven evaluation tasks while being 3. 9x smaller in model size.
C3-DINO: Joint Contrastive and Non-contrastive Self-Supervised Learning for Speaker Verification
On the basis of the pretrained CSSL model, we further propose to employ a negative sample free SSL objective (i. e., DINO) to fine-tune the speaker embedding network.
Diffsound: Discrete Diffusion Model for Text-to-sound Generation
In this study, we investigate generating sound conditioned on a text prompt and propose a novel text-to-sound generation framework that consists of a text encoder, a Vector Quantized Variational Autoencoder (VQ-VAE), a decoder, and a vocoder.
Ranked #13 on
Audio Generation
on AudioCaps
Hierarchical Context Tagging for Utterance Rewriting
HCT (i) tags the source string with token-level edit actions and slotted rules and (ii) fills in the resulting rule slots with spans from the dialogue context.
Automatic Prosody Annotation with Pre-Trained Text-Speech Model
Prosodic boundary plays an important role in text-to-speech synthesis (TTS) in terms of naturalness and readability.
UTTS: Unsupervised TTS with Conditional Disentangled Sequential Variational Auto-encoder
We leverage recent advancements in self-supervised speech representation learning as well as speech synthesis front-end techniques for system development.
LAE: Language-Aware Encoder for Monolingual and Multilingual ASR
Experiments conducted on Mandarin-English code-switched speech suggest that the proposed LAE is capable of discriminating different languages in frame-level and shows superior performance on both monolingual and multilingual ASR tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
NeuralEcho: A Self-Attentive Recurrent Neural Network For Unified Acoustic Echo Suppression And Speech Enhancement
Acoustic echo cancellation (AEC) plays an important role in the full-duplex speech communication as well as the front-end speech enhancement for recognition in the conditions when the loudspeaker plays back.
Towards Improved Zero-shot Voice Conversion with Conditional DSVAE
In our experiment on the VCTK dataset, we demonstrate that content embeddings derived from the conditional DSVAE overcome the randomness and achieve a much better phoneme classification accuracy, a stabilized vocalization and a better zero-shot VC performance compared with the competitive DSVAE baseline.
Distant finetuning with discourse relations for stance classification
Approaches for the stance classification task, an important task for understanding argumentation in debates and detecting fake news, have been relying on models which deal with individual debate topics.
FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis
Also, FastDiff enables a sampling speed of 58x faster than real-time on a V100 GPU, making diffusion models practically applicable to speech synthesis deployment for the first time.
Ranked #7 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
3M: Multi-loss, Multi-path and Multi-level Neural Networks for speech recognition
Recently, Conformer based CTC/AED model has become a mainstream architecture for ASR.
Ranked #2 on
Speech Recognition
on WenetSpeech
Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversion
A zero-shot voice conversion is performed by feeding an arbitrary speaker embedding and content embeddings to the VAE decoder.
Integrating Lattice-Free MMI into End-to-End Speech Recognition
However, the effectiveness and efficiency of the MBR-based methods are compromised: the MBR criterion is only used in system training, which creates a mismatch between training and decoding; the on-the-fly decoding process in MBR-based methods results in the need for pre-trained models and slow training speeds.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis
We propose a new bilateral denoising diffusion model (BDDM) that parameterizes both the forward and reverse processes with a schedule network and a score network, which can train with a novel bilateral modeling objective.
Ranked #1 on
Speech Synthesis
on LJSpeech
Towards Abstractive Grounded Summarization of Podcast Transcripts
Summarization of podcast transcripts is of practical benefit to both content providers and consumers.
Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations.
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References
And with our pretrained reader, the entire system improves by up to 4% in exact match.
Full RGB Just Noticeable Difference (JND) Modelling
Besides, the JND of the red and blue channels are larger than that of the green one according to the experimental results of the proposed model, which demonstrates that more changes can be tolerated in the red and blue channels, in line with the well-known fact that the human visual system is more sensitive to the green channel in comparison with the red and blue ones.
VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion
Though significant progress has been made for speaker-dependent Video-to-Speech (VTS) synthesis, little attention is devoted to multi-speaker VTS that can map silent video to speech, while allowing flexible control of speaker identity, all in a single system.
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows
Hence, we propose segment act, an extension of dialog act from utterance level to segment level, and crowdsource a large-scale dataset for it.
DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
Denoising diffusion probabilistic models (DDPMs) are expressive generative models that have been used to solve a variety of speech synthesis problems.
Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model
Then, the LM score of the hypothesis is obtained by intersecting the generated lattice with an external word N-gram LM.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Consistent Training and Decoding For End-to-end Speech Recognition Using Lattice-free MMI
Recently, End-to-End (E2E) frameworks have achieved remarkable results on various Automatic Speech Recognition (ASR) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorization
Conversational bilingual speech encompasses three types of utterances: two purely monolingual types and one intra-sententially code-switched type.
SpeechMoE2: Mixture-of-Experts Model with Improved Routing
Mixture-of-experts based acoustic models with dynamic routing mechanisms have proved promising results for speech recognition.
Multi-Channel Multi-Speaker ASR Using 3D Spatial Feature
Experimental results show that 1) the proposed ALL-In-One model achieved a comparable error rate to the pipelined system while reducing the inference time by half; 2) the proposed 3D spatial feature significantly outperformed (31\% CERR) all previous works of using the 1D directional information in both paradigms.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Meta-Voice: Fast few-shot style transfer for expressive voice cloning using meta learning
The task of few-shot style transfer for voice cloning in text-to-speech (TTS) synthesis aims at transferring speaking styles of an arbitrary source speaker to a target speaker's voice using very limited amount of neutral data.
Joint Neural AEC and Beamforming with Double-Talk Detection
We train the proposed model in an end-to-end approach to eliminate background noise and echoes from far-end audio devices, which include nonlinear distortions.
Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories
We then train a model to identify semantic equivalence between a target word in context and one of its glosses using these aligned inventories, which exhibits strong transfer capability to many WSD tasks.
FAST-RIR: Fast neural diffuse room impulse response generator
We present a neural-network-based fast diffuse room impulse response generator (FAST-RIR) for generating room impulse responses (RIRs) for a given acoustic environment.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
SynCLR: A Synthesis Framework for Contrastive Learning of out-of-domain Speech Representations
Learning generalizable speech representations for unseen samples in different domains has been a challenge with ever increasing importance to date.
Exophoric Pronoun Resolution in Dialogues with Topic Regularization
Resolving pronouns to their referents has long been studied as a fundamental natural language understanding problem.
Referee: Towards reference-free cross-speaker style transfer with low-quality data for expressive speech synthesis
The S2W model is trained with high-quality target data, which is adopted to effectively aggregate style descriptors and generate high-fidelity speech in the target speaker's voice.
Bilateral Denoising Diffusion Models
In this paper, we propose novel bilateral denoising diffusion models (BDDMs), which take significantly fewer steps to generate high-quality samples.
TexSmart: A System for Enhanced Natural Language Understanding
This paper introduces TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities.
Importance-based Neuron Allocation for Multilingual Neural Machine Translation
Multilingual neural machine translation with a single model has drawn much attention due to its capability to deal with multiple languages.
Do Boat and Ocean Suggest Beach? Dialogue Summarization with External Knowledge
To evaluate the inference capability of different methods, we also propose a new evaluation metric based on CODC.
Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognition
End-to-end speech recognition generally uses hand-engineered acoustic features as input and excludes the feature extraction module from its joint optimization.
Latency-Controlled Neural Architecture Search for Streaming Speech Recognition
Extensive experiments show that: 1) Based on the proposed neural architecture, the neural networks with a medium latency of 550ms (millisecond) and a low latency of 190ms can be learned in the vanilla and revised operation space respectively.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Experts
Recently, Mixture of Experts (MoE) based Transformer has shown promising results in many domains.
MIMO Self-attentive RNN Beamformer for Multi-speaker Speech Separation
The spatial self-attention module is designed to attend on the cross-channel correlation in the covariance matrices.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Conversational Semantic Role Labeling
Semantic role labeling (SRL) aims to extract the arguments for each predicate in an input sentence.
Video-aided Unsupervised Grammar Induction
We investigate video-aided grammar induction, which learns a constituency parser from both unlabeled text and its corresponding video.
MetricNet: Towards Improved Modeling For Non-Intrusive Speech Quality Assessment
The objective speech quality assessment is usually conducted by comparing received speech signal with its clean reference, while human beings are capable of evaluating the speech quality without any reference, such as in the mean opinion score (MOS) tests.
TeCANet: Temporal-Contextual Attention Network for Environment-Aware Speech Dereverberation
In this paper, we exploit the effective way to leverage contextual information to improve the speech dereverberation performance in real-world reverberant environments.
Towards Robust Speaker Verification with Target Speaker Enhancement
This paper proposes the target speaker enhancement based speaker verification network (TASE-SVNet), an all neural model that couples target speaker enhancement and speaker embedding extraction for robust speaker verification (SV).
NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversation
To facilitate the research on this corpus, we provide results of several benchmark models.
Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effect
We study the cocktail party problem and propose a novel attention network called Tune-In, abbreviated for training under negative environments with interference.
Contrastive Separative Coding for Self-supervised Representation Learning
To extract robust deep representations from long sequential modeling of speech data, we propose a self-supervised learning approach, namely Contrastive Separative Coding (CSC).
Sandglasset: A Light Multi-Granularity Self-attentive Network For Time-Domain Speech Separation
One of the leading single-channel speech separation (SS) models is based on a TasNet with a dual-path segmentation technique, where the size of each segment remains unchanged throughout all layers.
Ranked #8 on
Speech Separation
on WSJ0-3mix
Deep Learning based Multi-Source Localization with Source Splitting and its Effectiveness in Multi-Talker Speech Recognition
In addition to using the prediction error as a metric for evaluating our localization model, we also establish its potency as a frontend with automatic speech recognition (ASR) as the downstream task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Structural Information Preserving for Graph-to-Text Generation
The task of graph-to-text generation aims at producing sentences that preserve the meaning of input graphs.
Ranked #10 on
Data-to-Text Generation
on WebNLG
Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data
In spite of much recent research in the area, it is still unclear whether subject-area question-answering data is useful for machine reading comprehension (MRC) tasks.
Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks
Recent research on the time-domain audio separation networks (TasNets) has brought great success to speech separation.
Ranked #14 on
Speech Separation
on WSJ0-2mix
TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis
This technique report introduces TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities.
Robust Dialogue Utterance Rewriting as Sequence Tagging
The task of dialogue rewriting aims to reconstruct the latest dialogue utterance by copying the missing content from the dialogue context.
Multi-channel Multi-frame ADL-MVDR for Target Speech Separation
Many purely neural network based speech separation approaches have been proposed to improve objective assessment scores, but they often introduce nonlinear distortions that are harmful to modern automatic speech recognition (ASR) systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning
First, we examine a simple contrastive learning approach (SimCLR) with a momentum contrastive (MoCo) learning framework, where the MoCo speaker embedding system utilizes a queue to maintain a large set of negative examples.
Phonetic Posteriorgrams based Many-to-Many Singing Voice Conversion via Adversarial Training
In order to make timbre conversion more stable and controllable, speaker embedding is further decomposed to the weighted sum of a group of trainable vectors representing different timbre clusters.
BLCU-NLP at SemEval-2020 Task 5: Data Augmentation for Efficient Counterfactual Detecting
We demonstrate the effectiveness of our approaches, which achieves 0. 95 of subtask 1 in F1 while using only a subset of giving training set to fine-tune the BERT model, and our official submission achieves F1 0. 802, which ranks us 16th in the competition.
SHIKEBLCU at SemEval-2020 Task 2: An External Knowledge-enhanced Matrix for Multilingual and Cross-Lingual Lexical Entailment
In our system, we expand the number of external constraints in multiple languages to obtain more specialised multilingual word embeddings.
Improving RNN Transducer With Target Speaker Extraction and Neural Uncertainty Estimation
Target-speaker speech recognition aims to recognize target-speaker speech from noisy environments with background noise and interfering speakers.
Speech Enhancement
Speech Extraction
+1
Sound
Audio and Speech Processing
Audio-visual Multi-channel Integration and Recognition of Overlapped Speech
Automatic speech recognition (ASR) technologies have been significantly advanced in the past few decades.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Automatic Summarization of Open-Domain Podcast Episodes
Instead, we investigate several less-studied aspects of neural abstractive summarization, including (i) the importance of selecting important segments from transcripts to serve as input to the summarizer; (ii) striking a balance between the amount and quality of training instances; (iii) the appropriate summary length and start/end points.
Directional ASR: A New Paradigm for E2E Multi-Speaker Speech Recognition with Source Localization
The advantages of D-ASR over existing methods are threefold: (1) it provides explicit speaker locations, (2) it improves the explainability factor, and (3) it achieves better ASR performance as the process is more streamlined.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Replay and Synthetic Speech Detection with Res2net Architecture
This multiple scaling mechanism significantly improves the countermeasure's generalizability to unseen spoofing attacks.
Multi-Channel Speaker Verification for Single and Multi-talker Speech
We find the improvements from speaker-dependent directional features more consistent in multi-talker conditions than clean.
Better Highlighting: Creating Sub-Sentence Summary Highlights
Amongst the best means to summarize is highlighting.
High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies
We present a fully automatic system that can produce high-fidelity, photo-realistic 3D digital human heads with a consumer RGB-D selfie camera.
Token-level Adaptive Training for Neural Machine Translation
The vanilla NMT model usually adopts trivial equal-weighted objectives for target tokens with different frequencies and tends to generate more high-frequency tokens and less low-frequency tokens compared with the golden token distribution.
Semantic Role Labeling Guided Multi-turn Dialogue ReWriter
For multi-turn dialogue rewriting, the capacity of effectively modeling the linguistic knowledge in dialog context and getting rid of the noises is essential to improve its performance.
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge
In this paper, we aim to extract commonsense knowledge to improve machine reading comprehension.
Learned Transferable Architectures Can Surpass Hand-Designed Architectures for Large Scale Speech Recognition
Extensive experiments show that: (i) the architecture searched on the small proxy dataset can be transferred to the large dataset for the speech recognition tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Speech enhancement and speech separation are two related tasks, whose purpose is to extract either one or more target speech signals, respectively, from a mixture of sounds generated by several sources.
ADL-MVDR: All deep learning MVDR beamformer for target speech separation
Speech separation algorithms are often used to separate the target speech from other interfering sources.
Peking Opera Synthesis via Duration Informed Attention Network
In this work, we propose to deal with this issue and synthesize expressive Peking Opera singing from the music score based on the Duration Informed Attention Network (DurIAN) framework.
Comprehensive Image Captioning via Scene Graph Decomposition
We address the challenging problem of image captioning by revisiting the representation of image scene graph.
Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
Our core innovation is the learning of a region-phrase score function, based on which an image-sentence score function is further constructed.
ZPR2: Joint Zero Pronoun Recovery and Resolution using Multi-Task Learning and BERT
Zero pronoun recovery and resolution aim at recovering the dropped pronoun and pointing out its anaphoric mentions, respectively.
Speaker Independent and Multilingual/Mixlingual Speech-Driven Talking Head Generation Using Phonetic Posteriorgrams
Recent approaches mainly have following limitations: 1) most speaker-independent methods need handcrafted features that are time-consuming to design or unreliable; 2) there is no convincing method to support multilingual or mixlingual speech as input.
Investigating Robustness of Adversarial Samples Detection for Automatic Speaker Verification
Orthogonal to prior approaches, this work proposes to defend ASV systems against adversarial attacks with a separate detection network, rather than augmenting adversarial data into ASV training.
Audio-visual Multi-channel Recognition of Overlapped Speech
Automatic speech recognition (ASR) of overlapped speech remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
In this paper, we study machine reading comprehension (MRC) on long texts, where a model takes as inputs a lengthy document and a question and then extracts a text span from the document as an answer.
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning
Generating multi-sentence descriptions for videos is one of the most challenging captioning tasks due to its high requirements for not only visual relevance but also discourse-based coherence across the sentences in the paragraph.
Ranked #5 on
Video Captioning
on ActivityNet Captions
Neural Spatio-Temporal Beamformer for Target Speech Separation
Purely neural network (NN) based speech separation and enhancement methods, although can achieve good objective scores, inevitably cause nonlinear speech distortions that are harmful for the automatic speech recognition (ASR).
Audio and Speech Processing Sound
Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints
Text generation from a knowledge base aims to translate knowledge triples to natural language descriptions.
Dialogue-Based Relation Extraction
We present the first human-annotated dialogue-based relation extraction (RE) dataset DialogRE, aiming to support the prediction of relation(s) between two arguments that appear in a dialogue.
Ranked #6 on
Dialog Relation Extraction
on DialogRE
(F1c (v1) metric)
Multi-modal Multi-channel Target Speech Separation
Target speech separation refers to extracting a target speaker's voice from an overlapped audio of simultaneous talkers.
Enhancing End-to-End Multi-channel Speech Separation via Spatial Feature Learning
Hand-crafted spatial features (e. g., inter-channel phase difference, IPD) play a fundamental role in recent deep learning based multi-channel speech separation (MCSS) methods.
On the Role of Conceptualization in Commonsense Knowledge Graph Construction
Commonsense knowledge graphs (CKGs) like Atomic and ASER are substantially different from conventional KGs as they consist of much larger number of nodes formed by loosely-structured text, which, though, enables them to handle highly diverse queries in natural language related to commonsense, leads to unique challenges for automatic KG construction methods.
Coordinated Reasoning for Cross-Lingual Knowledge Graph Alignment
Existing entity alignment methods mainly vary on the choices of encoding the knowledge graph, but they typically use the same decoding method, which independently chooses the local optimal match for each source entity.
Multiplex Word Embeddings for Selectional Preference Acquisition
Therefore, in this paper, we propose a multiplex word embedding model, which can be easily extended according to various relations among words.
Audio-visual Recognition of Overlapped speech for the LRS2 dataset
Experiments on overlapped speech simulated from the LRS2 dataset suggest the proposed AVSR system outperformed the audio only baseline LF-MMI DNN system by up to 29. 98\% absolute in word error rate (WER) reduction, and produced recognition performance comparable to a more complex pipelined system.
Ranked #4 on
Audio-Visual Speech Recognition
on LRS2
Audio-Visual Speech Recognition
Automatic Speech Recognition (ASR)
+4
Synthesising Expressiveness in Peking Opera via Duration Informed Attention Network
This paper presents a method that generates expressive singing voice of Peking opera.
Learning Singing From Speech
The proposed algorithm first integrate speech and singing synthesis into a unified framework, and learns universal speaker embeddings that are shareable between speech and singing synthesis tasks.
A Unified Framework for Speech Separation
The initial solutions introduced for deep learning based speech separation analyzed the speech signals into time-frequency domain with STFT; and then encoded mixed signals were fed into a deep neural network based separator.
PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Network
However, the converted singing voice can be easily out of key, showing that the existing approach cannot model the pitch information precisely.
Modeling Fluency and Faithfulness for Diverse Neural Machine Translation
Neural machine translation models usually adopt the teacher forcing strategy for training which requires the predicted sequence matches ground truth word by word and forces the probability of each prediction to approach a 0-1 distribution.
Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition
In this work, we propose minimum Bayes risk (MBR) training of RNN-Transducer (RNN-T) for end-to-end speech recognition.
Joint Parsing and Generation for Abstractive Summarization
If generating a word can introduce an erroneous relation to the summary, the behavior must be discouraged.
Ranked #27 on
Text Summarization
on GigaWord
Improving Pre-Trained Multilingual Model with Vocabulary Expansion
However, in multilingual setting, it is extremely resource-consuming to pre-train a deep language model over large-scale corpora for each language.
BLCU-NLP at COIN-Shared Task1: Stagewise Fine-tuning BERT for Commonsense Inference in Everyday Narrations
This paper describes our system for COIN Shared Task 1: Commonsense Inference in Everyday Narrations.
Mixup-breakdown: a consistency training method for improving generalization of speech separation models
Deep-learning based speech separation models confront poor generalization problem that even the state-of-the-art models could abruptly fail when evaluating them in mismatch conditions.
DFSMN-SAN with Persistent Memory Model for Automatic Speech Recognition
Self-attention networks (SAN) have been introduced into automatic speech recognition (ASR) and achieved state-of-the-art performance owing to its superior ability in capturing long term dependency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multi-Document Summarization with Determinantal Point Processes and Contextualized Representations
Emerged as one of the best performing techniques for extractive summarization, determinantal point processes select the most probable set of sentences to form a summary according to a probability measure defined by modeling sentence prominence and pairwise repulsion.
Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks
Our best systems achieve a relative improvement of 11. 9% and 8. 3% on the TIMIT and WSJ tasks respectively.
Generating Diverse Story Continuations with Controllable Semantics
We propose a simple and effective modeling framework for controlled generation of multiple, diverse outputs.
Improving Pre-Trained Multilingual Models with Vocabulary Expansion
However, in multilingual setting, it is extremely resource-consuming to pre-train a deep language model over large-scale corpora for each language.
A Random Gossip BMUF Process for Neural Language Modeling
One important challenge of training an NNLM is to leverage between scaling the learning process and handling big data.
Audio-Visual Speech Separation and Dereverberation with a Two-Stage Multimodal Network
Background noise, interfering speech and room reverberation frequently distort target speech in real listening environments.
Audio and Speech Processing Sound Signal Processing
DurIAN: Duration Informed Attention Network For Multimodal Synthesis
In this paper, we present a generic and robust multimodal synthesis system that produces highly natural speech and facial expression simultaneously.
Maximizing Mutual Information for Tacotron
End-to-end speech synthesis methods already achieve close-to-human quality performance.
A Fast and Accurate One-Stage Approach to Visual Grounding
We propose a simple, fast, and accurate one-stage approach to visual grounding, inspired by the following insight.
Unsupervised Neural Aspect Extraction with Sememes
Aspect extraction relies on identifying aspects by discovering coherence among words, which is challenging when word meanings are diversified and processing on short texts.
Aspect Extraction
Aspect Term Extraction and Sentiment Classification
+1
Improving Reverberant Speech Training Using Diffuse Acoustic Simulation
We present an efficient and realistic geometric acoustic simulation approach for generating and augmenting training data in speech-related machine learning tasks.
Teach an all-rounder with experts in different domains
First, for each domain, a teacher model (domain-dependent model) is trained by fine-tuning a multi-condition model with domain-specific subset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
