Multimodal Sentiment Analysis

73 papers with code • 5 benchmarks • 7 datasets

Multimodal sentiment analysis is the task of performing sentiment analysis with multiple data sources - e.g. a camera feed of someone's face and their recorded speech.

( Image credit: ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection )

Libraries

Use these libraries to find Multimodal Sentiment Analysis models and implementations
3 papers
571

Most implemented papers

Multimodal Speech Emotion Recognition Using Audio and Text

david-yoon/multimodal-speech-emotion 10 Oct 2018

Speech emotion recognition is a challenging task, and extensive reliance has been placed on models that use audio features in building well-performing classifiers.

Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors

victorywys/RAVEN 23 Nov 2018

Humans convey their intentions through the usage of both verbal and nonverbal behaviors during face-to-face communication.

Multimodal Transformer for Unaligned Multimodal Language Sequences

yaohungt/Multimodal-Transformer ACL 2019

Human language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors.

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors

Justin1904/Low-rank-Multimodal-Fusion ACL 2018

Previous research in this field has exploited the expressiveness of tensors for multimodal representation.

Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment Analysis

robertjkeck2/EmoTe 17 Apr 2019

Therefore, in this paper, based on audio and text, we consider the task of multimodal sentiment analysis and propose a novel fusion strategy including both multi-feature fusion and multi-modality fusion to improve the accuracy of audio-text sentiment analysis.

M-SENA: An Integrated Platform for Multimodal Sentiment Analysis

thuiar/MMSA ACL 2022

The platform features a fully modular video sentiment analysis framework consisting of data management, feature extraction, model training, and result analysis modules.

Context-Dependent Sentiment Analysis in User-Generated Videos

senticnet/sc-lstm ACL 2017

Multimodal sentiment analysis is a developing area of research, which involves the identification of sentiments in videos.

Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning

pliang279/MFN 3 Feb 2018

In this paper, we propose the Gated Multimodal Embedding LSTM with Temporal Attention (GME-LSTM(A)) model that is composed of 2 modules.

Multi-attention Recurrent Network for Human Communication Comprehension

pliang279/MFN 3 Feb 2018

AI must understand each modality and the interactions between them that shape human communication.

Multimodal Sentiment Analysis To Explore the Structure of Emotions

anthonyhu/tumblr-emotions ICLR 2018

We propose a novel approach to multimodal sentiment analysis using deep neural networks combining visual analysis and natural language processing.