PropTest: Automatic Property Testing for Improved Visual Programming

no code implementations • 25 Mar 2024 • Jaywon Koo, Ziyan Yang, Paola Cascante-Bonilla, Baishakhi Ray, Vicente Ordonez

Visual Programming has emerged as an alternative to end-to-end black-box visual reasoning models.

Question Answering Referring Expression +3

Multimodal Multi-loss Fusion Network for Sentiment Analysis

no code implementations • 1 Aug 2023 • Zehui Wu, Ziwei Gong, Jaywon Koo, Julia Hirschberg

This paper investigates the optimal selection and fusion of feature encoders across multiple modalities and combines these in one neural network to improve sentiment detection.

feature selection Multimodal Sentiment Analysis

Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities

no code implementations • 14 Jun 2022 • Hammad A. Ayyubi, Christopher Thomas, Lovish Chum, Rahul Lokesh, Long Chen, Yulei Niu, Xudong Lin, Xuande Feng, Jaywon Koo, Sounak Ray, Shih-Fu Chang

To support research on this task, we introduce the Multimodal Hierarchical Events (MultiHiEve) dataset.