Step fusion: Local and global mutual guidance

29 Jun 2023  ·  Jiahao Qin, Yitao Xu, Zong Lu, Xiaojun Zhang ·

Feature alignment is the primary means of fusing multimodal data. We propose a feature alignment method that fully fuses multimodal information, which stepwise shifts and expands feature information from different modalities to have a consistent representation in a feature space. The proposed method can robustly capture high-level interactions between features of different modalities, thus significantly improving the performance of multimodal learning. We also show that the proposed method outperforms other popular multimodal schemes on multiple tasks. Experimental evaluation of ETT and MIT-BIH-Arrhythmia, datasets shows that the proposed method achieves state of the art performance.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Arrhythmia Detection MIT-BIH Arrhythmia Database ATD Accuracy 98.9 # 1
F1 98.2 # 1

Methods


No methods listed for this paper. Add relevant methods here