Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis
چکیده
Representation Learning is a significant and challenging task in multimodal learning. Effective modality representations should contain two parts of characteristics: the consistency difference. Due to unified annota- tion, existing methods are restricted capturing differenti- ated information. However, additional unimodal annotations high time- labor-cost. In this paper, we design la- bel generation module based on self-supervised learning strategy acquire independent supervisions. Then, joint training uni-modal tasks learn difference, respectively. Moreover, dur- ing stage, weight-adjustment strat- egy balance progress among different sub- tasks. That guide subtasks focus samples with larger difference between Last, conduct extensive experiments three public baseline datasets. The experimental results validate re- liability stability auto-generated supervi- sions. On MOSI MOSEI datasets, our method surpasses current state-of-the-art methods. SIMS dataset, achieves comparable performance than human- annotated labels. full codes available at https://github.com/thuiar/Self-MM.