Co-segmentation inspired attention module for video-based computer vision tasks

Video-based computer vision tasks can benefit from estimation of the salient regions and interactions between those regions. Traditionally, this has been done by identifying object in images utilizing pre-trained models to perform detection, segmentation and/or pose estimation. Although using is a viable approach, it several limitations need for an exhaustive annotation categories, possible domain gap datasets bias that typically present models. In work, we propose utilize common rationale sequence video frames capture set objects them, thus notion co-segmentation frame features may equip model with ability automatically focus on task-specific improve underlying task’s performance end-to-end manner. regard, generic module called “Co-Segmentation inspired Attention Module” (COSAM) be plugged any CNN promote based attention among features. We show application COSAM three video-based namely: (1) person re-ID, (2) Video captioning, & (3) action classification demonstrate able frames, leading notable improvements along interpretable maps variety tasks, other as well.