Existing vision-based action recognition is susceptible to occlusion and appearance variations, while wearable sensors can alleviate these challenges by capturing human motion with one-dimensional time-series signal. For the same action, knowledge learned from vision sensors, may be related complementary. However, there exists significantly large modality difference between data captured wearab...