PhysFormer++: Facial Video-Based Physiological Measurement with SlowFast Temporal Difference Transformer

Abstract Remote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e.g., remote healthcare affective computing). Recent deep learning approaches focus on mining subtle rPPG clues using convolutional neural networks with limited spatio-temporal receptive fields, neglect the long-range perception interaction for modeling. In this paper, we propose two end-to-end transformer based architectures, namely PhysFormer PhysFormer++, to adaptively aggregate both local global features representation enhancement. As key modules PhysFormer, temporal difference transformers first enhance quasi-periodic guided attention, then refine against interference. To better exploit contextual periodic clues, also extend two-pathway SlowFast PhysFormer++ cross-attention transformers. Furthermore, label distribution a curriculum inspired dynamic constraint frequency domain, provide elaborate supervisions alleviate overfitting. Comprehensive experiments are performed four benchmark datasets show our superior performance intra- cross-dataset testings. Unlike most needed pretraining large-scale datasets, proposed family can be easily trained scratch makes it promising as novel baseline community.