Self-Supervised Intensity-Event Stereo Matching

Event cameras are novel bio-inspired vision sensors that output pixel-level intensity changes in microsecond accuracy with high dynamic range and low power consumption. Despite these advantages, event cannot be directly applied to computational imaging tasks due the inability obtain high-quality events simultaneously. This paper aims connect a standalone camera modern so applications can take advantage of both sensors. We establish this connection through multi-modal stereo matching task. first convert reconstructed image extend existing networks multi-modality condition. propose self-supervised method train network without using ground truth disparity data. The structure loss calculated on gradients is used enable learning such Exploiting internal constraint between views different modalities, we introduce general functions, including cross-consistency loss, leading improved performance robustness compared approaches. Our experiments demonstrate effectiveness proposed method, especially synthetic real datasets. Finally, shed light employing aligned images downstream tasks, e.g., video interpolation application.