Exploit Camera Raw Data for Video Super- Resolution via Hidden Markov Model Inference

To the best of our knowledge, existing deep-learning-based Video Super-Resolution (VSR) methods exclusively make use videos produced by Image Signal Processor (ISP) camera system as inputs. Such are 1) inherently suboptimal due to information loss incurred non-invertible operations in ISP, and 2) inconsistent with real imaging pipeline where VSR fact serves a pre-processing unit ISP. address this issue, we propose new method that can directly exploit sensor data, accompanied carefully built Raw Dataset (RawVD) for training, validation, testing. This consists Successive Deep Inference (SDI) module reconstruction module, among others. The SDI is designed according architectural principle suggested canonical decomposition result Hidden Markov Model (HMM) inference; it estimates target high-resolution frame repeatedly performing pairwise feature fusion using deformable convolutions. elaborately Attention-based Residual Dense Blocks (ARDBs), purpose refining fused learning color needed generate spatial-specific transformation accurate correction. Extensive experiments demonstrate owing informativeness raw effectiveness network architecture, separation super-resolution correction processes, proposed achieves superior results compared state-of-the-art be adapted any specific camera-ISP. Code dataset available at https://github.com/proteus1991/RawVSR.