RENOIR - A dataset for real low-light image noise reduction

Many modern and popular state of the art image denoising algorithms are trained and evaluated using images corrupted by artificial noise. These trained algorithms and their evaluations on synthetic data may lead to incorrect conclusions about their performances on real noise. In this paper we introduce a benchmark dataset of uncompressed color images corrupted by natural noise due to low-light conditions, together with spatially and intensity-aligned low noise images of the same scenes. The dataset contains over 120 scenes and more than 400 images, including both 16-bit RAW images and 8-bit BMP pixel and intensity-aligned images from 2 digital cameras (Canon S90 and Canon T3i) and a mobile phone (Xiaomi Mi3). We also introduce a method for estimating the true noise level in each of our images, since even the low noise images contain a small amount of noise. Finally, we exemplify the use of our dataset by evaluating four denoising algorithms: Active Random Field, BM3D, Bilevel MRF optimization, and Multi-Layer Perceptron. We show that while the Multi-Layer Perceptron and Bilevel MRF algorithms work as well as or even better than BM3D on synthetic noise, they lag behind on our dataset.