Layered neural atlases for consistent video editing

We present a method that decomposes, and "unwraps", an input video into set of layered 2D atlases , each providing unified representation the appearance object (or background) over video. For pixel in video, our estimates its corresponding coordinate atlases, giving us consistent parameterization along with associated alpha (opacity) value. Importantly, we design to be interpretable semantic, which facilitates easy intuitive editing atlas domain, minimal manual work required. Edits applied single frame) are automatically consistently mapped back original frames, while preserving occlusions, deformation, other complex scene effects such as shadows reflections. Our employs coordinate-based Multilayer Perceptron (MLP) for mappings, alphas, jointly optimized on per-video basis, using combination reconstruction regularization losses. By operating purely 2D, does not require any prior 3D knowledge about geometry or camera poses, can handle dynamic real world videos. demonstrate various applications, including texture mapping, style transfer, image-to-video segmentation/labeling propagation, all produced by image.