Recent Advances in Non-Local Image Restoration and Video Inpainting
Résumé
Digital images and sequences are most often corrupted by noise, blur, occlusions and many other deteriorations. A detailed mathematical model of the acquisition device allows to cast the problem of restoring the original image from corrupted measurements as an ill-posed inverse problem. Hence, in order to solve it numerically, we need to adopt a regularizing prior. Most common regularizers used in the early days of image processing, were based on linear (Gaussian, Wiener) or adaptive filters. In contrast, non-local priors do not impose any particular structure on local image patches. They only assume that patches within a natural image are self-similar, meaning that the same patch appears in different locations of the same image. The introduction of non-local methods for image denoising in 2005 allowed a major leap forward in terms of restored image quality. More recently (since 2012) a more precise Bayesian formulation of the non-local prior allowed a further increase in performance, and further flexibility to apply it to more ill-posed inverse problems, including image inpainting which consists in reconstructing the image behind an occluding object, based on the visible surrounding context. From the computational perspective, however, non-local methods are several orders of magnitude more expensive than linear filtering. A major part of this talk will be devoted to the algorithmic acceleration techniques available for this kind of methods. In particular, a generalization of Adobe’s PatchMatch algorithm allows to reduce the computational complexity of video inpainting from several days to a few hours on a single processor. Similarly our covariance tree allows to accelerate the non-local Bayes restoration algorithm even in non-structured situations like 3D point clouds or learning-based restoration.