Deep palette-based color decomposition for image recoloring with aesthetic suggestion

Abstract

Color edition is an important issue in image processing and graphic design. This paper presents a deep color decomposition based framework for image recoloring, allowing users to achieve professional color edition through simple interactive operations. Different from existing methods that perform palette generation and color decomposition separately, our method directly generates the recolored images by a light-weight CNN. We first formulate the generation of color palette as an unsupervised clustering problem, and employ a fully point-wise CNN to extract the most representative colors from the input image. Particularly, a pixel scrambling strategy is adopted to map the continuous image color to a compact discrete palette space, facilitating the CNN focus on color-relevant features. Then, we devise a deep color decomposition network to obtain the projected weights of input image on the basis colors of the generated palette space, and leverage them for image recoloring in a user-interacted manner. In addition, a novel aesthetic constraint derived from color harmony theory is proposed to augment the color reconstruction from user-specified colors, resulting in an aesthetically pleasing visual effect. Qualitative comparisons with existing methods demonstrate the effectiveness of our proposed method.