Colorization for Monochrome Image with Texture

Colorization is a computerized process of adding color to a monochrome image. The authors have developed colorization algorithms which propagate colors from seeded color pixels. Since those algorithms are constructed based on a region growing approach, failure colorization occurs at the place where a luminance changes intensely such as edge and texture. Although we developed, in the previous work, a partitioning algorithm for preventing the error propagation at edge, numerous color seeds were required for accurate colorization of the image with texture. This paper presents a new algorithm for colorization with texture by blending seeded colors. In our algorithm, the color can be estimated depending on the Euclidean distance and the luminance distance between each pixel to be colorized. It is shown that the proposed approach can be successfully applied to the images with texture by sowing a small number of color seeds. Introduction Colorization is a computerized process that adds color to a black and white print, movie and TV pro-gram, supposedly invented by Wilson Markle. It was initially used in 1970 to add color to footage of the moon from the Apollo mission. The demand of adding color to monochrome images such as BW movies and BW photos has been increasing. For example, in the amusement field, many movies and video clips have been colorized by human’s labor, and many monochrome images have been distributed as vivid images. In other fields such as archaeology dealing with historical monochrome data and security dealing with monochrome images by a crime prevention camera, we can imagine easily that colorization techniques are useful. A luminance value of a monochrome image can be calculated uniquely by a linear combination of RGB color components. However, searching for the RGB components from a luminance value poses conversely an ill-posed problem, because there is several corresponding color to one luminance value. Due to these ambiguous, human interaction usually plays a large role in the colorization process. The correspondence between a color and a luminance value is determined through common sense (green for grass, blue for the ocean) or by investigation. Even in the case of pseudo-coloring, where the mapping of luminance values to a set of RGB components is automatic, the choice of the color-map is purely subjective. Since there are a few industrial software products, those technical algorithms are generally not available. However, by operating those software products, it turns out that humans must meticulously hand-color each of the individual image subjectivity. There also exist a few patents for colorization. However, those approaches depend on heavy human operation. Recently, simple colorization algorithms have been proposed by a few research groups. In 2002, one of the authors proposed a colorization algorithm in which a small number of color seeds were sown on a monochrome image and the remaining pixels are colorized by propagating seeds’ color to adjacent pixels. The algorithm has been improved in Refs.5-9. In the same year, Welsh et al. colorize a monochrome image by transferring color from a reference color image with a stochastic matching. The concept of transferring color from one image to another image was inspired by work in Ref.11. In the Welsh’s method, the source image, which is the same kind of image as a monochrome image, is prepared and the colorization is performed by color matching between both pictures. After that, Levin et al. mark a monochrome image with some color scribbles and adjacent pixels are colorized by formulating and solving an optimization problem. Those conventional algorithms are very simple and work well as an intuitive impression, especially for the image which can be segmented to a few large regions with the same chrominance components. However, it was difficult to perform accurate colorization for texture. In order to obtain an accurate colorized result for a texture image, Horiuchi’s algorithm requires many color seeds. In the case of Welsh’s algorithm, a specific reference image is required and Levin’s algorithm requires many color scribbles. This study aims to develop a new colorization algorithm for monochrome images with texture. This paper organized as follows: Section 2 presents our conventional algorithm and shows the problem for an image with texture. Section 3 presents the proposed colorization algorithm, and Section 4 demonstrates experiments. Finally, we conclude with a discussion in Section 5. Conventional Colorization by Propagating Seeded Colors Independently The most advanced colorization algorithm for still monochrome images by the authors' works can be shown in Ref.7. In this section, the conventional algorithm is explained briefly and a problem about texture is shown. Let ) , ( y x I = be a pixel in an input monochrome image and let { } p p p p y x S 1 ) , ( = = be a set of color seeds, where P is the total number of the seeds. The color seeds, which are color pixels strictly, are given manually as a prior knowledge by a user. The position of the seeds and their color are determined by the user. Note that the color must be chosen with keeping the luminance of the original monochrome pixels. We present our method in CIELAB color space, each monochrome pixel I is transformed (a) Position of seven color seeds on the monochrome image. (b) Colorized image. Figure 1. A colorized result by the method in Ref.7. into the luminance signal ) (I L . Each color seeds p S is also transformed into ) ( ), ( ), ( p p p S b S a S L , respectively. In Ref.7, each pixel I is colorized by )) ( ( )), ( ( ), ( I f b I f a I L in CIELAB color space. The function ) (⋅ f selects a color seeds which have the minimum Euclidean distance, and defined as:       − = 2 min ) ( p p p S I S I f (1) where 2 ⋅ means the Euclidean distance in the X-Y image space. Figure 1 shows an example of colorization by using the algorithm in Ref.7. Figure 1(a) shows an input monochrome image and the position of color seeds expressed by red circles. Each seeds were sown at the center of the circle. In this example, seven seeds were sown on the monochrome image by the user. Figure 1(b) shows the colorized results. Better result was obtained. Figure 2 shows another example. The image consists of texture such as petals and trees. By sowing five color seeds as shown in Fig.2(a), a failure colorized result was obtained as shown in Fig.2(b). In order to obtain more accurate result, the user has to sow color seeds for each small region in the texture. In actual application, it is impossible to sow numerous seeds on each region. Reference 7 also proposed a partitioning algorithm to prevent the error propagation at edge. However, it is difficult to determine a threshold of partition. Even if the user can set the partition, failure estimation will be occurred after collapsing the partition. Moreover, the method produces visible artifacts of block distortion. In order to solve the problem, we propose a new colorization algorithm by blending seed color in the next section. Proposed Colorization by Blending Many Seed Color Decision of the Chrominance Components In our algorithm, we use two properties of natural images. The first property is that pixels with similar luminance values should have similar colors. This property was used for solving colorization problem in Levin’s algorithm. The second property is that near pixels should have similar colors. This property was used in Horiuchi’s algorithm. In the proposed method, we express those (a) Position of five color seeds on the monochrome image. (b) Colorized image. Figure 2. A failure example of colorization by the method in Ref.7. Figure 4. Gamut mapping on a*-b* plane by clipping. two properties by distances NED and NLD as follows. (NED: Normalized Euclidean distance) We define the first distance ] 1 , 0 [ ) , ( 1 ∈ p S I d between I and p S