Colorization Using Optimization

Colorization Using Optimization

Anat Levin,  Dani Lischinski,  Yair Weiss

Abstract
Colorization is a computer-assisted process of adding color to a monochrome image or movie. The process typically involves segmenting images into regions and tracking these regions across image sequences. Neither of these tasks can be performed reliably in practice; consequently, colorization requires considerable user intervention and remains a tedious, time-consuming, and expensive task.

In this paper we present a simple colorization method that requires neither precise image segmentation, nor accurate region tracking. Our method is based on a simple premise: neighboring pixels in space-time that have similar intensities should have similar colors. We formalize this premise using a quadratic cost function and obtain an optimization problem that can be solved efficiently using standard techniques. In our approach an artist only needs to annotate the image with a few color scribbles, and the indicated colors are automatically propagated in both space and time to produce a fully colorized image or sequence. We demonstrate that high quality colorizations of stills and movie clips may be obtained from a relatively modest amount of user input.


Still Image Colorization Examples:

Click the images for uncompressed bitmaps.


Marked B/W image

Result

Marked B/W image

Result

Marked B/W image

Result

Marked B/W image

Result

Marked B/W image

Result

Marked B/W image

Result

Progressive colorization:

a1 - Initial set of scribbles

b1 - Scribbles have been added on the table cloth and the wall.

c1 - A few red pixels have been marked on the beads.

a2 - Initial result. Note that the table cloth gets the pink color of the girl's dress. Also, there is a cyan color spill from the pacifier on the wall behind.

b2 - Second result. Problems visible in a2 have been fixed.

c2 - Final result. We now have a string of red beads.

Recoloring Examples:

Click the images for uncompressed bitmaps.


Input image

Marked image

Result

Input image

Marked image

Result

Input image

Marked image

Result

Input image

Marked image

Result

Input image

Marked image

Result

Video Clip Colorization Examples:

The animated GIFs below use 8 bit color. For better color quality please play the QuickTime MPEG-4 movies.

Clip 1: Crater Lake.
83 frames, 7 marked frames
Click here for a QuickTime MPEG-4 movie.

Input grayscale clip
 
Marked frames

Colorized result


Clip 2: Toddler.
43 frames, 5 marked frames
Click here for a QuickTime MPEG-4 movie.
Input grayscale clip

Marked frames
Colorized result


Clip 3: Birthday.
62 frames, 10 marked frames
Click here for a QuickTime MPEG-4 movie.

Input grayscale clip

Marked frames
Colorized clip


Clip 4: Driving.
33 frames, 9 marked frames
Click here for a QuickTime MPEG-4 movie.
Input grayscale clip

Marked frames
Colorized clip