Abstract

This paper gives an overview of methods to correct a common problem with today's technology of CCD image capture. This problem is the presence of poorly performing pixels in the CCD array, which produce unsatisfactory images. The nature of this problem will be described below, along with several examples. The process of detecting and correcting these bad areas will be described, along with a discussion of the industry's current solutions. The area where this research treads new ground is that of using many images in a series to correct a single photo. This technique will be described in depth.

Introduction

"Dead pixels" are the largest problem in current CCD technology. Below is an example produced with a Kodak 4800. It is worth noting that between two Kodak cameras tested, a dead "pixel" was never a single pixel, but a minimum of four. And in every case the damage spread from those pixels to the immediately adjacent pixels for a total of at least 16 misreporting pixels.

Please read this side note about camera resolutions.

Figure 3:

A dead 'pixel' on the 3900

Figure 4:

The same location from another photo. Notice that the blocky characteristics are produced in large part to the jpeg compression. The centermost 16 pixels are completely destroyed, where in the 4800, the damage was less severe.

A CCD works by creating a two dimensional matrix of photo sensitive cells. The phenomenon of overly bright pixels is known as "dark current," or "dark light." The terminology comes from the fact that if the CCD takes a picture of pure darkness, those pixels will report nonexistent light.

Another problem is that of "leakage." Without compression, the Kodak 4800 is able to demonstrate this phenomenon well. Observe the image below.

Figure 5:

A red 'pixel' over-reporting.

Figure 6:

The same pixel magnified 3000%. Notice that the red "leaks" out of the 2x2 'pixel,' into the edge sharing neighbors.

Detection

The first step in automatically correcting these dead pixels in software is to locate them. Several methods were explored during the length of this research, and ultimately the best solution was to create what I dubbed an "ubermask." Because the CCD error is extremely consistent (non-random), building an average of many images all produced in the same time frame made filtering for dead pixels very accurate. Below is an example of an ubermask.

Figure 7:

"Ubermask." Click the image for the fill sized example. Notice in the full sized image how easily your eye can pick out dead pixels.

Mathematically speaking, averaging enough pictures together should create a solid gray image. Figure 7 above is the result of averaging 18 photos from the Kodak 4800. Because the bad pixels do not range from 0 to 255 as the functional pixels do, their average is not 128. Applying the appropriate filters to this ubermask will single out the pixels with high frequency (large change from their neighbors). At this point we have a nearly black image with lighter values in places of consistent high frequency. Notice in the image above, that sharp edges are still visible even after averaging 18 images. This will cause a problem, since sharp edges are a natural phenomenon, not a problem that needs adjusting. The solution is the enforce a threshold, such that errors below a certain intensity value are discarded. This creates a binary mask of exactly which pixels need to be corrected.

It is also worth noting that this process is run independently on each color channel. So if a pixel has a problem with red, the blue and green data is not discarded.

Correction

To repair the pixels a number of techniques were attempted. Initially the pixels were altered by a subtractive mask, but this produced inconsistent results depending on lighting conditions. It was then discovered that the bad pixels were reporting almost no useful data. Ultimately it became clear that hole-filling techniques were the most appropriate.

This again brings up the camera resolution honesty issue. If a single pixel in the final image is bad, the correction is trivial. One can simply average the immediate neighbors for a quite satisfactory result. However since the error is in the form of large blocks, correcting that area without throwing away potentially useful data, and maintaining an algorithm that would appropriately correct a single bad pixel if one should occur was difficult.

Below are several examples of camera output followed by the corrected result.

Industry Solutions

To the best of my knowledge, at the time of writing, no other software exists in the consumer market that corrects images in this manner. With less expensive cameras, the solution is to ignore the problem.

One manufacturer has come up with a hardware solution. In high-end Sony cameras (Mostly $1000+ SLR models), the camera takes a second shot, an instant after the user takes a picture. This second shot is taken with the aperture completely closed. Then, in hardware, the camera takes the user's photo, and subtracts the nearly black mask. The idea is that any dark current would be removed in the subtraction. I have not been able to obtain any samples of this technology, and so I cannot comment too authoritatively on the matter. The obvious objection to this method is that you could in fact produce the inverse of the problem, when taking brightly lit photos (the mask would remove light, creating dark spots). But in practice, according to the documentation, the camera only performs this action in low light conditions where the bright spots would be most obvious.