, , , , , , , , , , , , , ,

TMM recently announced that it was working with Raytheon to conduct research. This got me thinking about the applications Raytheon might deploy.

Because fractal compression exploits similarities between different parts of an image, it is unsuitable for survelliance applications. The basic problem is that it is based on the collage theorem (in the case of PIFS and SoftVideo, block copying), and a collage is not something you want when doing surveillance.

When an image is zoomed using traditional methods, each pixel being enlarged is considered alongside its immediate neighbours. Data from other parts of the image is not considered and can therefore not “pollute” the area being enlarged.

In a collage system however, the pixels used to enlarge an area may come from anywhere else in the image. In video, they can even come from a previous frame. I will illustrate the danger with a simple example.

Imagine a picture of two men, both standing in the same way and both facing the camera, and both with a similar head shape. Their facial features are also similiar but they are not identical. One of the men is standing twice as close to the camera and so he appears twice as large.

When this image is fractally compressed, a block containing the large man’s face will be shrunk to half size and compared to the block containing the small man’s face. The blocks will be similar enough that a match will be declared found, and a fractal code relating the two blocks will be written to the file.

During decompression, the face of the first man will be constructed by sampling the pixels from the face of the second man. If fractal zooming is used, the first man will look more like the second man.

Now, anyone using this decompressed image for survelliance will face three problems:

1. The identity of the first man has become lost, since his face is determined solely from the pixels in the face of the second man.

2. If the first man was a suspect, the second man will be incorrectly suspected instead.

3. The two men may be believed to be twins when they may not be brothers, or even related to each other at all.

This example is a little extreme, but you see the point. All sorts of other erroneous situations can occur. A bunch of car keys held in someone’s hands may be substituted by a gun held by another. The color of a person eyes may be shifted. An identifying birthmark or tattoo may be altered.

Long story short, all the details that appear when zooming can come from anywhere, and they may be significantly different than what would have been seen if the original scene had simply been viewed twice as close. If an investigator is biased towards interpreting a particular cluster of pixels, his bias may be erroneously strengthened when seeing them upscaled, as he is now effectively looking at a different part of the image.