6 - 1 Data Fusion for Long Range Target Acquisition

An approach to the long range automatic detection of vehicles, using multi-sensor image sequences is presented. The algorithm we use was tested on a database of six sequences, acquired under diverse operational conditions. The vehicles in the sequences can be either moving or stationary. The sensors are stationary, but can perform a pan/tilt operation. The presented paradigm uses data fusion methods at four different levels (feature level, sensor level, temporal level and decision level) and consists of two parts. The first part detects targets in individual images using a semisupervised approach. For each type of sensor a training image is chosen. On this training image the target position is indicated. Textural features are calculated at each pixel of this image. Feature level fusion is used to combine the different features in order to find an optimal discrimination between target and nontarget pixels for this training image. Because the features are closely linked to the physical properties of the sensors, the same combination of features also gives good results on the test images, which are formed of the remainder of the database sequences. By applying feature level fusion, a new image is created in which the local maxima correspond to probable target positions. These images coming from the different sensors are then combined in a multi-sensor image using sensor fusion. The local maxima in this multi-sensor image are detected using morphological operators. Any available prior knowledge about possible target size and aspect ratio is incorporated using a region growing procedure around the local maxima. A variation to this approach, that will also be developed in this paper, combines the previous feature and sensor level fusion, by extracting the features in each sensor as before but using the feature level fusion directly on the combination of all features from all sensors in what is sometimes called a « super feature vector ». Tracking is used in both cases to reduce the false alarm rate. The second part of the algorithm detects moving targets. First any motion of the sensor itself needs to be detected. This detection is based on a comparison between the spatial cooccurrence matrix within one single image and the temporal cooccurrence matrix between successive images in a sequence. If sensor motion is detected it is estimated using a correlationbased technique. This motion estimate is used to warp past images onto the current one. Temporal fusion is used to detect moving targets in the new sub-sequence of warped images. Temporal and spatial consistency are used to reduce the false alarm rate. For each sensor, the two parts of the algorithm each behave as an expert, indicating the possible presence of a target. The final result is obtained by using decision fusion methods in order to combine the decisions of the different experts. Several « k out of n » decision fusion methods are compared and the results evaluated on the basis of the 6 multi-sensor sequences.