Feature Tracking with Automatic Selection of Spatial Scales

When observing a dynamic world, the size of image structures may vary over time. This article emphasizes the need for including explicit mechanisms for automatic scale selection in feature tracking algorithms in order to: (i) adapt the local scale of processing to the local image structure, and (ii) adapt to the size variations that may occur over time. The problems of corner detection and blob detection are treated in detail, and a combined framework for feature tracking is presented in which the image features at every time moment are detected at locally determined and automatically selected scales. A useful property of the scale selection method is that the scale levels selected in the feature detection step re ect the spatial extent of the image structures. Thereby, the integrated tracking algorithm has the ability to adapt to spatial as well as temporal size variations, and can in this way overcome some of the inherent limitations of exposing xed-scale tracking methods to image sequences in which the size variations are large. In the composed tracking procedure, the scale information is used for two additional major purposes: (i) for de ning local regions of interest for searching for matching candidates as well as setting the window size for correlation when evaluating matching candidates, and (ii) stability over time of the scale and signi cance descriptors produced by the scale selection procedure are used for formulating a multi-cue similarity measure for matching. Experiments on real-world sequences are presented showing the performance of the algorithm when applied to (individual) tracking of corners and blobs. Speci cally, comparisons with xed-scale tracking methods are included as well as illustrations of the increase in performance obtained by using multiple cues in the feature matching step.