A Pixel Voting Method to Recover 3D Object Shape from 2D Images

The purpose of this study is to extract 3D information from 2D images passively. Though the voting technique is one of the passive methods to obtain the 3D information effectively, it cannot be applied to objects with smooth surfaces. It can measure only the feature points such as the edges which were previously extracted from each image. In this paper, we propose a new method to measure the 3D shape, which we call "a pixel voting method". In this method, the voting is simply done by using the information from the pixels of the images to measure the whole surface of the 3D object without feature extraction. 1. I N T R O D U C T I O N Many methods have been proposed to extract 3D information from 2D images. These are grouped into two categories. One is the active sensing method that uses special purpose apparatus such as laser scanners and spatially coded projectors. Another is the passive sensing method which deal with images with no special lighting mean. Though the active sensing method is often applied to industrial purposes, because it has better precision, there exist some problems in it. For example, the system tends to have a complex structure and the lighting condition is limited. On the other hand, the passive sensing method can be used in general cases. The binocular stereo technique is a typical passive sensing method. It is a technique which applies triangulation using two images from different viewing points. However, it has a serious problem of feature point matching that is to determine the corresponding points in each image. The voting technique was proposed to avoid the feature point matching. If we know the camera's positions and its orientations, 3D information can be extracted by using many images from the different camera position~!']-~~~ Since the usual simple voting technique measures only the feature points, such as edges previously extracted from each image, it is difficult to recover the shape of the curved surface or the surface where the feature points cannot be extracted. In this paper, we propose a new method to measure the 3D shape, which we call a pixel voting method. In this method, the voting is simply done by using the information from the pixels of the images. It is not necessary to extract the feature points from the images at all, which enables us to measure the whole surface of the 3D object!'] 2. BASIC IDEA OF THE PIXEL V O T I N G The simple voting method can be summarized as follows. If we know the camera's position and its orientation, we can calculate a line for each feature point in an image connected to center of the lens. Each line goes through a real feature point on the object in the object space. We call this line the back projection line. If we have images from different viewing points, the back projection lines corresponding to the same feature point cross at the real feature point in the object space. Fig.1 shows the back projection lines crossing at the real point. Therefore, if we draw the back projection lines for many object images, and find the point where many lines cross together in the object space, the positions of feature points on the object in 3D space can be extracted without difficulties of stereo matching. In the simple voting method mentioned above, the whole object shape cannot be measured, since only the previously extracted feature points are back projected into the object space. On the contrary, the pixel voting method, proposed here, can recover the whole object surface by using all the Back projection lines Fig. 1 . The simple voting with back projection lines.