A New Neural Network Structure for Camera Calibration

In this paper, a new design of neural network is proposed for camera calibration. Unlike most existing methods, where camera parameters are determined so that the projection from 3D points to their corresponding image points is as accurate as possible, the network of multilayer perceptrons designed here learns the transformation from image points to their corresponding rays of sight. Since this is a one-to-one mapping, whereas the projection is a many-to-one mapping, the transformation and its applications are quite straightforward. Being based on a geometric model, a network design process also does not require a tedious step of determining the numbers of hidden layers and nodes for efficient learning with given data. Key-Words: Camera calibration, neural network, multilayer perceptrons, projection, back-projection 3D points to image points is a many-to-one mapping; different 3D points may correspond to the same image point. Thus, stereoscopic backprojection, where a 3D point is uniquely determined from two matched image points, is more appropriate application in this approach [10]. The third approach is using a network designed to be capable of explicit calibration. The ANN presented by Ahmed and his colleagues might be the first of its kind [11,12]. Since the network was designed based on a physical model, the weights of the network synapses were related directly to the position, orientation and optical parameters of a camera calibrated. Therefore, no need for searching a good network structure is required unlike other techniques employing ANNs. This paper describes a new design of ANN for camera calibration. Like Ahmed's it is designed based on a physical camera model and the network can tell camera parameters explicitly. Therefore, all advantages of Ahmed's approach can be found here also. However, unlike almost all existing techniques including Ahmed's, where calibration is done by optimizing the mapping from 3D points to their corresponding 2D image points, the technique proposed in this paper learns the mapping from 2D or 3D points to their rays of sight. Since this is a one-to-one mapping uniquely determinable when a point is given, the projection and back-projection become straightforward and easy to be done. 2. Neural Network Design 2.1 Camera model Assuming pinhole camera model a 3D point at (x,y,z), its corresponding image point at (u,v), and the pinhole at focal distance f all are on the same ray l in the world coordinate system {W} as shown in Fig.1. Therefore, if we know two of the three points, the ray can be uniquely determined. Especially, if the focal point is known, we can find the ray from either an image point or a 3D point. Assuming a 3D frame {I} attached to the image plane as shown in Fig.2, an arbitrary image point is represented as T I v u p ) 0 , , ( = in {I}. From the image point, a ray of sight can be determined as it passes the focal point T I f f v u p ) , , ( 0 0 = in {I}, which can also be represented as T fz fy fx W f p p p p ) , , ( = in {W}. The aiming vector of the ray is defined then as I I