An Autonomous Three-Dimensional Vision Sensor with Ears

This paper describes our newly developed intelligent sensor system which comprises two eyes and four ears on a movable head. I t can acquire its dynamical visual and auditory image of its surrounding 3-D environment while showing humanlike behavior naturally and autonomously. The most important feature of the sensor system is in a unified sensory architecture throughout low-level and intermediate level visual and auditory functions. This enables us to achieve 1) rapid (5ms) and accurate (rt2deg) auditory localization, 2) rapid (0.5s/65536pixel) extraction of motion and texture features, 3) rapid (O.ls/4096pixel) reconstruction of 3-D object profile, 4) rapid (several T V frame times) eye movement and binocular fixation which is activated by auditory localization and motion detection. We describe in this paper the several key items for realizing this sensor.