3D Modelling with Vision Based Interaction in Semi-Immersive Enviroments

We have developed a 3D modelling system that can provide a stereo visualization in a back-projection screen environment. It takes a wireless interaction with user’s head and hand supported by a Vision Tracking System. Our model representation is based on the MeshLibrary, which is optimized for real-time rendering and can accommodate for changes of both vertex positions and mesh connectivity. Thus our modeller allows a user to manipulate and deform the meshes in a reliable and robust way. The mesh is controlled by tracking and recognizing the hand gestures, which enables natural and intuitive interaction without a particular hand-held interface. The user’s eye positions are approximated by head tracking and are used to render a optimal stereo view of the 3D mesh models at any given moment. This paper describes the infrastructure to construct a semi-immersive interactive environment, and it gives an overview of the hand gesture based 3D modelling system.