A fuzzy framework for human hand motion recognition

Unconstrained human hand motions consisting of grasp motion and inhand manipulation lead to a fundamental challenge that many recognition algorithms have to face, in both theoretical and practical development, mainly due to the complexity and dexterity of the human hand. The main contribution of this thesis is a novel fuzzy framework of three proposed recognition algorithms. This consists of extended Time Clustering (TC), Fuzzy Gaussian Mixture Model (FGMM) and Fuzzy Empirical Copula (FEC), using numerical values, Gaussian pattern and data dependency structure respectively in the context of optimal real-time human hand motion recognition. First of all, a fuzzy time-modeling approach, TC, is proposed based on fuzzy clustering and Takagi-Sugeno modeling with a numerical value as output. The extended TC is not only capable of learning repeated motions from the same subject but also can effectively model similar motions from various subjects. The recognition algorithm itself can identify the start point and end point of the testing motion. It is applicable to motion planning directly transfered from the recognition result. Secondly, FGMM is developed to effectively extract abstract Gaussian patterns to represent components of hand gestures with a fast convergence. The dissimilarity function in fuzzy C-means, which maintains the exponential relationship between membership and distance, is refined for FGMM with a degree of fuzziness in terms of the membership grades. Not only does it possess non-linearity but it also offers the characteristic of computationally inexpensive convergence. It is applicable to applications which have a small model storage space and require a method to generate the desired trajectory. Thirdly, FEC is proposed by integrating the fuzzy clustering by local approximation of memberships with Empirical Copula (EC). To save the computational cost, fuzzy clustering reduces the required sampling data and maintains the interrelations before data dependence structure estimation takes over. FEC utilizes the dependence structure among the finger joint angles to recognize the motion type. It is capable of effectively recognizing human hand motions for both single subject and multiple subjects with a few training samples. It can be used in applications requiring high recognition rate and no desired trajectory with limited training samples. All the proposed algorithms have been evaluated on a wide range of scenarios of human hand recognition: a) datasets including 13 grasps and 10 in-hand manipulations; b) single subject and multiple subjects. c) varying training samples. The experimental results have demonstrated that all the proposed methods in the framework outperform Hidden Markov Model (HMM) and Gaussian Mixture Model (GMM) in terms of both effectiveness and efficiency criteria.