Hype or Ready for Prime Time?: Speech Recognition on Mobile Handheld Devices (MASR)

The pervasiveness of mobile handheld devices and advancement in real-time continuous speech recognition technology has opened up a wide range of research opportunities in human-computer interaction for those devices. On the one hand, there has been an increasing amount of research on developing user-friendly speech recognition solutions and applications for mobile handheld devices. On the other hand, there are many distinct challenges in mobile speech recognition. Aiming to gain a good understanding of this emerging yet challenging area and provide a research map, this paper presents a state-of-the-art overview of this field. We will discuss three main architectures of mobile speech recognition systems, analyze their strengths and weaknesses, introduce some major research issues in the field, and highlight a number of major applications of speech recognition on handheld devices. The authors will also shed some light into important future research issues as a road map for researchers and practitioners. DOI: 10.4018/jhcr.2012100103 International Journal of Handheld Computing Research, 3(4), 40-55, October-December 2012 41 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. handheld devices, interacting with them has not always been effective and pleasant, which often suffers from a variety of significant usability problems attributable to their inherent physical constraints, such as small screen size, restricted interaction mechanisms, and low memory space. In particular, the physical or soft keypads of those devices are considerably small, clumsy to interact with, and error prone. Even with the latest touch-sensitive screens of cell phones such as Apple’s iPhones and HTC’s Android phones, the interaction through a soft keyboard remains a challenge. Users have to frequently zoom in or out in order to resize the content and to switch from one application to another, which increases users’ cognitive load, especially when users are on the move. In addition, users’ ability to interact with devices by hand could be severely hampered in situations when users have one or both hands occupied by something else, or are physically impaired with hand or vision problems. As a result, developing alternative user-friendly interaction mechanisms for handheld devices is essential. One natural way to solve or alleviate the problem of interaction with a handheld device is to enable speech input. Empowered by automatic speech recognition (ASR) technology, speech input can be much faster and easier than manual input/interaction through buttons, stylus, and keyboards. Speech is a natural skill of human beings. Speech input does not require much attention or direct observation to use a device; it can reduce or even completely eliminate the need for hand use during interaction, making mobile devices much more accessible. In particular, speech recognition technology for mobile handheld devices (MASR) could fundamentally help users who are physically challenged or visually impaired interact with those devices. In the past decade, there has been an increasing amount of research efforts toward developing MASR solutions (e.g., PocketSphinx) and applications. Some latest cell phones such as iPhone 4, Nexus One Smartphone, and Samsung Galaxy S III smartphone are equipped with speech recognition technology. In the meantime, this research field is still in its infancy. The growth of research interest is also accompanied by the increasing awareness of the actual intricacies of MASR, which often involves large vocabularies and complex processing. MASR not only inherits the challenges in recognizing natural speech from ASR systems on desktop computers, such as co-occurring ambient noise, continuous utterance, diversity in individual speakers’ pronunciation, and outof-vocabulary words (Zhou et al., 2006), but also faces significant resource barriers caused by the unique constraints of handheld devices and wireless networks. In addition, the often noisy environment could also pose challenges to the performance of speech recognition. This raises a key question about whether or not the ultimate goals of achieving full-fledged automatic speech recognition on mobile handheld devices can be truly achieved. This research attempts to answer the following research questions through an overview of the state-of-the-art MASR research, technology, and major applications: What are the unique requirements for MASR technologies to achieve their goals? What are alternative architectural designs and technical developments to better meet those requirements? What are major challenges and issues that need to be addressed in future research in order to maximize the potential of MASR? We searched for papers on MASR technology and applications published in IEEE Xplore and ACM digital libraries, as well as a couple of other related journals such as Personal and Ubiquitous Computing and Pervasive and Mobile Computing. We also used Google Scholar as a supplementary source for finding relevant papers. The keywords used in search included mobile computing, mobile devices, speech recognition, voice/speech input, and multimodal interface. Most papers we reviewed were published within the past six years, aiming to reflect the most recent research. The rest of the paper is organized as follows. First, we will introduce the concept of MASR and its unique challenges and describe three major types of MASR system architecture and compares their strengths and weaknesses, 14 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the publisher's webpage: www.igi-global.com/article/hype-ready-prime-time/73805