Internet of Things Track

UI generation stylesheet Abstract UI to XHTML stylesheet Abstract UI description XHTML [Java ME] Apache Cocoon framework OWL-S description Service User Interface Annotation description Abstract widget type modelwidget type model Abstract parameter type modelparameter type model Figure 2. The Pervasive Service Interaction (Perci) framework architecture. Perci provides for integrated interaction between mobile services and tagged objects. Authorized licensed use limited to: University of Texas at Arlington. Downloaded on January 12, 2010 at 19:20 from IEEE Xplore. Restrictions apply. NOVEMBER/DECEMBER 2009 77 Pervasive Service Interaction with the Internet of Things ing to the service parameters’ abstract widget types. Finally, the user can arbitrarily interact with the parameter tags on the poster and use the UI widgets to collect those tags’ information for further interaction with the service. The prototype uses the abstract parameter types to correctly allocate information from parameter tags to service parameters. Mobile services use the typing model to specify the types of parameter values they need to be invoked, and parameter tags on physical objects specify the type of information they can provide for that purpose. This way, the common use of a global typing model improves interoperability between information and services. PMI supports these interactions by breaking them down to the simple interaction with a few tags. Evaluation We conducted a qualitative user study with 17 subjects (13 male, four female, average age 29) to evaluate the prototype’s interaction design and compare its techniques for PMI.9 All subjects accomplished the same task — ordering a ticket via the movie poster — using these techRelated Work in Physical Mobile Interaction In 1999, Roy Want and his colleagues presented some of the first examples for linking everyday objects (such as books, documents, and business cards) with digital resources (such as electronic documents, URLs, and email addresses) through RFID tags.1 Since then, ubiquitous computing applications have used various technologies to implement the tagging of and interaction with physical objects, including Near Field Communication (NFC),2 Bluetooth, infrared beacons,3 laser pointers,4 GPS, or the recognition of visual markers,5 which will eventually be displaced by markerless image recognition.6 Researchers have built different interaction techniques and metaphors on top of these enabling technologies to make their use more familiar and intuitive. For example, Enrico Rukzio and his colleagues4 use the techniques touching (using NFC), pointing (using a laser pointer), and scanning (using Bluetooth) for interaction with smart-home appliances. Physical hyperlinks use the simple interaction with single NFC/RFID tags as shortcuts for more complex actions,7 such as setting up wireless connections. Hovering visualizes tags’ content before using them.8 Although these examples show single-tag interactions’ simplicity, others rely on more complex interactions with multiple tags. In one study, the authors present a framework for requesting services by touching RFID tags with different functionalities9 — for example, object identification or printing. Derek Reilly and his colleagues have augmented maps with RFID tags, turning them into interactive surfaces that support path-, lasso-, or menu-select with mobile devices.10 Robert Hardy and his colleagues have extended this static physical interface to a dynamic NFC display that projects an application interface onto a grid of NFC tags.11 Users manipulate the projected interface by touching the tags with NFC-enabled mobile devices. The development of physical mobile interaction (PMI) met up with the Internet of Things,12 whose infrastructure uses RFID tags to attach a unique Electronic Product Code (EPC) to objects to identify them on a network for monitoring, for example. PMI complements the Internet of Things because it relies on similar basic technologies. The research done in the Pervasive Service Interaction (Perci) project we describe in the main text is one of the first approaches to build on the integration of the technologies just presented to facilitate user interaction with mobile services and tagged, physical objects from the Internet of Things.