INVESTIGATING MULTIMODAL INTERACTIONS FOR THE DESIGN OF LEARNING ENVIRONMENTS: A CASE STUDY IN SCIENCE LEARNING by STAMATINA ANASTOPOULOU

thinking was related to interactions in the symbolic world. Considering visual representations and the operations taken place when people interact with them, the ‘external cognition’ perspective was explored for assessing more effectively how technological innovations in graphical representations should be approached. The ‘external cognition’ perspective focuses on the cognitive processing involved when interacting with external representations, the properties of the internal and the external structures, and the cognitive benefits of different graphical representations. There are three main advantages that people gain while interacting with representations: computational offloading that refers to the cognitive benefits from a representation, re-representation that relates to their structural properties, and graphical constraining that relates to possible processing mechanisms. Whether the user will take these advantages from interacting with representations depends on the level of expertise, the knowledge domain and the type of task (Scaife 1996). To be able to benefit from the advantages of using representations, learners need support in interpreting the graphs and be able to use them repetitively in context. The view of mental interactions with graphical representations was extended to include physical interactions with them. To explore physical interactions with representations, the focus is on multimodal interactions within the digitally enhanced world (M-DEW). These M-DEW interactions were explored regarding the transition between the real-world and symbolic world, their impact in learning and in the symbolic world. Regarding the transition between real-world and symbolic world interactions, it was argued that the process of combining real-world and symbolic modalities facilitate the linking between the two worlds. Coupling the physical activity with the relevant symbols and linking physical changes to symbolic variables is identified as a way to facilitate the smooth interplay from the real to the symbolic world and vice versa. Linking the real world to the symbolic was regarded as a repetitive circular process where learners’