Emulators as sources of hidden cognitive variables

I focus on the distinction between sensation and perception. Perceptions contain additional information that is useful for interpreting sensations. Following Grush, I propose that emulators can be seen as containing (or creating) hidden variables that generate perceptions from sensations. Such hidden variables could be used to explain further cognitive phenomena, for example, causal reasoning. I have a great deal of sympathy for Grush’s emulator model. Albeit still rather programmatic, it promises a powerful methodology that can generate a multitude of applications in the cognitive sciences. Grush presents some evidence concerning the neural substrates of the emulators. However, this evidence is based on different kinds of neuroimaging. In my opinion, one should rather be looking for functional units in the brain, described in neurocomputational terms that can be interpreted as some kind of Kalman filter. At a low level, the example from Duhamel et al. (1992) concerning saccade anticipation seems to be such a system. However, the functional units should be searched for at higher levels of cognition as well. What ought to be developed is a way of combining the modeling techniques of artificial neuron nets with the control theoretical principles of Kalman filters (see the volume by Haykin [2001] for some first steps). What is needed, in particular, is an account of how a Kalman filter can adapt to the successes or failures of the controlled process. As used in traditional control theory, Kalman filters operate with a limited number of control variables. In his presentation in section 2.3, Grush presumes that the emulator has the same set of variables as the process to be controlled. Although he notes that this is a special case and mentions that the variables of the emulator may be different from those of the process itself, he never presents alternative versions of the filters. Now, from the perspective of the evolution of cognition, the distinction between sensation and perception that Grush makes in section 5.1 is of fundamental importance (Gärdenfors 2003; Humphrey 1993). Organisms that have perceptions are, in general, better prepared for what is going to happen in their environment. My proposal is that perceptions are generated by emulators and they function as forward models. One important property of an emulator is that it does not need to rely exclusively on the signals coming from sense organs; it can also add on new types of information that can be useful in emulating. As a matter of fact, Grush (1998) has written about this possibility himself: The emulator is free to “posit” new variables and supply their values as part of the output. A good adaptive system would posit those variables which helped the controller [. . .] They are variables which are not part of the input the emulator gets from the target system. They may be the actual parameters of the target system, they may not. But what is important is that the emulator’s output may be much richer than the sensory input it receives from the target system. (emphasis in original) It does not matter much if the added information has no direct counterpart in the surrounding world as long as the emulation produces the right result, that is, leads to appropriate control signals. The information provided by these variables is what generates the difference between sensations and perceptions. For example, when the system observes a moving object, its sensations consist only of the positions of the object, whereas the forces that influence the movement of the object are not sensed. However, if the system has been able to extract “force” as a hidden variable and relates this to the sensations via something like Newton’s Second Law, then the system would be able to make more efficient and general, if not more accurate, predictions. In section 2.2, Grush makes the point that emulators must have a certain degree of plasticity. This is not sufficient: A general theory must also account for how an emulator can learn to control a system. Supposedly, it slowly adjusts its filter settings (and set of variables) on the basis of some form of reward or punishment feedback from the process to be controlled. This would be analogous to how artificial neuron networks learn. Such a form of learning may pick up higher-order correlations between input and output. These correlations may be expressed by the hidden variables of the emulator. The hidden variables of the multimodal emulators that Grush discusses in section 6.1, may provide the system (the brain) with cognitive abilities such as object permanence. More generally, one would expect the multimodal emulator to represent the world in an object-centered framework, rather than in a viewer-centered one (Marr 1982). As Grush (1998) writes: “[S]pace is a theoretical posit of the nervous system, made in order to render intelligible the multitude of interdependencies between the many motor pathways going out, and the many forms of sensory information coming in. Space is not spoon-fed to the cognizer, but is an achievement.” Another speculation is that phenomena related to categorical perception are created by the hidden variables of the emulator. More generally, different kinds of emulators may produce the variables that are used in causal reasoning. An interesting finding is that there is a substantial difference between humans and other animal species. As has been shown by Povinelli (2000) and others, monkeys and apes are surprisingly bad at reasoning about physical causes of phenomena. Tomasello (1999, p. 19) gives the following explanation of why monkeys and apes cannot understand causal mechanisms and intentionality in others: “It is just that they do not view the world in terms of intermediate and often hidden ‘forces,’ the underlying causes and intentional/mental states, that are so important in human thinking.” On the other hand, even very small human children show strong signs of interpreting the world with the aid of hidden forces and other causal variables. Gopnik (1998, p. 104) claims that “other animals primarily understand causality in terms of the effects of their own actions on the world. In contrast, human beings combine that understanding with a view that equates the causal power of their own actions and those of objects independent of them.” Apparently, humans have more advanced causal emulators than other animals. Finally, as Grush mentions in section 6.3.2, another relevant area is our “theory of mind,” that is, the ability of humans to emulate (yes, not simulate) the intentions and beliefs of other individuals. An important question for future research then becomes: Why do humans have all these, apparently very successful, emulators for causes and a theory of mind, and why do other species not have them? A research methodology based on emulators and Kalman filters may provide the right basis for tackling these questions. Commentary/Grush: The emulation theory of representation: Motor control, imagery, and perception BEHAVIORAL AND BRAIN SCIENCES (2004) 27:3 403