Active tactile exploration using a brain-machine-brain interface

Brain–machine interfaces useneuronal activity recorded from the brain to establish direct communication with external actuators, such as prosthetic arms. It is hoped that brain–machine interfaces can be used to restore the normal sensorimotor functions of the limbs, but so far they have lacked tactile sensation. Here we report the operation of a brain–machine–brain interface (BMBI) that both controls the exploratory reaching movements of an actuator and allows signalling of artificial tactile feedback through intracortical microstimulation (ICMS) of the primary somatosensory cortex. Monkeys performed an active exploration task in which an actuator (a computer cursor or a virtual-reality arm) was moved using a BMBI that derived motor commands from neuronal ensemble activity recorded in the primary motor cortex. ICMS feedback occurred whenever the actuator touched virtual objects. Temporal patterns of ICMS encoded the artificial tactile properties of each object. Neuronal recordings and ICMS epochs were temporally multiplexed to avoid interference. Two monkeys operated this BMBI to search for and distinguish one of three visually identical objects, using the virtual-reality arm to identify the unique artificial texture associated with each. These results suggest that clinical motor neuroprostheses might benefit from the addition of ICMS feedback to generate artificial somatic perceptions associated with mechanical, robotic or even virtual prostheses. Brain–machine interfaces (BMIs) have evolved from 1-d.f. systems tomany-d.f. robotic arms andmuscle stimulators that perform complex limb movements, such as reaching and grasping. However, somatosensory feedback, which is essential for dexterous control, remains underdeveloped in BMIs. With the exception of a few studies combining BMIs with tactile stimuli applied to the body, existing systems rely almost exclusively on visual feedback. Prosthetic sensation has been studied in the context of sensory substitution and targeted reinnervation; however, these approaches have limited application range and channel capacity. To provide a proof-of-concept method of equipping neuroprostheses with sensory capabilities, we implemented a BMBI that extracts movement commands from the motor areas of the brain while delivering ICMS feedback in somatosensory areas to evoke discriminable percepts. This idea received support from our pilot study, in which amonkey responded to ICMS cues with the movements of a BMI-controlled cursor. However, the ICMS cue did not provide feedback of object–actuator interactions in this previous demonstration. TheBMBIdeveloped here allowed active tactile exploration during BMI control (Fig. 1a). Two monkeys (M and N) received multielectrode implants in the primary motor cortex (M1) and the primary somatosensory cortex (S1) (Fig. 1b). They explored virtual objects using either a computer cursor or a virtual image (avatar) of an arm (Supplementary Fig. 1a, b). In ‘hand control’, the monkeys moved a joystick with their left hands to position the actuator. They searched through a set of virtual objects, selected one with a particular artificial texture conveyed by ICMS, and held the actuator over that object to obtain reward (Fig. 1a and Supplementary Fig. 1c, d). During ‘brain control’, the joystick was disconnected and the actuator was controlled by the activity of right-hemisphere M1 neurons. The behavioural tasks varied in the number of objects on the screen, the artificial textures used and the actuator type (Fig. 2a), and were more difficult than previously reported BMI tasks because of the presence of multiple objects in the workspace, a prolonged object selection period and the necessity of interpreting ICMS feedback. ICMS was delivered through two pairs of microwires to the hand representation area of S1 in monkey M (Fig. 1c) and through one pair