Exploring Mixed-Initiative Interaction for Learning with Situated Instruction in Cognitive Agents

Human-agent interaction for learning with instruction can be viewed on a continuum of instructor/agent control. At one extreme are systems that learn by instructor-driven interactions, such as learning by demonstration, examples, or imitation. The other extreme of the continuum is occupied by systems where instructor interaction is limited to responding to the questions posed by the agent or to providing feedback on agent’s performance. There are advantages to an approach that explores a mixed-initiative instructional dialog. First, an agent that can assume control can guide its own learning by requesting clarifications, asking for missing information, and correcting instructor's response based on its own understanding of the state. Second, an agent that can relinquish control can take advantage of the instructor's knowledge of task structuring and goal decomposition. In a mixed control setting, the instructor can ask the agent for information regarding its state and the environment, verify an agent's learning by questioning the agent, and provide corrections. To be able to maintain the state of interactions with the instructor while acting in the environment, and to be able to learn from these instructions in the context they were provided in, the agent needs a model of task-oriented interaction. Such a model is required to support the properties described below. 1. Both the instructor and the agent can assume control of the interactions at any time. 2. The interaction model provides a context for instructor's elicitation, allowing the agent to take relevant actions. 3. The interactions by the agent should be informed by agent's reasoning, learning and acting mechanisms. 4. The interaction model and the sequence of interactions should inform agent's learning. A Motivating Example To demonstrate our implementation of the interaction model and how it informs agent’s learning, we use a scenario in a grid-world type domain, TankSoar (Figure 1). The figure also describes the state representation used and actions known to the agent. The sequence of actions/subtasks required to successfully complete the required task depends on whether the agent is carrying a missile. If the agent is carrying a missile, the task execution would involve pointing the tank in at the enemy tank (align tank), and firing the missile. However, if the agent is not carrying a missile, the task involves picking up the missile (pick up missile), aligning the tank and firing. The environment is partially observable to the instructor and the task is unknown to the agent, necessitating mixed initiative, bi-directional information transfer.