Learning to predict human behaviour in crowded scenes

Humans are much more predictable in their transit patterns than we expect. In the presence of su cient observations, it has been shown that our mobility is highly predictable even at a city-scale level [1]. The location of a person at any given time can be predicted with an average accuracy of 93% supposing 3 km of uncertainty. How about at finer resolutions such as in shopping malls, in airports, or within train terminals for safety or resource optimization? What are the relevant cues to best predict human behavior within a margin of few centimeters? Recently, Kitani et al. [2] showed that scene semantics provide strong cues for forecasting pedestrians’ trajectories. Helbing et al. [3, 4] also showed that our mobility is influenced by our neighbors, either consciously, e.g., by relatives or friends, or even unconsciously, e.g., by following an individual to facilitate navigation. More broadly, when humans walk in a crowded public space such as a train terminal, mall, or city centers, they obey a large number of (unwritten) common sense rules and comply with social conventions. For instance, as they consider where to move next, they respect personal space and yield right-of-way. The ability to model these rules and use them to understand and predict human motion in complex real world environments is extremely valuable for a wide range of applications from the deployment of socially-aware robots [5] to the design of intelligent tracking systems [6] in smart environments. In this chapter, we present two families of methods to forecast human trajectories in crowded environments. The first one is based on the popular Social Forces model [3] where the causalities behind human navigation is hand-designed by a set of functions that have been carefully chosen based on our understanding of physics underlying social behaviour. The second method is a fully data-driven approach based on Recurrent Neural Networks [7] that does not impose any hand-designed functions or explicit mobility based constraints. The causality behind human mobility is an interplay between both observable and nonobservable cues (e.g., intentions). Humans have the innate ability to “read” one another. When they need to avoid each other, there is an implicit cooperation on where to move next. They have the ability to get along well with each other by preserving a personal distance. These capabilities are often referred to as Social Intelligence [8]. Any forecasting method needs to infer the same behaviors to develop socially-aware intelligent systems. This requires