Virtual Mind Meeting 9th September: Professor Nathaniel Daw

In many learning and decision scenarios, especially sequential tasks like mazes, it is easy to state an objective function for optimal choice, but difficult to actually compute it: for instance because this can require enumerating many possible future trajectories. This motivates a variety of simpler but less accurate approximations believed to be used as shortcuts by the brain, which then, in turn, raise questions about when a resource-efficient agent should invest in evaluating candidate actions more carefully. Previous work has used a simple all-or-nothing version of this reasoning as a framework to explain many phenomena of automaticity, habits, and compulsion in humans and animals: a rational cost-benefit account for why and when we exhibit habits and slips of action.

Here, I present a more finegrained theoretical analysis of deliberation, which attempts to address not just whether to deliberate vs. act, but which of many possible actions and trajectories to consider. Empirically, I first motivate and compare this account to nonlocal representations of spatial trajectories in the rodent place cell system, which are thought to be involved in planning. I also consider its implications, in humans, for variation over time and situations in subjective feelings of mental effort, boredom, and cognitive fatigue. Finally, I present results from a new study using magnetoencephalography in humans to measure subjective consideration of possible trajectories during a sequential learning task, and study its relationship to rational prioritization and to choice behavior.