Model-based hierarchical reinforcement learning and human action control

Matthew Botvinick; Ari Weinstein

doi:10.1098/rstb.2013.0480

Model-based hierarchical reinforcement learning and human action control

Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):20130480. doi: 10.1098/rstb.2013.0480.

Authors

Matthew Botvinick¹, Ari Weinstein²

Affiliations

¹ Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ 08540, USA matthewb@princeton.edu.
² Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ 08540, USA.

Abstract

Recent work has reawakened interest in goal-directed or 'model-based' choice, where decisions are based on prospective evaluation of potential action outcomes. Concurrently, there has been growing attention to the role of hierarchy in decision-making and action control. We focus here on the intersection between these two areas of interest, considering the topic of hierarchical model-based control. To characterize this form of action control, we draw on the computational framework of hierarchical reinforcement learning, using this to interpret recent empirical findings. The resulting picture reveals how hierarchical model-based mechanisms might play a special and pivotal role in human decision-making, dramatically extending the scope and complexity of human behaviour.

Keywords: goal-directed behaviour; hierarchy; reinforcement learning.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Decision Making / physiology*
Goals*
Humans
Learning / physiology*
Models, Neurological*
Reinforcement, Psychology