Reinforcement learning processes as forecasters of depression remission

Abstract

Background: Aspects of reinforcement learning have been associated with specific depression symptoms and may inform the course of depressive illness.

Methods: We applied support vector machines to investigate whether blood‑oxygen-level dependent (BOLD) responses linked with neural prediction error (nPE) and neural expected value (nEV) from a probabilistic learning task could forecast depression remission. We investigated whether predictions were moderated by treatment use or symptoms. Participants included 55 individuals (n = 39 female) with a depression diagnosis at baseline; 36 of these individuals completed standard cognitive behavioral therapy and 19 were followed during naturalistic course of illness. All participants were assessed for depression diagnosis at a follow-up visit.

Results: Both nPE and nEV classifiers forecasted remission significantly better than null classifiers. The nEV classifier performed significantly better than the nPE classifier. We found no main or interaction effects of treatment status on nPE or nEV accuracy. We found a significant interaction between nPE-forecasted remission status and anhedonia, but not for negative affect or anxious arousal, when controlling for nEV-forecasted remission status.

Limitations: Our sample size, while comparable to that of other studies, limits options for maximizing and evaluating model performance. We addressed this with two standard methods for optimizing model performance (90:10 train and test scheme and bootstrapped sampling).

Conclusions: Results support nEV and nPE as relevant biobehavioral signals for understanding depression outcome independent of treatment status, with nEV being stronger than nPE as a predictor of remission. Reinforcement learning variables may be useful components of an individualized medicine framework for depression healthcare.

Keywords: Cognitive neuroscience; Depression; Machine learning; Predictive modeling; Reinforcement learning; Task functional imaging.