Neural computations underlying value learning in the ventral tegmental area and orbitofrontal cortex of rhesus macaques

The study of how the brain computes value is essential to understanding the decision making process that organisms engage in throughout their life span. In the 90's, Schultz and colleagues (1997) proposed a pivotal theory describing how dopamine neurons encode a reward prediction error (RPE). The RPE theory elegantly characterized the learning process involved in associating a stimulus or action and reward (Bush and Mosteller 1951; Sutton and Barto 1998; Rescorla and Wagner 1972). The discovery of RPE and its role in reinforcement learning launched a division of decision-making research that has focused on dopamine neurons and its efferents. The following research takes cues from the field and investigates the neural computation of dopamine neurons in value learning and explores the value signals arising from a dopamine neuron efferent, the orbitofrontal cortex.;The primary data supporting the idea that dopaminergic activity encodes an RPE signal has been correlative, showing that dopaminergic firing rates match quantitative predictions of reinforcement learning (RL) models (Schultz et al, 1997; Bayer and Glimcher, 2005; Nakahara et al. 2004). We provide causal evidence for the RPE theory by electrically activating dopamine neurons directly following reward. In our task we observed learning in real time and found that the subject allocated a continuously increasing fraction of its choices to the stimulation-associated action and that this effect was dependent on D2 receptor activation.;The monkey orbitofrontal cortex has been the subject of scrutiny during the last decade by scholars interested in its role in decision-making. Studies have given rise to the notion that neurons in the OFC encode the properties of rewards in their firing rates but that these firing rates are independent of the actions required to obtain those rewards (Padoa-Schioppa & Cai, 2011). We set out to specifically test the hypothesis that OFC value-related neurons in area 13m do not carry information about the action required to obtain that reward. Our detailed analysis of the response fields indicates that these neurons are insensitive to the amplitude or direction of the saccade required to obtain these rewards.