A neural network model of reinforcement-driven acquisition and performance of timed action switching in corticostriatal circuits

A central aspect of intelligence is the ability to apprehend and exploit temporal regularities in information streams. Efficient selection of actions often requires sensitivity to the time elapsed since event onsets. This thesis investigates neural mechanisms underlying learned internal representations of temporal intervals and their linkage via reinforcement learning to the production of timed actions. The focus is on interval timing in the seconds-to-minutes range within which most timed choices occur. Instrumental conditioning tasks have shed light on the underlying neural structures, pharmacology, and psychophysical properties associated with interval timing. This thesis presents a novel, neurobiologically plausible network model of reinforcement-driven interval timing. The model learns to control the temporal onset and offset of voluntary actions using the experienced outcome of each action as a teaching signal. Simulations of the model replicate experimental observations from the fixed interval (FI) procedure, the peak interval (PI) procedure, and the free-operant psychophysical procedure (FOPP).;A statistical regularity observed in interval timing studies is the scalar property, a temporal analogue of the Weber-Fechner law. This property can be achieved using various combinations of mechanisms and assumptions, so further constraints are needed from neurobiology. Consistent with observations of ramping neurons in frontal cortex, the thesis demonstrates the scalar property for a neural system in which timing is governed by a bounded integrator whose adaptive integration rate becomes proportional to the reinforcement rate experienced during task exposures. Neural and behavioral data further implicate cortico-basal ganglia circuitry, and reveal that dopamine and acetylcholine have distinctive effects on learning and timed action.;The simple reinforcement rate-based model was therefore embedded into a more capable neurobiological model, in which learned stimulus- and context-sensitive representations become selective for particular temporal intervals. Phasic dopamine responses constitute teaching signals that establish such time-sensitive cortical representations, and also serve in the credit-assignment of these representations to basal ganglia pathways that control response selection. Competition among the response-selecting cells determines the time of switching between response options. This new circuit model is not solely dedicated to timing, but exhibits interval timing as an aspect of learned instrumental behavior that efficiently exploits intermittent resources.