“A distributional code for value in dopamine-based reinforcement learning”
Matthew Botvinick, MD, PhD
Director of Neuroscience, DeepMind

Thursday, November 21, 2019
4:00pm
328 Mellon Institute
Twenty years ago, a link was discovered between the neurotransmitter dopamine and the computational framework of reinforcement learning. Since then, it has become well established that dopamine release reflects a reward prediction error, a surprise signal that drives learning of reward predictions and shapes future behavior. According to the now canonical theory, reward predictions are represented as a single scalar quantity, which supports learning about the expectation, or mean, of stochastic outcomes. I’ll present recent work in which we have proposed a novel account of dopamine-based reinforcement learning, and adduced experimental results which point to a significant modification of the standard reward prediction error theory. Inspired by recent artificial intelligence research on distributional reinforcement learning, we hypothesized that the brain represents possible future rewards not as a single mean, but instead as a probability distribution, effectively representing multiple future outcomes simultaneously and in parallel. This idea leads immediately to a set of empirical predictions, which we tested using single-unit recordings from mouse ventral tegmental area. Our findings provide strong evidence for a neural realization of distributional reinforcement learning.

Presenter: Darby Losey

Time: 12:30 PM, November 25th
Location: Mellon Institute 130
Title: Evidence of a memory trace in motor cortex after short‐term learning
Committee: Steve Chase, Rob Kass, Matt Smith, Byron Yu

Abstract: Does learning a new task change the neural activity patterns that the brain uses to perform a previously learned “original” task? We hypothesized that neural activity used to re-perform an original task would remain appropriate for the recently learned task. To address this, we leveraged a brain-computer interface (BCI) in the primary motor cortex of rhesus macaques, where the mapping between neural activity and behavior is specified by the experimenters. Experiments utilized an “A-B-A” block design. Monkeys controlled the cursor with BCI mapping A. Switching to mapping B induced learning, and after several hundred trials, mapping A was reinstated in a “washout” period. We found evidence of a memory trace, in that neural activity late in the washout period remained appropriate for mapping B despite the monkey proficiently controlling the cursor with mapping A. This may be a mechanism by which the brain could more rapidly learn when re-exposed to the same perturbation, a phenomenon known as “savings”.

The Pittsburgh Hearing Research Center,

Department of Otolaryngology Presents a Seminar:

“Novel Modes of Synaptic Transmission Used by Cerebellar Interneurons”

Laurence Trussell, PhD

Professor, Oregon Hearing Research Center

Senior Scientist, Vollum Institute
Oregon Health & Science University

Tuesday, November 26, 2019

4:00-5:00pm

Eye and Ear Institute 

5^th floor, Boardroom 520

http://phrc.pitt.edu/news-and-events