Driving Temporally Precise Learning in Individual Premotor Neurons using Closed-Loop Neurofeedback

Abstract

Much of human existence is based on our ability to learn complex sequences of motor movements. Speech, writing, and tool use all require activating a series of different muscles in a precisely timed pattern, and these patterns are learned through a long process of trial and error. How does the neural circuitry in our motor system learn to generate the activity patterns that drive these sequences? This question can be explored by studying a similarly precise learned motor pattern in a different organism, the learned song of the songbird zebra finch.

Zebra finches learn to sing a stereotyped song through a process of vocal experimentation and comparison to an internal template. Every time a bird sings, it varies the acoustic parameters of its song and determines whether each variation brings the song closer to its internal template. Variations that result in a better match are then repeated in subsequent renditions of the song, in a trial and error process suggestive of reinforcement learning. The learning process requires a basal ganglia-thalamocortical loop called the anterior forebrain pathway (AFP) that is similar to basal ganglia-thalamocortical circuitry in mammals. Existing evidence suggests that the AFP learns a time-dependent bias signal that steers the motor pathway to avoid vocal errors. This bias signal is known to be dependent on the cortical output of the AFP known as LMAN (lateral magnocellular nucleus of the anterior nidopallium). However, little is known about the neural code in LMAN that underlies this bias signal, or how this neural code is learned and generated.

We address these questions by building a neural feedback system that allows us to impose correlations between the activity of individual LMAN neurons and a dopaminergic reward signal. We designed a low-latency feedback system that records neural activity from a chronic Neuropixels 2.0 implant, extracts the activity of specific neurons, and plays noise bursts to the bird contingent on the activity of those neurons. We used this system to perform feedback based on the activity of an arbitrarily chosen neuron in LMAN within a given 10 ms window in songs. All birds responded to the feedback by learning to bias the activity of the chosen LMAN neuron up or down within the chosen time window, transiently driving firing rates up by as much as 200 Hz. We observed a remarkable degree of timing precision in the learned bias, with birds able to control the activity of the chosen neuron at single millisecond levels of rise time and jitter. This high degree of precision informs models of the basal ganglia circuit architecture thought to drive learning. We also found the learned bias to be specific to the LMAN neurons correlated with reward, with neighboring uncorrelated neurons exhibiting no change in firing rate during learning. This single-neuron specificity strongly constrains the spatial precision of axonal targeting from thalamic regions that are thought to propagate the learned bias signal from the basal ganglia to LMAN. Finally, we demonstrated that fluctuations in neural activity of a given LMAN neuron drive transient and predictable changes in vocal output approximately 25 milliseconds later, consistent with what is known about signal propagation speeds in the song system. This fact, together with the results of our feedback experiments, combine to confirm our central hypothesis that LMAN drives song learning by independently activating LMAN neurons at precise points in time in order to bias vocal output and avoid vocal errors.