# 10.02: Biology of PFC/BG and Dopamine Supporting Robust Active Maintenance

• Contributed by

Figure $$10.1$$: Schematic of functional relationships and connectivity of PFC, BG, and SNc phasic dopamine signals in relationship to basic loop between sensory input and motor output. The PFC provides top-down context and control over posterior cortical processing pathways to ensure that behavior is task and context appropriate. The BG exerts a disinhibitory gating influence over PFC, switching between robust maintenance and rapid updating. The SNc (substantia nigra pars compacta) exhibits phasic dopamine (DA) firing to drive learning and modulation of BG circuits, thereby training the BG gating signals in response to task demands (external reward signals).

The overall connectivity of the areas that are particularly important for executive function are shown in Figure 10.1, in relation to the sensory and motor processing associated with posterior cortex (temporal, parietal and occipital lobes) and motor frontal cortex (i.e., frontal cortex posterior to the prefrontal cortex). The PFC is interconnected with higher-level association cortical areas in posterior cortex where highly processed and abstracted information about the sensory world is encoded. It also interconnects with higher-level motor control areas (premotor cortex, supplementary motor areas), which coordinate lower-level motor control signals to execute sequences of coordinated motor outputs. With this pattern of connectivity, PFC is in a position to both receive from, and exert influence over, the processing going on in posterior and motor cortex.

Figure $$10.2$$: Parallel circuits through the basal ganglia for different regions of the frontal cortex -- each region of frontal cortex has a corresponding basal ganglia circuit, for controlling action selection/initiation in that frontal area. Motor loop: SMA = supplementary motor area -- the associated striatum (putamen) also receives from premotor cortex (PM), and primary motor (M1) and somatosensory (S1) areas -- everything needed to properly contextualize motor actions. Oculomotor loop: FEF = frontal eye fields, also receives from dorsolateral PFC (DLPFC), and posterior parietal cortex (PPC) -- appropriate context for programming eye movements. Prefrontal loop: DLPFC also controlled by posterior parietal cortex, and premotor cortex. Orbitofrontal loop: OFC = orbitofrontal cortex, also receives from inferotemporal cortex (IT), and anterior cingulate cortex (ACC). Cingulate loop: ACC also modulated by hippocampus (HIP), entorhinal cortex (EC), and IT.

The Basal Ganglia (BG), which consists principally of the striatum (caudate, putamen, nucleus accumbens), globus pallidus, and subthalamic nucleus, is densely interconnected with the PFC by way of specific nuclei of the thalamus Figure 10.2. As described in detail in Motor Control and Reinforcement Learning, the BG provides a dynamic, adaptive gating influence on the frontal cortex, by disinhibiting the excitatory loop between PFC and the thalamus. In the context of the PFC, this gating influence controls the updating of information that is actively maintained in the PFC, using the same mechanisms that control the initiation of motor actions in the context of motor control. Also, top-down projections from PFC to the subthalamic nucleus support a type of inhibitory control over behavior by detecting conditions under which ongoing action selection should be halted or switched and preventing the rest of the BG circuitry from gating the planned motor action.

The final major component of the executive control system consists of the substantia nigra pars compacta (SNc) and several other associated brain areas that together drive phasic dopamine neuromodulation of the BG, resulting in reinforcement learning of its gating actions. This system, summarized computationally using the PVLV model as described in Motor Control and Reinforcement Learning, interacts with the active maintenance of information in PFC to be able to reinforce a gating signal in the BG that leads to subsequent good performance and reward later in time. This time-travel property of the phasic DA reinforcement learning is essential for training a system that maintains information over time.

In the following subsections, we summarize the biological properties of each of these systems and their relevance to executive function. The prefrontal cortex basal ganglia working memory model (PBWM) then integrates all of these elements into a functioning computational model that can perform complex executive function tasks, as we explore in the remainder of the chapter.

## Robust Active Maintencance in the PFC

Figure $$10.3$$: An example of sustained delay period activity (Panel B). Histogram (background dots) and curve of activity rate for an individual cell recorded in the frontal eye fields (FEF) during a delayed saccade task. The target stimulus is only on briefly at the beginning of the trial (Panel A, Targ, Mem.). This cell maintained its activity during the delay so as to enable other cells to generate a correct saccade at the end of the trial (Panels D,E). Adapted from Sommer & Wurtz, 2000, Figure 2.

The ability of PFC neurons to exhibit sustained active firing over delays, as initially discovered by (Fuster & Alexander,1971; Kubota & Niki, 1971), is shown in Figure 10.3 panel B ("Neuron with delay signal"), in the context of the delayed saccading task described in the introduction. Other subsets of PFC neurons also exhibit other firing patterns, such as responding transiently to visual inputs (Panel C) and initiating movements (Panel D). This differentiation of neural response patterns in PFC has important functional implications that we capture in the PBWM model described later.

Figure $$10.4$$: Two types of reverberant loops can support actively maintained representations in cortical tissue: 1) Corticocortical interconnections among pyramidal neurons within the same PFC stripe (horizontal blue arrows); 2) Thalamocortical connections between PFC and thalamic relay cells (TRC's) (vertical blue arrows). Both use mutually supportive recurrent excitation plus intrinsic maintenance currents (NMDARs; mGluRs).

There are two primary biological mechanisms that enable PFC neurons to exhibit sustained active firing over time:

• Recurrent excitatory connectivity: Populations of PFC neurons have strong excitatory interconnections (Figure 10.4), such that neural firing reverberates back-and-forth among these interconnected neurons, resulting in sustained active firing. There are two types of such connections: 1) a corticocortical loop among pyramidal cells in the same PFC stripe, and; 2) a corticothalamocortical loop between lamina VI pyramidal cells in PFC and the thalamic relay cells that project to that particular group of cells.
• Intrinsic excitatory maintenance currents: At the synapses formed by both of the recurrent excitatory loops there are NMDA and metabotropic glutamate (mGluR) receptors that, once opened by high frequency activity, provide a longer time window of increased excitability so as to keep reverberant activity going. Recall from the Learning Chapter that the NMDA channel requires the neuron to be sufficiently depolarized to remove the Mg+ (magnesium) ions that otherwise block the channel. This activity-dependent nature of the NMDA channel makes it ideally suited to providing a "switched" or dynamically gated form of active maintenance -- only those neurons that have already been sufficiently activated will benefit from the increased excitation provided by these NMDA channels. This provides a "hook" for the basal ganglia system to control active maintenance: when the thalamic neurons are disinhibited via a BG gating action, the ensuing burst of activity enables a subset of PFC neurons to get over their NMDA Mg+ block thresholds, and thereby continue to fire robustly over time.

## Functional Specialization Across PFC Areas

The mechanisms for robust active maintenance exist across the PFC, but different PFC areas have been associated with different contributions to overall executive function. We will explore the idea later in this chapter that these different functional contributions can be explained in terms of differences in connectivity of these PFC areas with other parts of the brain, within the context of the unifying framework that all PFC areas share robust active maintenance as a critical feature.

Figure $$10.5$$: Brodmann numbers for areas of the prefrontal cortex, each of which has been associated with a different mixture of executive functions. Reproduced from Fuster, 2001.

Anatomically, the frontal lobes constitute those cortical areas anterior to the central sulcus. Immediately anterior to the central sulcus, and thus most posteriorly in frontal cortex, is the primary motor cortex (M1), which is most prominently seen on the lateral surface but extends all the way over the dorsal surface and onto the medial side. Contiguous tissue roughly anterior to M1 makes up planning motor areas, the premotor (PM) cortex (laterally) and supplementary motor areas (SMA, pre-SMA; medially). Then, anterior to that are the PFC areas, labeled with their Brodmann numbers in Figure 10.5.

Figure $$10.6$$: The What vs. How distinction for posterior cortex can be carried forward into prefrontal cortex, to understand the distinctive roles of the ventral and dorsal areas of PFC. Reproduced from O'Reilly (2010).

At the broadest level, the PFC areas can be divided along the major axes of medial vs. lateral and dorsal vs. ventral. Generally speaking, the lateral PFC areas are interconnected with sensory and motor areas in more posterior cortex, and are thought to play a role in controlling the processing in these areas. In contrast, the medial PFC areas are more strongly interconnected with subcortical brain areas associated with affective and motivational functions. Functionally we can characterize the lateral areas as being important for "cold" cognitive control, while the medial areas are important for "hot" emotional and motivational processing Figure 10.6. However, this distinction is not as clear cut as it sounds, as even the lateral areas are subject to modulation by motivational variables and BG/dopamine gating signals based on the extent to which maintained cognitive information is predictive of task success (a form of reward).

The functional significance of the dorsal vs. ventral distinction has been considerably more controversial in the literature, but anatomically it is clear that dorsal PFC areas interconnect more with the dorsal pathway in the posterior cortex, while ventral PFC interconnects with the ventral posterior cortex pathway. As we saw in the Perception Chapter, the dorsal pathway in posterior cortex is specialized for perception-for-action (How processing): extracting perceptual signals to drive motor control, while the ventral pathway is specialized for perception-for-identification (What processing). This functional specialization in posterior cortex can be carried forward to the associated dorsal and ventral areas of PFC (Figure 10.6), such that dorsal lateral PFC (DLPFC) areas are particularly important for executive control over motor planning and the parietal cortex pathways that drive motor control, while ventral lateral PFC (VLPFC) areas are particularly important for control over the temporal lobe pathways that identify entities in the world, and also form rich semantic associations about these entities.

On the medial side, the dorsal medial PFC is also known as the anterior cingulate cortex (ACC), which has been shown to encode the affective aspects of motor control variables (e.g., how much effort will an action take, what is its probability of success, how much conflict and uncertainty is there in selecting a response), which is consistent with a "hot how" functional specialization. Dorsomedial PFC areas also project to the subthalamic nucleus within the BG, and serve to delay motor responding to prevent impulsive choice under difficult response selection demands (Frank, 2006; Aron et al, 2007; Cavanagh et al, 2011). The ventromedial areas of PFC (VMPFC) including the orbital frontal cortex (OFC) have been shown to encode the affective value of different sensory stimuli, consistent with the idea that they are the "hot what" areas. See O'Reilly (2010) for more discussion of the What/How functional specialization idea.

### Substructure within PFC Areas: Stripes

Figure $$10.7$$: Four separate dye injections in monkey PFC demonstrate four sets of sausage-shaped stripes. Reproduced from Levitt et al., 1993, Figure 6.

Figure $$10.8$$: Higher magnification of lower left injection from Figure 10.7. Lower image is same as upper but differentiates among two levels of dye concentration, illustrating multiple stripe clusters for each injection (black, gray). Reproduced from Levitt et al., 1993, Figure 5.

Within each functional PFC area, there is some interesting topographic organization of neurons into hypercolumns, macrocolumns or stripes (each of these terms is typically associated with a similar type of neural organization, but in different parts of the cortex, with stripes being specific to the PFC) (Figure 10.7, Figure 10.8). In all areas of cortex, one can identify the smallest level of neural topological organization as a cortical column or microcolumn (to more clearly distinguish it from the larger macrocolumn), which contains roughly 20 pyramidal neurons in a region that is roughly 50 microns across. A stripe contains roughly 100 of these microcolumns, generally organized in an elongated shape that is roughly 5 microcolumns wide (250 microns) by 20 microcolumns long (1000 microns or 1 millimeter). Each such stripe is interconnected with a set of roughly 10 or more other stripes, which we can denote as a stripe cluster. Given the size of the human frontal cortex, there may be as many as 20,000 stripes within all of frontal cortex (including motor areas).

In PFC and other areas, neurons within a microcolumn tend to encode very similar information, and may be considered equivalent to a single rate-coded neuron of the sort that we typically use in our models. We can then consider an individual stripe as containing roughly 100 such rate-coded neuron-equivalents, which provides sufficient room to encode a reasonably large number of different things using sparse distributed representations across microcolumns.

Functionally, we hypothesize in the PBWM model that each stripe can be independently updated by a corresponding stripe-wise loop of connectivity with an associated stripe of neurons through the BG system. This allows for very fine-grained control by the BG over the updating and maintenance of information in PFC, as we describe next.

## Basal Ganglia and Dynamic Gating

Figure $$10.9$$: How the basal ganglia (BG) can modulate active maintenance in the PFC. (A) In the default state of no BG activity, or NoGo (indirect) pathway firing in the striatum, the PFC continues to maintain information in an active state. The SNr (substantia nigra pars reticulata) or GPi (globus pallidus internal segment) exhibits tonic (sustained) activity, that inhibits neurons in the thalamus, thereby shutting down the thalamocortical loop. (B) Go (direct) pathway firing triggers updating of a PFC stripe to encode new information (e.g., new stimulus inputs that are behaviorally relevant). This occurs by inhibiting the SNr/GPi neurons, thereby opening up the thalamocortical loop, resulting in a burst of activity in the PFC that drives updating to a new pattern of neural firing, including new intrinsic maintenance currents that will continue to sustain this new pattern going forward. This mechanism is identical to that for gating motor actions via BG interactions with pre/motor areas in parallel circuits (see Motor Chapter). Reproduced from O'Reilly, 2006, Science

As we discussed in the Motor and Reinforcement Learning Chapter, the Basal Ganglia (BG) is in a position to modulate the activity of the PFC, by virtue of its control over the inhibition of the thalamic neurons that are bidirectionally connected with the PFC (Figure 10.9). In the default state of no striatal activity, or firing of indirect (NoGo) pathway neurons, the SNr (substantia nigra pars reticulata) or GPi (globus pallidus internal segment) neurons tonically inhibit the thalamus. This prevents the thalmocortical loop from being activated, and it is activation of this loop that is thought to be critical for initiating motor actions or updating PFC active memory representations. When the striatal Go (direct) pathway neurons fire, they inhibit the tonic SNr/GPi inhibition, thereby allowing the excitatory thalamocortical loop to be activated. This wave of excitatory activation can activate a new population of PFC neurons, which are then actively maintained until a new Go signal is fired.

## Phasic DA and Temporal Credit Assignment

Figure $$10.10$$: The phasic activity of dopamine (DA) neurons in the midbrain exhibits the appropriate form of time travel required for learning what information is useful to maintain in the PFC. DA moves from firing at the time of a primary reward (R) to the onset of a Conditioned Stimulus (CS) that is predictive of subsequent reward. Items that prove useful for later behavior are effectively CS's, and phasic DA at their onset can drive learning to reliably encode this information into PFC.

Another critical biological mechanism for executive function, which we also discussed in the Motor and Reinforcement Learning Chapter, is the firing of phasic dopamine neurons in the midbrain (ventral tegmental area (VTA) and substantia nigra pars compacta (SNc)) (Figure 10.10). These neurons initially respond to primary rewards (e.g., apple juice), but then learn to fire at the onset of conditioned stimuli (CS's) that reliably predict these primary rewards. This amounts to a form of time travel that solves a critical problem for the PFC active maintenance system: how does the system learn what to maintain, given that the decision for what to maintain typically occurs well in advance of the subsequent value of having maintained something useful. If you think of the maintenance of useful information in the PFC as a kind of CS (because they should reliably be associated with positive outcomes), then the dopamine neurons will learn to fire at the onset of such a CS. Having this phasic DA signal at CS onset can then reinforce the decision to maintain this information in the first place, thus solving the time travel problem.

The computational model described next incorporates this key idea, by having the phasic DA signal at CS onset drive learning of the BG Go neurons that update new information into PFC active maintenance. The model shows that this core idea is sufficient to support the learning of complex executive function tasks.