The cerebral cortex or neocortex is composed of roughly 85% excitatory neurons (mainly pyramidal neurons, but also stellate cells in layer 4), and 15% inhibitory interneurons (Figure 3.1). We focus primarily on the excitatory pyramidal neurons, which perform the bulk of the information processing in the cortex. Unlike the local inhibitory interneurons, they engage in long-range connections between different cortical areas, and it is clear that learning takes place in the synapses between these excitatory neurons (evidence is more mixed for the inhibitory neurons). The inhibitory neurons can be understood as "cooling off" the excitatory heat generated by the pyramidal neurons, much like the cooling system (radiator and coolant) in a car engine. Without these inhibitory interneurons, the system would overheat with excitation and lock up in epileptic seizures (this is easily seen by blocking inhibitory GABA channels, for example). There are, however, areas outside of the cortex (e.g., the basal ganglia and cerebellum) where important information processing does take place via inhibitory neurons, and certainly some researchers will object to this stark division of labor even within cortex, but it is nevertheless a very useful simplification.
Figure \(3.1\): Neural constituents of the neocortex. (A) shows excitatory pyramidal neurons, which constitute roughly 85% of neurons, and convey the bulk of the information content via longer-range axonal projections (some of which can go all the way across the brain). (B) shows inhibitory interneurons, which have much more local patterns of connectivity, and represent the remaining 15% of neurons. Reproduced from Crick & Asanuma (1986).
Figure \(3.2\): A slice of the visual cortex of a cat, showing the six major cortical layers (I - VI), with sublayers of layer IV that are only present in visual cortex. The first layer (I) is primarily axons ("white matter"). Reproduced from Sejnowski and Churchland (1989).
Figure \(3.3\): The thickness of the different cortical layers varies depending on the location in cortex -- this is an important clue to the function of these layers (and the cortical areas). A) shows primary visual cortex (same as Figure 3.2) which emphasizes input layer 4. B) shows extrastriate cortex which processes visual information, and emphasizes superficial layers 2/3. C) shows primary motor cortex, which emphasizes deep layers 5/6. D) shows prefrontal cortex ("executive function") which has an even blend of all layers. Reproduced from Shepherd (1990).
Figure \(3.4\): Function of the cortical layers: layer 4 processes input information (e.g., from sensory inputs) and drives superficial layers 2/3, which provide a "hidden" internal re-processing of the inputs (extracting behaviorally-relevant categories), which then drive deep layers 5/6 to output a motor response. Green triangles indicate excitation, and red circles indicate inhibition via inhibitory interneurons. BG = basal ganglia which is important for driving motor outputs, and Subcortex includes a large number of other subcortical areas.
The neocortex has a characteristic 6-layer structure (Figure 3.2), which is present throughout all areas of cortex (Figure 3.3). However, the different cortical areas, which have different functions, have different thicknesses of each of the 6 layers, which provides an important clue to the function of these layers, as summarized in (Figure 3.4). The anatomical patterns of connectivity in the cortex are also an important source of information giving rise to the following functional picture:
- Input areas of the cortex (e.g., primary visual cortex) receive sensory input (typically via the thalamus), and these areas have a greatly enlarged layer 4, which is where the axons from the thalamus primarily terminate. The input layer contains a specialized type of excitatory neuron called the stellate cell, which has a dense bushy dendrite that is relatively localized, and seems particularly good at collecting the local axonal input to this layer.
- Hidden areas of the cortex are so-called because they don't directly receive sensory input, nor do they directly drive motor output -- they are "hidden" somewhere in between. The bulk of the cortex is "hidden" by this definition, and this makes sense if we think of these areas as creating increasingly sophisticated and abstract categories from the sensory inputs, and helping to select appropriate behavioral responses based on these high-level categories. This is what most of the cortex does, in one way or another. These areas have thicker superficial layers 2/3, which contain many pyramidal neurons that are well positioned for performing this critical categorization function.
- Output areas of cortex have neurons that synapse directly onto muscle control areas ("motor outputs"), and are capable of causing physical movement when directly stimulated electrically. These areas have much thicker deep layers 5/6, which send axonal projections back down into many different subcortical areas.
In summary, the layer-wise (laminar) structure of the cortex and the area-wise function of different cortical areas converge to paint a clear picture about what the cortex does: it takes in sensory inputs, processes them in many different important ways to extract behaviorally relevant categories, which can then drive appropriate motor responses. We will adopt this same basic structure for most of the models we explore.
Patterns of Connectivity
Figure \(3.5\): Typical patterns of connectivity between cortical areas (Feedforward and Feedback) and within cortical areas (Lateral). Information flows from the Hidden layers "up" in a feedforward direction into the Input layers of "higher" areas (from which it flows into the Hidden layer of that area), and flows back down in a feedback direction from Hidden and Output (output typically stronger as indicated) back to Hidden and Output layers in "lower" areas. All areas have lateral connections that similarly originate in the Hidden and Output layers, and connect to all three layers within another part of the same cortical area. Based on Figure 3 from Felleman and Van Essen (1991).
The dominant patterns of longer-range connectivity between cortical areas, and lateral connections within cortical areas are shown in Figure 3.5. Consistent with the Input-Hidden-Output laminar structure described above, the feedforward flow of information "up" the cortical hierarchy of areas (i.e., moving further away from sensory inputs) goes from Input to Hidden in one area, and then to Input to Hidden in the next area, and so on. This flow of information from sensory inputs deeper into the higher levels of the brain is what supports the formation of increasingly abstract hierarchies of categories that we discuss in greater detail in the next section.
Information flowing in the reverse direction (feedback) goes from Hidden & Output in one area to Hidden & Output in the previous area, and so on. We will see later in this chapter how this backward flow of information can support top-down cognitive control over behavior, direct attention, and help resolve ambiguities in the sensory inputs (which are ubiquitous). One might have expected this pattern to go Hidden to Output in one area, to Hidden to Output in the previous area, but this pattern is only part of the story. In addition, the Hidden layers can communicate directly to each other across areas. Furthermore, Output areas can also directly communicate with each other. We can simplify this pattern by assuming that the Output layers in many cortical areas serve more as extra copies of the Hidden layer patterns, which help make additional connections (especially to subcortical areas -- all cortical areas project to multiple subcortical areas). Thus, the essential computational functions are taking place directly in the Hidden to Hidden connections between areas (mediated by intervening Input layers for the feedforward direction), and Output layers provide an "external interface" to communicate these Hidden representations more broadly. The exception to this general idea would be in the motor output areas of cortex, where the Output layers may be doing something more independent (they are at least considerably larger in these areas).
Each cortical area also has extensive lateral connectivity among neurons within the same area, and this follows the same general pattern as the feedback projection, except that it also terminates in layer 4. These lateral connections serve a very similar functional role as the feedback projections as well -- essentially they represent "self feedback".
Figure \(3.6\): Connectivity matrix between cortical areas, showing that when a given area sends a feedforward projection to another area, it typically also receives a feedback projection from that same area. Thus, cortical connectivity is predominantly bidirectional. Reproduced from Sporns & Zwi (2004).
The other significant aspect of cortical connectivity that will become quite important for our models, is that the connectivity is largely bidirectional Figure 3.6. Thus, an area that sends a feedforward projection to another area also typically receives a reciprocal feedback projection from that same area. This bidirectional connectivity is important for enabling the network to converge into a coherent overall state of activity across layers, and is also important for driving error-driven learning as we'll see in the Learning Chapter.
Next, let's see how feedforward excitatory connections among areas can support intelligent behavior by developing categorical representations of inputs.