Connectionist models represent information throughout a connected network
of units. In some connectionist networks each unit has a particular meaning
(e.g., a single unit represents the idea “dog”). In another class
of these models, termed “neural networks”, the units are each individually
meaningless and information is represented only in a distributed fashion,
as a function of the simultaneous activation of multiple units. In a connectionist
model, each unit receives “activation” from other units to which it is
connected in response to the stimulation of these units. The unit then
sends a transfer function of the activations coming into it to other units
to which it is connected. The transfer function may be a function of the
sum of the units inputs or may be more complex, involving temporal factors
such as the average input over time. Thus, connections between units may
be excitatory (if the sum is greater than zero, leading to relative activation
of the unit to which receives the connection) or inhibitory (if the sum
is less than zero, leading to relative inactivation of the unit which receives
the connection). The degree to which such units resemble biological neurons
varies across implementations. Most articles to be reviewed here use relatively
simple analogs of neurons as depicted in Figure 1. The lines on the left
of the node in Figure 1 represent connections from other units coming into
the node, much as dendrites are connected to a biological neuron. The lines
on the right of the node represent connections extending out from the node,
through which its activation will propagate, much as an axon caries the
activations of a biological neuron. Often, each unit is also given a weight
or "bias" loosely corresponding to a neuron's threshold for activation.
Units in a connectionist network have thus been likened to biological neurons
which receive information on dendrites and send out an aggregate of that
informtion over axons. Environmental or internal factors (e.g., visual
stimuli) which cause units to be “stimulated” are determined and modeled
explicitly by the connectionist modeler.
Figure 1: Common computational
analog of a neuron. Activation enters through channels on the neuron's
left which represent dendrites. The strength of activation over a given
pathway is governed by the activation coming from the neurons to which
the paths are connected and the strength of the paths themselves. The neuron's
body computes a function of the sum of these activations (e.g., a sigmoid
function) and propagates this function through the connections to other
units depicted on the neuron's right.
These neuron-like nodes may be assembled in arbitrary patterns. One such pattern is shown in Figure 2. The structure, or "architecture" of the network governs what information may pass into the network, the manner in which nodes may activate each other, and what information may be said to flow out of the network. Association with a single representation of a concept (e.g., entity or relationship) in a neural network may involve activation of a number of nodes in the connected cognitive network, such that the resultant pattern of activation represents a meaningful association though no individual node which is activated connotes this meaning. For example, activation of the highlighted nodes in Figure 2 may represent a particular concept such as a letter of the alphabet. Were some of these nodes not activated at a particular time, the entire concept may be said to be incompletely activated (e.g., if the network represents cognitive association, the network may be interpretted as not generating a clear reminding in response to a stimulus). Thus, the lack of central control in such networks, robustness of each node or information processing element to noise, and ability to model changes in state via changing activations make such networks particularly suited to modeling information processing tasks (Cohen and Servan-Schreiber, 1992).
A connectionist network’s response to a stimulus generally involves successive activation of a number of nodes in the network. This pattern of activations can represent an association in memory between two stimuli, a reaction to an external stimulus, or some other construct based on the network’s architecture and the designer’s conception of the network. Thus the processes operating within connectionist networks can be assumed to correspond to neuronal, cognitive, or behavioral events based on the intuitions of the network’s designer. For this reason, it is said that connectionist networks can model multiple psychological “granularities” or “levels of analysis”.
By strategically modifying the strengths of connections between units in response to a stimulus the network can be made to associate one set of activations with another (e.g., representing an association in memory between two stimuli). For example, the activation of one node may be associated with the activation of other nodes by increasing the strength of connections between these nodes. When this process is automated using a mathematical algorithm, the network is said to "learn" an association. As learned associations have been the foundation for many schools of psychology (e.g., behaviorism and structuralism) this process by which the network is made to associate one set of values (possibly representing environmental stimuli) with another (e.g., behaviors) is often implicated in models of psychopathology. Numerous procedures for making the network learn have been proposed (e.g., Rumelhart, Hinton & Williams (1986), Fallman & Lebiere (1991)), all of which involve minimizing the errors the network makes in associating an "input" with an "output", successively, over time.
Once a connectionist network model has been created, its behavior can be evaluated on a number of dimensions. The choice of what dimension is to be evaluated generally reflects the processes which the network is designed to simulate. For example, if the network is designed to simulate performance on some information processing task, associations made by the network could be compared to correct assocations; an analog of human error rate could thus be established. Similarly, it may take a number of associative steps for a network to settle on a learned association. The number of associative processing cycles or “epochs” the network needs to associate a stimulus with a particular response can be examined as an analog of reaction time. Alternately, if the network is designed to simulate association with a single concept in response to a stimulus, the amount of randomness in the network’s associations over time can be examined as an analog of flightiness of ideas.
To find out more about neural network and connectionist models, check
out: http://www.neuronet.ph.kcl.ac.uk/neuronet/places.html or http://www.statsoftinc.com/textbook/stneunet.html