A Conversation with David Plaut

Championing Connectionist Modeling





JF:
For the sake of argument, I am going to put forth this position of some hypothetical researcher on the biological side and say that connectionist modeling is, at best, irrelevant because:
  1. The neurally inspired connectionist units have abstracted away far too much of the interesting behavior of real neurons.
  2. The learning rules used by these networks have only limited biological support and in some cases are highly implausible.
  3. The results of these models often match the data but they provide very little top-down guidance as far as things that the biological researcher can directly look for.
How would you respond to that kind of position?
DP:
Well, I think that it's important to acknowledge that it is true, in fact, that the kinds of computational elements and learning algorithms that are typically employed in connectionist modeling - and certainly are employed in my research - really do not make direct contact with neurobiological and neurophysiological data, and they're not intended to. It's important to point out that it is a developing field in which people are pushing the algorithms closer and closer to the neurobiology. I think that's important not to lose sight of; that, even if one takes that as the goal and acknowledges that we are not there yet, it is an area in which a lot of progress is being made. And one hopes that that actually connects up with what we do know about processing in the brain.

But I think there is a more important point to make about current work and even current work that isn't directly trying to introduce more neurobiological verisimilitude. It's the idea that the kind of processing that the brain engages in can be, and needs to be, described at a number of different levels. And the level at which one best operates really depends on the kind of phenomena that you're interested in. For very basic processing dynamics in networks, in groups of neurons, I think it's critical to introduce the biophysical parameters that we know, or at least our current best guesses, our current best models of the biophysics of neurons and synapses and their interactions, because the data require it. The data on the actual behavior of single neurons and groups of neurons require that we have sophisticated segment models and that we look at the local non-linear interactions in dendritic trees and so on. So, at that level, the data that one is interested in require one to introduce complexities into how we think of neurons operating.

I think the same principle applies at other levels, which is to say, one introduces complexity into a model when the data require it. One is looking for abstractions of a system that can capture the central principles, the critical underlying variables, without introducing unnecessary complexity. If it turns out that understanding the effects of some kind of brain damage on cognitive processing requires low level detailed interactions in dendritic trees, then that does need to be introduced into the model. If the explanatory principles are at a more general level, in terms of the way larger groups of units represent information in patterns of activity and interact and settle in a stable representation, then it may be that those principles and those data can be captured at a more general level without introducing the complexity.

Now, of course, depending on the details of the data, as the data get more and more detailed, more needs to be incorporated into any model and that may involve introducing more neurophysiological properties into simulations. It may also involve introducing more sophistication at the cognitive level, to understand the range of strategies that subjects can introduce and so forth. So, I tend to think of models, and explanations, as starting at a particular level of description, that is the researcher's best guess as to where the action is for a particular set of phenomena, and then sort of growing from there, introducing more properties of what is known (not just towards the neurophysiology, but also towards more complicated behavior), as required by the data, as such complexity is needed to explain the kind of phenomena that you are interested in.

So, the notion that connectionist modeling, or modeling that doesn't introduce very specific neurophysiological properties into the system, would be irrelevant for brain functioning is essentially to claim that the level of description that connectionist models provide is lacking something fundamental that's required to explain a particular body of data. And my counter-argument is to put forth research that demonstrates that, in fact, this level of description can provide coherent explanations for phenomena, like deep dyslexia. That's not to preclude developing that approach further, both towards the neurophysiology and broadening it to other kinds of behavior. But I think that it's a mistake to believe that, once we know about more complex properties at one level, then all other levels of description should include them. It's not as if I am not aware that the units I use are implausible, it's that I am not trying to make them plausible. I am trying to keep them as simple as I can and yet still account for the behavioral data I see in patients. And, of course, that's a theoretical claim that could be wrong, or misguided, or in need of revision, and that's why I think it is important that the work continue to extend to a broader range of data, to try to include more about what's really known about the etiology of the actual impairment, the actual lesion, as well as more detailed data on the behavior of the patient.


JF:
From the huge expectations that were generated by Frank Rosenblatt, to Minsky and Papert's criticisms, to the resurgence in the '80s, it's been a kind of up and down road for connectionist models. Do you see it being now firmly established as an inter-disciplinary kind of approach?
DP:
Firmly established? I imagine there will be ups and downs in the future. In particular, I think it's misleading to think of the kind of work that's going on now as just an upswing of the work that was going on in the 60s and that Minsky and Papert criticized. In fact, the critical thing that happened between those two time periods is the discovery of algorithms that allow systems to learn their own internal representation to solve tasks in ways that don't directly correspond to the inputs and the outputs that the environment or an experimenter might provide. That is a fundamental shift in how one thinks about neural systems developing and learning.

I think what is also going to happen down the road - and I can tell you the particular areas that I think are most in need of developing to avoid this - I am sure what will happen is that there are limitations on what we currently do, like there were limitations in the sixties on what single-layer networks could do, that fundamentally limit the questions we can ask. And, to the extent that we run up against those limitations, the current style of modeling will gradually sort of ebb away. And my bet is that that will be replaced by more sophisticated modeling, hopefully more neurobiologically plausible modeling.

Back to the index. Back to the index.


Got a suggestion?
Send us a note!
Have and idea to share?
Post it to the CNBC Newsgroup!