[This is a backup page for people with no initial access to Blackboard, and will not be updated. Use Blackboard instead if possible.]

85-419/719: Introduction to Parallel Distributed Processing

Spring 2014, Tue/Thu 10:30-11:50am, Baker 336B

Instructor: David Plaut
Baker 254N, x85145



The goal of the course is to introduce the basic principles of parallel distributed processing (also known as connectionist or neural network modeling) and to illustrate how these principles provide insight into human cognitive processing. In addition, the course will cover some issues in neural and cognitive development, cognitive impairments due to brain damage, and some basic computational issues. The course also attempts to introduce the general practice of studying cognition through computational modeling and analysis. There will be computer simulation exercises in addition to readings. Homework assignments will generally require you to report the results of simulations you have carried out, to analyze these results, and to think critically about some issues raised in the readings. There will also be a final project that will typically involve simulation modeling.

The course is divided into five sections. The first three cover basic topics in parallel distributed processing. For each of these, a homework assignment is handed out at the beginning of the section and is due at the end of the section. At the end of the third section, you will also be required to submit a one-page proposal outlining the final project you intend to carry out. This will be returned with feedback at the beginning of the fourth section (right after Spring Break), and you will be expected to get started on your project immediately thereafter. You should be on the lookout throughout the earlier sections of the course for topics or issues that you find particularly interesting and would like to pursue in more detail in a project. The fourth section focuses on applications from a range of perceptual, linguistic and cognitive domains, and will be followed by a take-home essay (1200-1500 words) based on class lectures and readings. The final section will be devoted to brief oral reports from each student on the topic of their project. A 12-15 page final paper (5000-7000 words) based on the project is due at the end of this last section. There is no final exam for the course.

In general, there are assigned readings for each lecture that are intended to prepare you to participate in the class discussion for that day. In addition, there may be optional background readings (marked with "opt:" in the Syllabus) that serve either as the basis for the lecture, to present an alternative point of view, or simply to make available to you relevant material that we won't have time to cover in class. Optional readings are also a good source of ideas for projects. There are no required readings on days when something is due, but you are still expected to attend class, hand in your homework, and draw on the material you have already learned in order to participate in the discussion.

Course Goals and Assessment

Below are the broad goals of the course and how each is assessed (listed in brackets).
The grading in the class will be divided up as follows:

Homework 1:   10%
Homework 2:   15%
Homework 3:   20%
Project proposal:     5%
Take-home essay:   15%
Oral presentation/class participation:    5%
Final project   30%

Assignments should be handed in as physical print-outs and are due at the beginning of class on the date listed in the Syllabus (usually a Tuesday). Late penalties will be assessed as follows: Homeworks handed in late but before 5pm of the next day (usually a Wednesday) will be penalized by 5% of the total possible points; those handed in before 5pm of the following weekday (usually a Thursday, but a Monday if the homework was due on a Thursday) will be penalized by 10%; those handed in later than that but before graded papers are returned will be penalized by 15%. Papers may not be handed in for credit after other students' graded homeworks are returned and feedback is posted to the course webpage, unless you get explicit permission from the instructor. Late homeworks may be submitted to the instructor by email (pdf file). The 5% for class participation will be based on contributions to class discussions throughout the semester, and on the quality of the oral project report.


There is no required text for the course. All assigned and optional readings are available as downloadable pdf files from links in the Syllabus below. Other course materials (e.g., handouts, assignments, etc.) will be made available via links at the top of this web page. The following texts contain some of the course readings and may be useful as general references:


We will be using a software package called "Lens" (for Light Efficient Network Simulator), developed by former CMU CS graduate student Doug Rohde. Lens runs under Windows, Mac OSX, and Linux. The main website for Lens is http://tedlab.mit.edu/~dr/Lens/ but don't install Lens from that site. You can download a file containing a precompiled version of Lens here:

If you have any problems getting Lens running, contact the instructor. After installing Lens, you should look at the online manual at http://tedlab.mit.edu/~dr/Lens/, particularly the instructions under "Running Lens" and the Tutorial Network under "Example Networks". The precompiled versions of Lens come with a offline (local) copy of the manual that can be accessed by pointing your web browser at Manual/index.html in the Lens directory.


This syllabus is subject to change throughout the course, so be sure to revisit this web page frequently.

Section 1: Processing and Constraint Satisfaction

Jan 14 (Tue): Overview and basic principles (slides) [HOMEWORK 1 POSTED] [Install Lens; see "Software" section in Overview]

Jan 16 (Thu): Lens tutorial

Jan 21 (Tue): Constraint satisfaction

Jan 23 (Thu): Schema theory (slides)

Jan 28 (Tue): Psychological implications (slides) [HOMEWORK 1 DUE]

Section 2: Simple Learning and Distributed Representations

Jan 30 (Thu): Correlation-based learning (Hebb rule) (slides) [HOMEWORK 2 POSTED]

Feb 4 (Tue): Error-correcting learning (Delta rule)

Feb 6 (Thu): Distributed representations (slides)

Feb 11 (Tue): Psychological implications (slides) [HOMEWORK 2 DUE]

Section 3: Learning Internal Representations

Feb 13 (Thu): Back-propagation (slides) [HOMEWORK 3 POSTED]

Feb 18 (Tue): Temporal learning and recurrent networks (slides)

Feb 20 (Thu): Generalization and overfitting (slides)

Feb 25 (Tue): Contrastive Hebbian learning (slides)

Feb 27 (Thu): Unsupervised learning (slides)

Mar 4 (Tue): Reinforcement learning and forward models (slides)

Mar 6 (Thu): Psychological implications [PROJECT PROPOSAL DUE] [HOMEWORK 3 DUE]

Mar 11 (Tue): NO CLASS (Spring Break)

Mar 13 (Thu): NO CLASS (Spring Break)

Section 4: Applications

Mar 18 (Tue): Cognitive development (slides)

Mar 20 (Thu): Semantics (slides)

Mar 25 (Tue): Language: Morphology (slides)

Mar 27 (Thu): Language: Word reading (slides)

Apr 1 (Tue): Language: Sentence processing (slides)

Apr 3 (Thu): Memory and the hippocampus (slides) [TAKE-HOME ESSAY POSTED]

Apr 9 (Tue): High-level vision and attention (slides) [TAKE-HOME ESSAY DUE]

Apr 10 (Thu): NO CLASS (Spring Carnival)

Apr 15 (Tue): NO CLASS (Passover)

Apr 17 (Thu): Cognitive control and executive function

Section 5: Project Progress Reports

Apr 22 (Tue):
Apr 24 (Thu):
Apr 29 (Tue):
May 1 (Thu):