15-386/686 Neural Computation

Carnegie Mellon University

Spring 2019

Course Description

Neural Computation is an area of interdisciplinary study that seeks to understand how the brain learns and computes to achieve intelligence. It seeks to understand the computational principles and mechanisms of intelligent behaviors and mental abilities -- such as perception, language, motor control, decision making and learning -- by building artificial systems and computational models with the same capabilities. This course explores computational issues at multiple levels, from individual neurons to circuits and systems, with a view to bridging brain science and machine learning. It will cover basic models of neurons and circuits, computational models of learning, memories and inference, and quantitative approaches to neural system analysis in real and artifical systems. Concrete examples will be drawn from the visual system and the motor system, with emphasis on relating current deep learning research and the brain research, from hierarchical computation, attention, recurrent neural networks, to reinforcement learning. Students will learn to perform quantitative analysis as well as computational experiments using Matlab and deep learning platforms. No prior background in biology or machine learning is assumed. Prerequisites: 15-100, 21-120 or permission of instructor. 21-241 preferred but not required.

Course Information

Instructors Office (Office hours) Email (Phone)
Tai Sing Lee (Professor) MI 115. Friday 9:00-10:00 tai@cnbc.cmu.edu (412-268-1060)
Ziniu Wu (TA) GHC Tuesday 4:30-5:30 ziniuw@andrew.cmu.edu
Stella Yuan (TA) GHC Monday 5:30-6:30 yixiny@andrew.cmu.edu
Xuyang Fang (CA) no office hour xuyangf@andrew.cmu.edu

Recommended Supplementary Textbook

Classroom Etiquette

386 Grading Scheme

Evaluation% of Grade
5 Assignments 70
Midterm 10
Final Exam 20
Optional term project (Replacement of 1 HM or Midterm) 10-12
  • Grading scheme: A: > 88 B: > 75. C: > 65.
  • 686 Grading Scheme

    Evaluation% of Grade
    Option 1. 5 Assignments 35
    Option 2. Weekly Journal Club (reading / 3 presentations) 35
    Option 3. Term project 35
    Midterm 10
    Final Exam and/or Quiz 2 20
  • 686 students can choose two of the above three options, i.e. they have the three options to earn the 70 points: (1) term project + 5 problem sets; (2) term project + journal club (can miss at most 3 weeks); (3) journal club + 5 problem sets.
  • Grading scheme: A: > 88 B: > 75. C: > 65.
  • Assignments

    Term Project

    Examinations

    Late Policy

    Syllabus

    Date Lecture Topic Relevant Readings Assignments
      LEARNING AND REPRSENTATION    
    W 1/14 1. Introduction NIH Brain Facts (chapter 1)  
    M 1/16 2. Neurons and Membranes F. Rosenblatt - Perceptron.  
    M 1/21 Martin Luther King Day (no class)    
    W 1/23 3. Spikes and Cables Trappenberg Ch 1.1-2.2 (C) HW 1 out
    M 1/28 4. Synapse and Neural Net Trappenberg Ch 3.1  
    W 1/30 5. Polar Vortex (no class) McCulloch and Pitts (1943)  
    M 2/4 6. Synaptic plasticity Trappenberg Ch 4 Abbott and Nelson (2000)  
    W 2/6 7. Hebbian Learning Trappenberg Ch 4, HPK Ch 8 Oja (1982) HW 2 out, HW 1 in
    M 2/11 8. Features and Convolutions HPK Ch 8  
    W 2/13 9. Computaitonal Maps HPK Ch 9  
    M 2/18 10. Source Separation    
    W 2/20 11. Hopfield Net   HW 2 in, HW 3 out.
    M 2/25 12. Composition    
    W 2/27 13. Multi-layer perceptron    
    M 3/4 14. Midterm    
    W 3/6 15. Deep convolutional neural network   HW4 out.
    M 3/11 Midterm grade, Spring break    
    W 3/13 Spring break    
      ASSOCIATION and INTERACTION    
    M 3/18 16. Local Recurrent Circuits    
    W 3/20 17. Bayesian Inference and Decoding   HW 3 in
    M 3/25 18. Memories and imagination    
    W 3/27 19. Semantic category development    
    M 4/1 20. Gated Machines   HW 4 in, HW 5 out
    W 4/2 21. Decision Processes    
    M 4/8 22. Reinforcement Learning    
    W 4/10 23. Predictive Learning    
    F 4/13 Spring Carnival    
    M 4/15 24. Attention and Feedback Network    
    W 4/17 25. Cell, Circuit and System ID HW5 in
    M 4/22 26. Reading Neural Codes    
    W 4/24 27. Brain Machine Interface    
    M 4/29 28. Neural Manifold Learning    
    W 5/1 29. Review and Summary   Term paper deadline
    F 5/3 30. 686 project presentation   presentation deadline
    X 5/X Final Exam    
    R 5/16 Final Grade due 4 p.m. for Graduates    
    T 5/21 Final Grade due 4 p.m.    

    Journal Club and Relevant Reading List

  • Every Friday 1:30-3:00, except 3/15 Spring Break, and 4/12 Spring Carnival. Total 13 weeks. Minimal attendance: 10 times (10 points). 20 points for 3-4 Presentations.

    Week 1 and 2 (1/18, 1/25): The Computer and the Brain

    Week 3 (2/1): Relating Structures, Circuits and Functions -- Retina Example

    Week 4 (2/8): Synaptic Plasticity and Biological Learning Rules

    Week 5 (2/15): More on Biological Learning Rules (BCM and PCA)


    Questions or comments: contact Tai Sing Lee
    Last modified: spring 2019, Tai Sing Lee