86-375/675 (15-387) Computational Perception

Carnegie Mellon University

Fall 2016

Course Description

The perceptual capabilities of even the simplest biological organisms are far beyond what we can achieve with machines. Whether you look at sensitivity, robustness, or sheer perceptual power, perception in biology just works, and works in complex, ever changing environments, and can pick up the most subtle sensory patterns. Is it the neural hardware? Does biology solve fundamentally different problems? What can we learn from biological systems and human perception?

In this course, we will first study the biological and psychological data of biological perceptual systems in depth, and then apply computational thinking to investigate the principles and mechanisms underlying natural perception. The course will focus primarily on visual perception this year. You will learn how to reason scientifically and computationally about problems and issues in perception, how to extract the essential computational properties of those abstract ideas, and finally how to convert these into explicit mathematical models and computational algorithms. The course is targeted to neuroscience and psychology students who are interested in learning computational thinking, and computer science and engineering students who are interested in learning more about the neural basis of perception. Prerequisites: First year college calculus, some linear algebra, probability theory and programming experience are desirable.

Course Information

Instructors Office (Office hours) Email (Phone)
Tai Sing Lee (Professor) Mellon Inst. Rm 115 tai@cnbc.cmu.edu (412-268-1060)
Feng Wang (TA) Mellon Inst. Rm 115 euphoria.wang@gmail.com
Jessica Lee (TA) main campus jklee@andrew.cmu.edu
Yi-Ching Lee (TA) main campus yichingl@andrew.cmu.edu

Recommended Textbook

Classroom Etiquette

Grading Scheme

Evaluation% of Grade
Assignments 60
Midterm 10
Final Exam 30
Term project 20 (substitition for homework)
675 Term Project Required
  • Grading scheme: A: > 88, B: > 75. C: > 65.


    Term Project


    Late Policy


    Date Lecture Topic Relevant Readings Assignments
    M 8/29 1. Introduction and philosophy ch. 1, Marr  
    W 8/31 2. Overview: visual system ch. 9, 10, Van Essen  
    W 9/07 3. Retinal processing ch 6 Meister Homework 1
    M 9/12 4. Linear Transform ch 5 Abbot
    W 9/14 5. Representations handout  
    M 9/19 6. Primary visual cortex paper handout  
    W 9/21 7. Sparse coding paper handout, Olshausen Homework 2
    M 9/26 8. Source separation Sejnowski, Hyvarinen  
    W 9/28 9. Inferring What and Where Sompolinsky Homework 3
    M 10/3 10. Edges and contours ch 5, Mumford  
    W 10/5 11. Statistics and Bayesian inference ch 13, Geisler  
    M 10/10 12. Lightness and Color ch 16, Adelson  
    W 10/12 13. Retinex ch 17, Land, Morel &Homework 4
    M 10/17 14. Review/Discussion  
    W 10/19 15. Midterm    
    M 10/24 16. Figure-ground segregation Ch 7 Midterm grade due
    W 10/26 17. Depth and Stereo ch 18,19 Homework 5
    M 10/31 18. Texture and surfaces ch 2  
    W 11/2 19. Shape from Shading Zucker, Potetz  
    M 11/7 20. Structure from motion ch 14,15  
    M 11/14 22. Cue combination and selection ch 20  
    W 11/16 23. Object perception ch 8 Homework 6
    M 11/21 24. Context and scenes Torrelba, Oliva  
    W 11/23 Thanksgiving    
    M 11/28 25. Hierarchical organization ch 10  
    W 11/30 26. Attention and Saliency    
    M 12/5 27. Predictive and constructive vision    
    W 12/7 28. Review   Term paper due
    X 12/X Final Examination    

    Questions or comments: contact Tai Sing Lee
    Last modified: March 1, 2016, Tai Sing Lee