86-375/675 (15-387) Computational Perception

Carnegie Mellon University

Spring 2018

Course Description

The perceptual capabilities of even the simplest biological organisms are far beyond what we can achieve with machines. Whether you look at sensitivity, robustness, or sheer perceptual power, perception in biology just works, and works in complex, ever changing environments, and can pick up the most subtle sensory patterns. Is it the neural hardware? Does biology solve fundamentally different problems? What can we learn from biological systems and human perception?

In this course, we will first study the biological and psychological data of biological perceptual systems, particularly the visual system, in depth, and then apply computational thinking to investigate the principles and mechanisms underlying natural perception. The course will focus primarily on visual perception. You will learn how to reason scientifically and computationally about problems and issues in perception, how to extract the essential computational properties of those abstract ideas, and finally how to convert these into explicit mathematical models and computational algorithms. The course is targeted to neuroscience and psychology students who are interested in learning computational thinking, and computer science and engineering students who are interested in learning more about the neural basis of perception. Prerequisites: First year college calculus, some linear algebra, probability theory and programming experience are desirable.

Course Information

GHC 5th Floor Citadel Teaching Commons (Friday 4:30-5:30)
Instructors Office (Office hours) Email (Phone)
Tai Sing Lee (Professor) Mellon Inst. Rm 115 (Friday 1:30-2:30) tai@cnbc.cmu.edu (412-268-1060)
Ge (Summer) Huang Mellon Inst. Rm 115 (Tuesdsy 4-5 p.m) hgesummer@gmail.com
Shefali Umrania sumrania@andrew.cmu.edu

Recommended Textbook

Classroom Etiquette

Grading Scheme

Evaluation% of Grade
Assignments 60
Midterm 10
Final Exam 10
class Participation/Quiz 10
Term paper and Presentation 10
Term project 10 for replacing assignments or midterm only
675 Term Project Required
  • Grading scheme: A: > 88, B: > 75. C: > 65.


    Term Project


    Late Policy


    Date Lecture Topic Relevant Readings Assignments
    W 1/17 1. Philosophy of computational approach ch. 1, Marr  
    M 1/22 2. Vision and retina ch. 6 (retina), Meister  
    W 1/24 3. Linear System ch 3 (receptive fields), Simoncelli Homework 1
    M 1/29 4. Representation ch 5 (edges features)
    W 1/31 5. Frequency analysis Ch 4 (frequency analysis)  
    M 2/5 6. Principle components ch 9 (V1, neurons), Shlens  
    W 2/7 7. Source separation Fodiak, Hyvarinen, Olshausen, Sejnowksi Homework 2
    M 2/12 8. Bayesian inference ch 13 (inference)  
    W 2/14 9. What and where Sompolinsky, retina, V1, object detection  
    M 2/19 10. Lightness and color ch 16, Land, Horn, Morel  
    W 2/21 11. Intrinsic images and Retinex ch 17, Adelson, Weiss, Freeman Homework 3;
    M 2/26 12. Features and segmentaiton ch 7 (figure/ground)  
    M 2/28 Midterm    
    W 3/05 13. Texture and surfaces ch 2, Julesz, Simoncelli  
    W 3/07 15. Perceptual Organization ch 7 Homework 4
    M 3/12 Spring Break    
    W 3/14 Spring Break    
    M 3/19 16. Visual Hierarchy ch 10 (brain maps), Van Essen  
    W 3/21 17. Depth and 3D ch 18,19  
    M 3/26 18. Shape from Shading Horn, Zucker  
    W 3/28 19. Style and Contents Bethge Homework 5
    M 4/02 20. Motion Perception ch 14,15, Weiss  
    W 4/04 22. Cue Integration ch 20, Bank, Ma  
    M 4/09 23. Belief and Association Hinton, Lee, Gilbert, Geisler  
    W 4/11 24. Context and scenes Torrelba, Oliva Homework 6
    M 4/16 25. Object recognition ch 8 (objects), Sinha, LeCun, Hinton  
    W 4/18 26. Composition and Grammar ch 11 (complexity) Yuille, Mumford  
    Th 4/19 Spring Carnival 20/21    
    M 4/23 27. Attention and routing ch 22, Hinton, Arathon, Olshausen  
    W 4/25 28. Review   Homework 6 due
    M 4/30 29. Final Quiz   Term paper due
    W 5/02 30. Project presentations Term paper due
    M 5/14 FCE deadline  
    M 5/X Final Exam (hold)  

    Supplementary Readings (Relevant to Understanding Lectures)

    Part 1: Vision, Perceptual Systems and Philosophy

    Part 2: Neural Codes, Features and Representational Learning

    Part 3: Lightness and color perception, Retinex

    Part 4: Mid-level vision: Texture, depth and motion Perception

    Part 5: Visual Hierarchy, Object recognition, Abstract Representations

    Part 6: Generative models, Art, Abstract Representations

    Part 7: Belief, memories and Association

    Part 8: Composition and Grammar

    Part 9: Attention, Eye Movement and Routing

    Questions or comments: contact Tai Sing Lee
    Last modified: Jan 2018, Tai Sing Lee