86-375/15-387 (9 units) option of the course will require 10 homework exercises/tests (50%), 5 programming assignments (50%)
86-375 students can replace some or all of the programming assignments with reading reports and term papers.
86-675 (12 units) 86-375 or 15-387 requirements plus a term project (+30%).
Some Matlab or programming background are required for 15-387 and 86-675. There will be Matlab tutorials and TA help this semester.
Prerequisites: First year college calculus, some linear algebra, probability theory and programming experience are desirable. Discuss with instructor if you have any question.
Instructors | Office (Office hours) | Email (Phone) |
---|---|---|
Tai Sing Lee (Professor) | Mellon Inst. Rm 115 (Thursday 3:00-4:30), Wednesday 3:30-4:30 | tai@cnbc.cmu.edu (412-268-1060) |
Evaluation | % of Grade |
---|---|
Homework/Quizzes | 50 |
Programming Problem sets | 50 |
Term Project (685) | 30 |
Date | Lecture Topic | Relevant Readings | Assignments | |
---|---|---|---|---|
BIOLOGY OF PERCEPTION | ||||
MWF 8/26 week | Perception and Illusion | FS. Ch 1 | ||
MWF 9/2 | Philosophy, History and the Senses | FS. Ch 1, Marr, Ch 1 | ||
MWF 9/9 | Retinal computation and tunings | FS. Ch 6, Meister, Masland reviews | ||
MWF 9/16 | Linear System, Fourier Transform, Pyramid | FS ch 5, 6 , Abbott and Dayan Ch 1 and 2 | ||
MWF 9/23 | Striate and Extrastriate cortex | ch 9,10 | ||
M 9/30 | Simple cell and image representation | Abbott and Dayan chapter 1 and 2 | ||
F 10/4 | Guest Lecture: Art and Perception | handout |   | |
MODELS OF PERCEPTION | ||||
MWF 10/7 | Efficient and sparse codes | handouts (Olshausen) | ||
M 10/14 | Midterm Evaluation | |||
WF 10/16 | Mid-Semester Break | |||
MW 10/21 | Models Lightness perception (retinex,intrinsic) | FS ch 16, Land's papers | ||
F 10/25 | Guest Lecture: Electric Fish perception. | Papers | ||
M 11/4 | Shape from shading | Papers | W 11/6 | Binocular Stereo | FS ch 18,19 | < tr> | M 11/11 | Perceptual organization | FS ch 8, 13 | < tr> | WF 11/13 | Texture Perception | FS ch 2 | < tr> | MWF 11/18 | Motion recognition | FS ch 14,15 | < tr> | MWF 11/25 | Object recognition | FS ch 8 | < tr> | MWF 12/2 | Scene, Context and Attention | Papers |
F 12/13 | FINAL EXAM (1:00-4:00) |
Some additional attention will be allocated to the following topics:
Topic 1: Scene statistics, sensory and cortical representation
To undersatnd perception, we must understand the natural environments which shape our brain and our perceptual computational machinery. Central to to understanding the neural basis of perceptual inference from a Bayesian perspective is understanding how the statistical regularities in natural scenes are encoded in cortical representation to serve as priors in the inference process. Natural images however are enormously complex and maybe best expressed in hierarchical forms. Thus, a major challenge in computational vision is to understand the basic vocabulary of images, and the computational rules with which elementary components can be composed to form successive compositional structures to encode the hierarchical priors of natural scenes. We will explore statistical models of images, as well as compositional models such as DBN (Deep belief net) and RCM (Recursive compositional models) for learning the hierarchical language of vision. We will explore how these hierarchical scene priors are encoded in neural tunings and neural connectivities to faciliate perceptual inference.
Topic 2: Probabilistic models and algorithms of perception
While perception has been popularly formulated in terms of Bayesian inference in the theoretical level, little is known about the computational algorithms and implementation of perceptual inference. We will explore mechanistic and normative models for motion, binocular stereo, texture, surface and contour perception, perceptual organization and hierarchical models for object recognition, drawing knowledge from works in computer vision and computational neural models. We will study a number of algorithms that have been effective in computer vision for performing learning and inference, including gradient descent, particle and Kalman filtering, MCMC sampling and mean field approximation, and explore the links between observed neural dynamics and these inference algorithms. We will explore various theoretical frameworks on how perceptual representations are encoded and represented in neuronal ensembles, including the issue of population codes, synchrony and binding.
Topic 3: Neural decoding, mental representation and perceptual synthesis
With an understanding of cortical representation and neural mechanisms for perceptual inference, we can begin to explore how neural decoding and neural simulation technology can be coupled with large-scale multi-electrode array to decode mental images in our brain as well as to generate perceptual representation in the brain by electrical stimulation. There are over 40 million blind individuals in the world. A variety of invasive and noninvasive procedures have emerged over the years to use electrical stimulation to "restore" or create vision, ranging from retinal implant to electrical stimulation in LGN and stimulation of the visual cortex. We will investigate how V1 and the extrastriate cortex can represent mental images and precepts individually and together, both in terms of theories, models and neural evidence. We will study literature of artificial vision in human and animal models and explore paradigms for the development of visual prosthesis by integrating computer vision, electrical recording and stimulation technology.
Topic 4: Perception, computation and art
Visual perception and artistic expression are deeply connected at many levels. In fact, visual perception in the brain might involves both analysis and synthesis. That is, our perception is not simply analyzing what is out there, but an active synthesis of an internal mental representation of what is out there, sometimes leading to illusion and hallucination. We will explore this synthesis process and how it might be tied to aesthetics and art making. The integration of visual art and the experimental study of vision has its roots in formal analysis of paintings. Advances in our understanding of how our brain or perception works have lead to resurgence of interests in linking art with vision science. Here, we will explore some of the new links between neuroscience, computational vision and the art, with a view to enrich our understanding and making of arts -- how artistic expression is rooted in perceptual computation and how scientific understanding of vision have transformed arts over the centuries.