document version 2.7

last updated 5 September, 1997

The CORTEX (version 5)

User's Manual


Steve Macknik, Andrew Mitz, & Robert Desimone

Lab Neuropsychology, NIMH


1. Introduction

1.1. What is Cortex?

1.1.1. A real-time program for experiments

Cortex: A Program for COmputerized Real Time EXperiments. Cortex is a program for running neurophysiological and behavioral experiments on a PC. Although it is still undergoing revisions, it is a working program that we have been using to collect data for more than four years. The integrity of the data, timing accuracy, etc. have all been validated. Cortex runs behavioral paradigms designed by the user, displays visual stimuli to the subject, collects neurophysiological and behavioral data, monitors and stores eye position data, and shows the results to the user during the experiment. Because Cortex has the capability to display and manipulate any image that has been previously created and saved on disk by the user, the variety of visual stimuli it can present is practically unlimited. The program is oriented toward experiments in vision, but it can be used in a variety of other contexts.

Data acquisition capabilities include (dependent on the available I/O hardware) up to 16 spike inputs (TTL +5v digital inputs), specialized functions for monitoring hand-bar and lever movements (TTL +5v digital inputs), eye-movement monitoring functions (A/D inputs), and keyboard inputs. Reward triggering for alert-animal experiments is accomplished through TTL +5v digital output. The user retains access to the full-array of possible uses for the I/O equipment on-board (such as D/A output, or specialized digital I/O) with easy-to-use C functions. Multiple I/O devices may be accessed by CORTEX, limited only by the number of slots available in the computer.

Graphical capabilities include a device-generalized graphics kernel that allows the user to run the same experiments with either a Number Nine Co. Sgt Pepper board, or the new #9GXi-TC board. Many of the low-level technical aspects of graphical programming are handled directly by CORTEX, thus the graphical programming environment is relatively simple to use.

CORTEX is designed to simplify combined behavioral and neurophysiological experimentation by directly handling trial randomization and simple staircasing. The user retains complete control of the trial staircasing, if desired, and can generate any staircasing paradigm needed. The user can even change the staircasing design on-line, in any way, based on behavioral responses.

Associated with Cortex are a number of utility programs. Procortex is the basic numerical analysis program for the Cortex data file, and it can format the output in a varieties of ways, including Lotus 123 ASCII format, text format, and formats compatible with various statistical programs. It can either be run by itself, through command line options, or it can be run under the control of STATS100. STATS100 is a master control program that can automate the running of Procortex and can create and run batch files for sending the output of Procortex to compatible statistical programs such as SYSTAT or BMDP. STATS100 can also be used to direct the statistical programs to perform any sort of analysis on the data, but to use STATS100 you must already know the ins and outs of the statistical program itself. Cortex files can also be analyzed using PCOFF, which was developed in the Laboratory of Neurophysiology (NIMH). PCOFF is a comprehensive event code sequence search program that supports graphical display of rasters, histograms, reciprocal interval plots, using a number of different schemes to align data on specific events. The program also does analog signal processing (e.g. integration, averaging, etc.) as well as non-parametric and descriptive statistics. RAST62 and GRAST are programs for drawing histograms, rasters, eye position records, etc. off line, using the data collected by Cortex. The output of GRAST and RAST62 can be either low-resolution screen dumps, 300 dpi high resolution plots for laser printers, or CGM-compatible files that can be imported into Lotus Freelance for customizing the figures. Within Freelance or similar software, you can add receptive field diagrams, arrows, charts, etc. to your histograms and rasters. Other associated programs include various utility programs for drawing sinewave gratings, etc.

1.2. Hardware Requirements

1) Basic computer: AT class machine (80286 or better) with 640K memory and mouse. A VGA video board (or campatible video card...such as a Hercules monochrome board) is needed in addition to a compatible monitor. Estimated Cost = $1000-3000.

2) For visual stimulation studies: Pepper SGT-plus graphics card with at least 1 MB of memory, or a #9GXi-TC graphics card with 4megs VRAM and at least 1 meg DRAM, for the subject's visual display. The Pepper SGT will work with any VGA or multiscan monitor and the #9GXi-TC can support multiple resolutions and scanrates. Estimated cost = $1400 for either the Pepper SGT (1 Meg), or the #9GXi-TC (4 megs VRAM/1 meg DRAM), and $300-2000 for a compatible monitor. Many labs use two subject monitors (one for the animal and one for the user....they may be in separate rooms) and use a video splitter such as the VOPEX-2V-H from NTI).

3) Metrabyte Dash-16 for analog and digital input/output ($1000), or Metrabyte PIO24 for digital input/output only ($200). If you don't need to monitor eye movements or deal with analog data, the PIO24 is an economical behavioral input device. A highly recommended Metrabyte Dash-16 clone is the Compuboard CIO-AD16/50K. It includes a DASH-16 equivalent as well as a PIO24 digital i/o on the same board, and it costs somewhat less than the Dash-16. There is also a Compuboard clone of the PIO24 board, for about $60.00, not including cable or interface box.

1.3. History (you may skip this part)

The remote ancestor of Cortex can be traced to an assembly language program for collecting spike data and controlling an optical bench, written for the PDP-12 by David Bender in Charlie Gross' lab at Princeton during the early 1970's. The PDP-12 was a state-of-the art laboratory computer with 8K of core memory (and a 12 bit word), a bit-mapped graphics display, a real-time programmable clock, and analog and digital interfaces built in. After Dave left Princeton for Buffalo, I assembled the necessary components for a PDP-11 system (which, shockingly!, had no lab interfaces built in) and sketched the requirements for a new spike collection program that could make use of the luxurious 64K of memory (and 2.5 MB hard disk!) that was then available. The core of this program was written in 1980 in Whitesmith's C by a moonlighting mathematician, Phil Thrift, and a high school student who had been programming in C since age 12. When I left Princeton for NIMH, the program branched in its development. One branch continued to be developed at Princeton by Tom Albright, and the other branch developed at NIMH. Jeff Moran at NIMH significantly rewrote the program, then called "Behave", and added modules for controlling the monkey's behavior and controlling an external Jupiter Graphics system for presenting stimuli.

As to be expected, a memory size that once seemed luxurious eventually became ridiculously too small. After the introduction of the IBM-AT, Stan Schein and I decided to switch from the PDP-11, in part because of the great difference in price and in part because the C compilers for the AT allowed transparent access to 640K of memory, whereas it was difficult to use more than 64K on the PDP-11. A Pepper SGT graphics card from Number Nine Computer was eventually chosen to replace the Jupiter graphics system. Stan, who moved from NIH to Harvard Medical School, hired Thuan Tran, then a computer science student at MIT to port the program after I had naively (and innocently!) assured Stan that the porting would take at most six weeks, once we understood all of the new hardware. Because the old program was inflexible and filled with kludges to get around memory limitations, Thuan did not actually port it. Although he retained a few design aspects of the original PDP-11 program, he completely rewrote and greatly expanded the program over the course of the next couple of years after graduating from MIT, with direction on the design by Stan and myself and support from Stan's grant (R01-EYO6096).

In 1989, Thuan took a full time job with a computer company but continued to work part-time on the state-system interface to Cortex, on contract to NIMH. At about the same time, I hired Tom White at NIMH, and he significantly rewrote most of Cortex again, in order to make it more user friendly, to greatly upgrade its stimulus display capabilities, and to improve the maintainability of its source code. He also took over the development of GRAST, a histogram display program, and Procortex and STATS100, numerical analysis programs, greatly expanding and rewriting them over the course of the past year. Tom integrated STATS100 with SYSTAT, a commercial statistical program, and wrote a data analysis compiler that users can use to write their own analysis routines, without doing any actual programming. Tom is now at Cornell Medical School in an MD/PhD program, but continues to work on Cortex occasionally, out of the goodness (truly) of his heart. I myself work on Cortex code, in my spare milliseconds, and am reasonably familiar with the workings of most of the modules. Rosalyn Merrill (NIMH), Amir Geva, and Jeff Moran also worked on small pieces of code for either Cortex or GRAST at one time during their development. Most recently, Marcello Gattass, the brother of Ricardo Gattass, has taken over the development of GRAST for the time being. Marcello promises a MS Windows version of GRAST sometime in the future. Steve Wise at NIMH has contributed towards some of the considerable development expense of Cortex. Finally, Cortex has benefited from the suggestions (and, unfortunately, bug-finding) of many of its users, particularly Mark Wessinger, Lin Li, Sidney Lehky, John Duncan, Earl Miller, Jennifer Hart, Driss Boussaoud, Josef Rauschecker, Ricardo Gattass, Leonardo Chelazzi, Andy Mitz, Richard Jeo, Carl Olson, Rebecca Hoag, and Peter DeWeerd.


2. Setting Up the Hardware

Typical Metrabyte Dash-16 settings

1) Address: 0x300 or 0x310

2) address switch settings (from left to right): OFF OFF ON ON ON OFF

3) set for bipolar, 16 analog inputs

4) gain = whatever is convenient

5) analog eye inputs: analog input channels 3 and 4

6) bar contact input: digital input 2

7) solenoid reward output: digital output 1 (puts out brief (< 1 ms) TTL pulse to trigger a solenoid control circuit)

8) spike inputs (2 or more separate spike channels): digital inputs 0 and 1 (for the first two inputs), which are fed by flip flops (provided by the user's electronics shop...see below). Typically one uses the "user 1 and 2" lugs on the Metrabyte interface connector board to access the flip flops.

9) output to clear flip flops: digital output 0

Typical Compuboard CIO A/D-16 settings:

The Compuboard CIO AD-16 can be considered to be a combination of a DASH-16 A/D board and a PIO24 digital I/O board, with the address space of the PIO24 following that of the DASH-16. Because of the PIO24 on board, it is usually necessary to use the Compuboard with a base address of 0x300 for the DASH-16 component rather than 0x310. If the DASH-16 component has an address of 0x300, the PIO24 component can be set with an address of 0x310, which is a safe address. Otherwise, if you set the base address for the DASH-16 component at 0x310, the address space of the PIO24 is automatically set at 0x320, which apparently can conflict with other devices on the AT bus.

Pepper SGT settings:

1) 640x480 resolution (default)

2) set for no emulation

3) put the following line in your config.sys file

device=nnios.sys /m=A000:64;

4) put the nnios.sys file in your root directory. When your computer boots up, it should tell you that it has loaded the Pepper SGT device driver successfully.

#9GXi-TC settings:

1) install the board and run the installation program (it comes with the board)

2) replace the line "c:\tiga2\tigacd.exe" in your autoexec.bat file with "c:\cortex5\util_bin\ticortcd.exe". Where "c:" is the drive where these directories reside.

3) when your computer boots up, it should tell you that it has loaded the TIGA device driver successfully.

Spike Flip Flop Circuitry:

The Dash-16 does not latch digital inputs (for purposes of spike acquisition, neither does the PIO24). This seems to be typical for digital inputs boards for the PC, except possibly for a handshaking line. So if you want to collect neuronal spikes, you must add flip flops between the TTL pulse of your window discriminator (a +5v digital pulse) and the digital input ports of the board. Otherwise, if your TTL pulse is less than 1 msec long, it may be "gone" by the time the Cortex program gets around to examining the port; if longer than 1 msec, Cortex will read more than one spike for each pulse. A circuit diagram for the flipflops can be obtained by contacting Bob Desimone at The flipflops are cleared by digital output #0 (courtesy of the software and the flipflop circuit). The spike inputs go to user inputs 0 and 1. Behavioral inputs go to digital inputs 2 and 3 (bar up/down = 2, bar left = 2, bar right =3), which naturally are not latched.


Although Cortex does not require any interface beyond the screw-terminal box or panel sold by Metrabyte or Compuboard (plus a couple of flip flops for the spikes), Andy Mitz developed a fabulous general-purpose interface called Thalamus. Thalamus is an interface and interconnect box that goes between the computer I/O boards and the outside world. It contains BNC connectors for up to 8 A/D channels and up to 8 spike inputs. It has input latches for all 8 spike inputs. Four of the 8 A/D inputs can have built-in anti-aliasing filters. Thalamus also has "D" type connectors for digital input and output. There are 8 "touch pad" circuits in the Thalamus unit to aid with touch switches or to condition other switches. There is also a one-shot and driver for delivering rewards. Thalamus was developed by the Laboratory of Neurophysiology, NIMH. Drawings for the Thalamus are available through the Research Services Branch of the NIMH.

3. Using Cortex

Cortex is trial-oriented. A fundamental feature of the program is that it is organized around "trials". That is, data are collected in discrete trials that last anywhere from a few milliseconds to a few seconds, followed by an "intertrial" interval in which the program does bookkeeping such as storing data from the previous trial and setting up the conditions for the next trial. The duration of the intertrial interval is set by the user, but the minimum is about half a second or so (or longer if you need to read in large images from disk files). Before Cortex begins running trials, the user must specify the different varieties of experimental conditions that are to be run in the session.

3.1. Setting up the experimental conditions.

To set up the experimental conditions, it is necessary to write at least three ascii text files "off-line", which you then import into Cortex. These three files can be written using your favorite text editor. I personally use the Brief Editor, but you can use the Norton Editor, or any other editor you like. Word processing programs cannot be used unless they can output a standard ascii text file, which some word processing programs have as an option. It is best to work from an already existing set of files (such as the examples and demos provided with Cortex), which you can edit and modify, rather than to try to write the files from scratch the first time.

The three files you must write are items, conditions, and timing (state system) files. For a given experimental session, you can import only a single item and condition file into Cortex, but multiple timing files.

For the items and conditions files, you will note that at the top of each file, there are headings such as ITEM, SIZE, X_POSITION, etc. These headings are special keywords that you should not alter. The first and last character of each keyword define the beginning and end of the data field columns located under the keywords. That is, you enter your specifications in the columns situated under each heading word, taking care not to extend any data beyond the first or last character of the heading word. Because "tabs" can mean any number of spaces, depending on your text editor, Cortex cannot interpret tabs in the files. Thus, DO NOT USE TABS. USE ONLY SPACES between values in the items and conditions files. If you use tabs, previous versions of Cortex would read in your file in a jumbled manner and become very confused. The most recent version at least warns you that tabs are present in your file. If you get a warning about tabs, re-edit your file to remove them and then reimport the file. The state system file is free-format, so tabs are fine. The format of the files is given next.

3.2. Item files

For experiments in vision, you probably have in mind a number of visual stimuli that you want to present to a particular neuron, person, animal, or whatever. Before specifying how and when you want the stimuli to be presented, it is first necessary to describe to Cortex the stimuli you want to present. These stimuli are referred to as ITEMS in Cortex.

You must describe each item you want to use in a given experimental session in the ITEMS ascii file, which you import into Cortex at the start of the session. For example, you might want to compare the responses of a neuron to 20 bars of light that vary in color and orientation. In the items ascii file, you would describe the 20 items in the format required by Cortex, including the type of visual stimulus (in this case, bars of light), their height, width, orientation, color, and position relative to some reference point on the display screen (to be described later).

An example items file is shown below:


-4 1 1 0.00 0.00 0 1.00 1.00 0.00 0 0 0 x

-3 1 1 0.00 0.00 0 0.30 0.30 0.00 255 255 255 x

-2 1 1 0.00 0.00 0 1.00 1.00 0.00 255 255 255 x

-1 1 1 0.00 0.00 0 1.00 1.00 0.00 255 255 255 x

0 1 0 0.00 0.00 0 3.00 1.00 0.00 50 50 50 x

1 1 0 0.00 0.00 1 3.00 1.00 45.00 100 100 100 x

2 8 0.00 0.00 2 1.8 1.8 230 0 230 x face.vb

3 1 1 0.00 0.00 1 3.00 1.00 45.00 250 250 250 x

Some of the columns you must fill in within the items file are as follows:

ITEM. The item refers to the item number, which you henceforth use in Cortex as the name or identifier of the stimulus you have created. Some items have negative numbers. These are "special" items that are used internally by cortex. Their meaning is as follows:

-4 Background item

-3 Fixation spot item



TYPE. Cortex can handle many different kinds of items, which generally fall into two classes. The first class Cortex "knows" how to create itelf. This class includes bars, circles, ellipses, annuli, and ascii characters. Each of these types has an associated code number, which you enter in this column (see below for a list of type codes). In addition, you must enter the appropriate information about the items in the other relevant columns in the file. For example, bars need a length and width, circles need a radius, etc. The second class is bitmaps and movies, which are items that you have created outside of Cortex and saved in a disk file. These also have a "type" number, which you enter in this column. For these bitmap and movie items, you must also give the name of the disk file that holds them in the FILENAME column. For the format of these bitmapped files, see Appendix A. All items, regardless of type also need an X_POSITION and a Y_POSITION, which you enter in the appropriate column.

Item type codes:

1 bar

2 circle

3 circular annulus (outer and inner diameter should be entered in the inner and outer columns)

4-6 reserved

7 ascii character

8 bitmap (name of file must be in FILENAME column)

9 ellipse

10 annular ellipse

11 movie (name of file must be in FILENAME column)

12 regular polygon

13 ascii string

FILLED. Objects such as bars, circles and annuli can either be filled or just an outline. If you put a '1' in this column the object will be filled, and if you put a '0' it will be an outline.

CENTERX, CENTERY. Each object must have a location on the screen. This sets the x and y axis location, in degrees, from the reference point.

BITPAN. Objects that are bitpannable means that you plan to pan them around inside a stationary window on the screen. A '0' in this field means that the object is not bitpannable, and cortex should treat it like any other object. A '1' in this field means that it is bitpannable, that cortex should utilize the window size you specify (see below), and that cortex should replicate the object 4 times, to give you more object to pan around with. A '2' in this field means the same as 1, but cortex won't replicate it. If you use option 1, there may be sharp transitions in the image at the boundaries of the four replications, but for some purposes, this is o.k.. Also, you may need to set the bitpan field to 2 if you have an extremely large object. The pepper card will not allow you to import a bitmap larger than the display screen (640 x 480), unless you specify a window around it that is smaller than 640 x 480.

WINWIDE, WINTALL. When you give a window length and width (in degrees), and the object is bitpannable, Cortex will create a window around your item. This will allow you to pan a large object around inside a smaller window.

HEIGHT, WIDTH, ANGLE. Bars require a length, width and orientation, in degrees.

INNER, OUTER. Annuli require an inner and outer diamter.

C (Color), -R-, -G-, -B-. For items that Cortex knows how to create, such as bars and circles, you must tell Cortex what color they should be. In the color column you may, if you wish, give a letter representing the color name you want. 'R', for example, stands for red. However, these default colors will probably not have the exact red, green, and blue values you want, so you should really list the actual rgb values you want in the file instead of just giving a color name. To give the actual rgb values, use 'X' as the color name in the color column, and then enter the appropriate values in the -R-, -G-, and -B- columns. These values range from 0 (off) to 255 (maximal brightness) for each red, green and blue gun on the CRT.

For items that are stored as bitmaps, the color you list in the items file is ignored. The colors are determined by the color lookup table, which is described in Appendix A. The name of a color lookup table file can be entered in a menu within Cortex or listed in the conditions file (see below).

FILENAME. For bitmap items, you must give the filename. The name can include a path, if it is not in the current directory.

3.3. Conditions files

The second ascii file you must import into Cortex is the conditions file. Cortex can interleave trials from a large number of experimental conditions within a single experimental session. Each condition determines what happens on a particular type of trial. In this file, you specify which items are linked together within each condition, and which timing structure is associated with that condition.

An example of a conditions file is as follows:


1 2 2 1 2 0 -3 lookup.0

2 3 6 7 5 4 1 1 0 -3


3 4 4 1 2 0 -3 lookup.0

4 6 7 8 9 1 1 0 -3

5 10 9 3 5 1 1 0 -3

6 12 13 12 13 1 3 0 -3

7 13 12 13 12 1 3 0 -3

COND# The condition number is, as you might expect, the number of each experimental condition. Each line typically defines a condition, but if you need to list more than two items on a given test screen, you can list the additional items on the next line. In the above example, condition number 2 extends across two lines. Depending on the version of cortex you are using, the maximum number of items per test screen is either 4 or 8.

TEST0, TEST1, etc. For each test screen you list the items that you want to appear together. The item numbers refer the items listed in the items file. On a single trial, you can have up to ten successive screens worth of images presented. For example, in condition 2, items 3, 6, and 5 will appear together first on the display screen. Then, items 7 and 5 will appear together, followed by item 4 alone. The timing and sequence of the display screens is given in the state system. You can switch from one display screen to the next in one frame, i.e. 16.6 milliseconds.

BCKGND. For the background, you list an item that you would like to have as the background on each test screen. If the background item is a bar, only the color of the bar will be used. This is the typical situation. If you list an item number that is, for example, a bitmapped image of a forest, the forest will appear as the background on each test screen at the start of the trial.

TIMING. In this column, you list which timing structure, i.e. state system file, you want to run with this condition. The state system files are imported separately, and are refered to by their number.

TRIAL_TYPE. Currently not used.

FIX_ID. In this column, you list which item you want for the fixation stimulus. This is typically item -3, the item number that Cortex uses internally to hold the fixation stimulus. However, you may list any item.

---COLOR-PALETTE---. The color lookup table for bitmapped items can either be entered from within one of Cortex's menus, or listed on a line in the conditions file. This should be the name of a disk file that holds the lookup table. Each condition can have its own lookup table. Lookup tables can also be loaded from the state system, for dynamic changes in color during a trial.

We have neglected the issue of timing -- when and for how long the stimuli remain on the screen. Each experimental condition is associated with a timing file. The same timing file may be used by all conditions or each condition may have its own. The timing file is discussed in the next section.

3.4. Old style timing files [OBSOLETE]

The timing structure is responsible not only for informing Cortex when stimuli should go on and off, but also for any behavioral contingencies such as monitoring for a lever press, checking eye position, giving rewards, etc. The timing structure of Cortex has been completely rewritten since the early days of Cortex. It now implements a powerful state system language. The language is "C-like" but also shares some similarities with BASIC and with SPOT, the state system language that is used by Barry Richmond and the Wurtz lab.

Within a single experimental session, several timing (or state system) files may be used. From inside Cortex you import each timing file you will use in an experiment, and essentially assign each a sequential number 1, 2, 3, etc. These numbers are used as names for the timing structures. In the conditions file (see above) you specify which timing structure is associated with each condition. For example, you may want to use two different timing structures in an experiment. Each will have its own ascii file, with its own file name, and you will import (from within Cortex) the one timing file as timing structure #1 and the other as timing structure #2. In the conditions file, some conditions must list timing structure #1 as the necessary timing structure and other conditions must list timing structure #2.

3.5. Old style state system programs [OBSOLETE - new users go to section 3.6]

For the user, the purpose of writing a state system in CORTEX is to direct the flow of control during a trial, including the timing of stimuli, the specification of behavioral responses, and the marking of event codes in the data file. If you have been using CORTEX version 1.0, the state system replaces the "timing file". As you will see, the state system is much more flexible than the former timing file and it now supports stimulus motion. An additional advantage for programmers is that it is now much easier to add to and makes changes to the CORTEX program itself. The price for this flexibility is that much more input is now required from the user. Now you must program every aspect of the flow of control, every contingency, every error state, etc., all of which used to be hidden from you. If you know the C programming language, the state language of CORTEX will seem very familiar - the language is essentially a subset of C, with a few added features.

Just as with the previous timing files, each state system is written as an ascii file and saved with a file name. After you have written and stored your state systems as disk files, you then import one or more of the files into CORTEX and give them each a number. As was true of the former timing files, each experimental condition (in the Conditions file) can call a state system, by its number.

3.5.1. Functions

When you write your state system specification, you will be making use of a library of functions, or subroutines, that have been prewritten in C and form a part of the compiled CORTEX program itself. A list of these functions and a description of what each does is included in a later section. Each of these functions has a name and is called from the state system like this:


This statement will call the "do_this" function from your state system (this just illustrates the form, there is not actually a "do_this" function in CORTEX). Note that, as in C, every statement must end with a semicolon.


In the newest version of the state system, you can place comments in your state code to enhance its readibility and maintainability. Comments are delimited by: /* and */, as in C. Thus, a legal comment is:

/* this is a comment */


Some functions must be passed one or more parameters. For example, CORTEX contains a function to initialize a timer, called set_timer. You must pass the function a time, in milliseconds. Parameters are passed within the parenthesis following the function name (note: all function names are followed by a pair of parenthesis, whether or not they are sent a value). Thus, to call the set_timer function with a time of 100 msec, you enter the following statement into your state system:


Unlike in C, only integer values can be passed to functions in CORTEX.

3.5.2. Functions that return values

In addition to receiving a value from the state system, CORTEX functions can also return values to the state system. For example, one CORTEX function, get_fixation_state(), returns a '1' if the subject is fixating and a '0' if not. The function timer_expired() returns a 0 if the timer is still ticking and a 1 if the time has run out.

3.5.3. States

Within the state system, you will write your own equivalents of functions. These special functions are called 'states', and because they are yours, rather than CORTEX's, you give them your own name. For example, you may want to define a state arbitrarily called 'iti', in which you will wait in an intertrial interval. To define the state, you enter the following statement into your state system:

state iti(){


Within the curly brackets, you place the statements that belong to your iti state. For example, to include the set_timer initialization within your iti state, you would enter:

state iti(){



Incidently, indenting is optional but it often helps you to read and understand our state system. The curly brackets that delimit the insides of the state are not an option; they must be used and they must be pointed in the correct directions. You can use spaces (or not) liberally in your state system, but you can't use them to break up the name of any function or state, i.e. names cannot have spaces within them. Thus, 'goto_state' is legal but 'goto state' is not because the name of a special keyword has been broken up.

At the end of each state (and often within, as you will see later), you must specify which state you will go to next. For example, you might define two states, an iti state and wait_fixation state. To go from the iti state to the wait_fixation state, you enter the following:

state iti(){

do_some_functions(); /* put in legal functions here */

goto_state wait_fixation();


state wait_fixation(){

do_some_functions(); /*put in legal functions here*/

} Starting and ending states

The very last state (or states, if your system can end in more than one way) should end with a call to exit(), which is a special function that terminates the state system.

The beginning state is called by a special function called main(). Every state system must have a call to main(). Here is a typical example:


goto_state iti();


state iti(){


goto_state wait_fixation();


state wait_fixation(){




It doesn't matter where you place the main() function in your file as long as it is outside of any other state or function. Neither is the order in which you list the states within your file important, although it would be helpful for readability if they were in logical order (i.e. the same order in which they were executed). Note that main() is not preceded by the word 'state'. Branching and logic testing

In the CORTEX state system, as in C, sequential statements are executed sequentially, unless you implement some branching or testing conditions. Indeed, you will often want to execute a statement or branch to another part of the state system depending on the value of some experimental variable. For example, if the subject fixates, you may want to go from the wait_fixation() state to the sample() state, but if the subject does not fixate, you may want to go to an error() state. Or, you may want to execute a function that turns on a stimulus, but only if the timer says it is the correct time. Thus, you must have some way of testing for a value and executing or branching depending on the result. The testing, or logical, operators in the CORTEX state system are as follows:

== logical equal

!= logical not equal

The branching or testing keyword is 'if'. To execute a function depending on the value of a variable, you can use 'if' as follows:

if(x == y){



Or, to implement a branch:

if(x == y){

goto_state iti();


if(x != y){

goto_state fixation();


In a real state system, 'x' in the above examples would be replaced by a function that returns a value, and y would be replaced by a number. For example, recall that the get_fixation_state function returns a 1 if the subject is fixating and a 0 if not. In the CORTEX state language, as in C, the value returned by a function is inserted into the place where the function name appears in the statement . Thus:

if(get_fixation_state() == 1){ /*if the subject is fixating */

goto_state sample(); /* go to state that turns on stim. */


if(get_fixation_state() == 0){ /*if subject is not fixating */

goto_state error(); /* go to an error state */


In this example, the program control goes to the sample() state if the subject is fixating, and to the error() state if not. 'While' loops

The condition following 'if' is only tested once, and if it is true, the statements inside the 'if' brackets are executed once. However, it is often desirable to execute one or more statements repeatedly as long as some condition is true (i.e. a "loop"). This accomplished with the 'while'


For example:

while(x == y){



If the condition following the 'while' is true, then the statements in the brackets are executed and the condition is then retested. If true again, then the statements inside the curly brackets are executed again, and on and on until the condition finally tests false. When the condition is false, program control procedes to the statement following the end of the 'while' end-bracket. As a practical example, you may want to test for whether the subject achieves fixation during a certain time interval. Recall that the function timer_expired() returns a 1 if the timer has run out and a 0 otherwise. Thus:

state wait_fixation{


while(timer_expired() == 0){

if(get_fixation_state() == 1){

goto_state success();



/* You only get to the next statement if fixation was not achieved */

/* before the timer ran out */

goto_state error();


In this example, the timer_expired function is called repeatedly at the start of the while loop. As long as it returns a 0, meaning that the timer has not expired, the statements inside the while loop are executed. When the timer_expired() function returns a 1, meaning the timer has expired, the loop is exited, and the statement following the loop is then executed (in this case, it is a branch to an error state). Note that it is also possible to get out of the loop if the get_fixation() function returns a 1, meaning that the subject has achieved fixation. In this case, the program branches to the 'success' state.

The above example illustrates nested 'while' and 'if' statements, which is

a common construction. Proper indenting of brackets helps you keep track of

the beginning and end of the various states, while loops, if statements, etc.

Actually, it is possible to eliminate one set of brackets in the above

example. If a 'while' or 'if' condition is followed by just a single

statement, you don't need the curly brackets. Thus, the following is a legal


if(get_fixation_state() == 1)

goto_state success();

3.5.4. Running the state system

In version 2.5 of CORTEX, the preprocessor (preppy.exe) and compiler (state.exe) have been removed from the Cortex program itself, and operate as stand alone executable programs that are "spawned" by Cortex. This is transparent to the user, except that you must make sure that these two other programs exist on your disk, in addition to Cortex. They may be located either in your current directory or a directory of your choice, provided that you specify the path in your autoexec.bat file. The path to each is set through two environment variables, preppy and state. To specify the path for preppy, add the line: preppy=\path\preppy to your file, where 'path' is the path you choose. Likewise, for the state compiler program, add the line:


You import the state system using the 'timing' menu, as in the previous version of CORTEX. For state system #1, choose 'modify', then choose sequence '1', then give the name of your state file. CORTEX will then immediately load in the file, compile, and assemble it. You can add additional state systems by choosing 'add' from the menu. If you get an error message (as opposed to a warning), the compilation failed and you should check your state system code. If, after you have entered your state systems, you want to change one of the timing sequences to a different state file, just choose 'modify', give the sequence number, and then the new file name.

3.5.5. Preprocessor directives: #define and #include

Many state functions require that a special code or value be passed to them. Although these can be passed by value, it is convenient to give some common values special names, which have some meaning. For example, rather than pass the encode() function the value 23, which is the value for turning the sample stimulus on, it is convenient to define, say, TURN_SAMPLE_ON to mean 23, and then use TURN_SAMPLE_ON rather than 23 throughout the remainder of the file (it is a C convention to use capital letters for #defined names - it is desirable, but not essential that you use this convention). To set up this definition, you can add the line:

#define TURN_SAMPLE_ON 23

at the start of your state file.

If you have a large number of #defines at the start of each of your state files, and these #defines are the same for every file (as they almost certainly will be), it is convenient to stick them in one special file, and then attach this definition file to each of your state files. To do this, you put the line:

#include <filename.h>

at the start of each state file, where filename.h is whatever file name you have given your special "header" file. Typically, this is defs.h.

3.5.6. Library of CORTEX functions

A brief description of each function appears in a later section. However, two sorts of functions require that you understand a little bit about of how CORTEX works. One of the functions is encode(). You pass encode() a number which represents the code of an event that you want to mark in the raw data file. Place the call to encode immediately following the event. The time that the encode() function is executed is also written to the data file. These codes are critical for later data analysis. The only way that STATS (the analysis program) or RASTHIST (the graphing program) knows when a stimulus went on or off, or when a certain interval took place, or when a certain behavioral response took place, is from the codes written in the data file. Thus, it is critical that you mark the (correct) code for every type of event that occurs during the trial. You can verify these codes by choosing the 'display' option from the run menu after you run a trial. Incidently, before the state system interface these codes used to be marked for you, in a hidden and quasi-mystical fashion, by CORTEX itself. Now, since you mark the codes, you have the opportunity to totally screw up the data file. So, mark them carefully, using the codes listed in simple.c or later in this manual.

The second sort of functions that require some explanation are a set of behavioral error and success marking functions, such as missing_response(), success(), break_fixation(), etc. These functions do not affect the codes and times in the data file. Rather, they only affect the "header" for each trial's worth of data in the file. Like the codes and times, this header information is used by STATS and RASTHIST for analysing the data. Specifically, it is used for sorting trials, so that you can ask STATS, for example, to only analyze the "correct trials". STATS knows what is a correct trial from the information you place in the header for each trial. The header information is also used by STATS to give you the "behavioral report" of the number of correct trials, or trials with loss of fixation, or no response, or an incorrect response, etc. As with the codes, this information used to be placed in the header for you by CORTEX - now you get to do it yourself.

3.6. Technical information for C programmers

As is probably clear, the CORTEX state language implements a subset of the C language. For example, you cannot declare structures.

If you need any of these more advanced features of C, you should write a C function and incorporate it into the CORTEX program itself.

You may be interested in how the state system works. Briefly, the state system 'engine' consists of a preprocessor, compiler and an assembler. The preprocessor and compiler and stand-alone executables that are spawned by CORTEX, but the assembler is part of CORTEX itself. The preprocessor simply handles the #defines, #includes, and comments. Its output is an ascii file named 'preppy.out', which you can examine. The compiler has been written in LEX and YACC (actually, Bison and Flex), with the resulting C code modified to work inside CORTEX. The compiler produces something that looks like assembly language. You can examine its output by looking at the 'state.out' ascii file which is produced every time you import a state system. The output of the assembler is, in part, a table of function pointers. The assembler output is executed by an interpreter during the execution of the trial.

You can actually run the preprocessor and compiler on their own, outside of cortex, by executing them with the state system files on the command line:

preppy state.1 > preppy.out

state preppy.out > state.out

The output files can then be examined with any text editor.

3.7. Current state system programs

(0) Backwards compatability:

IMPORTANT NOTE TO USERS OF 3.0: CORTEX 3.0 added the graphics kernel before CORTEXCC (CORTEX's new C Compiler) was develloped, and thus used the archaic calling convention. Since the kernel is designed to last CORTEX for years, I had to upgrade the calling conventions to take advantage of FLOATs and LONGs. Thus, the prototypes have been changed so that you can, and are now required to, pass the actual degrees of visual angle, instead of 100* them. It is hoped that few, if any people, will be inconvienced by this.


The old state system was such a mess that it had to be completely scrapped. However, the files written for versions of CORTEX prior to 4.0 are easily upgraded:

(1) "goto_state sample();" -> "goto sample;"

(2) "state sample() {" -> "sample: {"

(3) "exit();" -> "return;"

(4) enclose all states within the braces of main() { }, since the old state system was really only one function, with a bunch of "goto"s.

If you use BRIEF, which I highly recommend, the following translate commands will do the above specified conversions:

(0) commands will be specified as Pattern: "????" -> Replacement: "???"

(1) P: "goto_state{*}()" -> R: "goto \0"

(2) P: "<state{*}()" -> R: "\0:" // assumes state at left edge

(3) P: "exit()" -> R: "return"

(4) move "main()" and its body to the top, and put its closing brace after all of the states.


CSS is a subset of "C". With the help of BISON to do the parsing, the CSS simulator is able to do all of the standard looping operations, function calls, global and local variable declarations (so recursion is possible) of the types {char,int,long,float,pchar...,ppchar...} where "pchar" == "(char *)" and "ppchar" == "(char **)".

Because CSS is an interpreter, there are two types of functions available: those written in "C" and declared as system functions, and those written in CSS. Obviously, the "C" functions run a bit faster, but they require a "C" compiler and linker. Thus, CSS should suffice, but if you have some time critical functions, it is best to port them (fairly directly) over the appropriate "C" file and re-compile CORTEX.

The following are the incompatabilites with "C":

(1) "typedef", "union", "struct" and "enum" are not supported.

(2) Only unidimensional arrays are supported.

(3) Hex and octal numbers are parsed, but numbers with exponents aren't.

(4) In the logical operations && and ||, only those args necessary to determine the answer are evaluated (e.g. (A&&B) does not eval B if A==0, and (C||D) does not eval D if C==1). The only deviation from "C" is that this is implemented via conditional jumps, so that (5||7), (0||5), and (5&&3) all == 5 instead of 1. Therefore, when using these operators, it is best to ask "if (A&&B) stmt;" instead of "if (A&&B == 1) stmt;" since the later may not work, and the former is faster anyway.

(5) Variable length arg lists are only supported for System Fuctions

3.7.1. State system writer's reference to Preppy

PREPPY is a nearly full implementation of MICROSOFT's C pre-processor.

The following directives are supported:

(1) #include "filename" // but not #include <filename>

(2) #define NAME val // e.g. PI -> 3.14

(3) #A // stringize A's value: #PI -> "3.14"

(4) arg1##arg2 // fuse: "Pi = "##PI -> "Pi = "3.14

(5) "str1" "str2" // concatenate: "Pi = " #PI -> "Pi = 3.14"

(6) #define MACRO(a,b,...) val2

// #define MATH_ALU(class,op,op_name) CSS_##class##_##op_name(void)

// MATH_ALU(char,+,add) -> CSS_char_add(void)

(7) #define lines ended with '\\' append next line to #define value.

(8) #ifdef, #ifndef, #else, #endif // conditional compilation

(9) #elif not supported: replaced with #else #ifdef or #else #ifndef since all directives can be embedded, like #include in included files, nested #ifdef loops, and #define macros which reference other macros.

3.7.2. State system writer's reference to Cortexcc

(1) Data types: (char,int,long,float,pchar,...,ppchar,...} where pchar = (char *) and ppchar = (char **). Data is stored internally in a the union:

union dataEL {

char _char; int _int; long _long; float _float;

pchar _pchar; pint _pint; plong _plong; pfloat _pfloat;

ppchar _ppchar; ppint _ppint; pplong _pplong; ppfloat _ppfloat;


Macros are heavily used in the code, as cssmath.c makes evident.

(2) Data scope: (static,extern,auto)

Only "static" requires a reserved word. If variables are declared globally (outside of functions), they are automatically static, which means that they are stored in the data segment. "Static" scoping within functions is fully supported.

Extern vars are special, since this implies access to variables

within CORTEX proper. Within the file "ufnlist.c" is an array

EXTERN_VAR ExternVars[], to which new externs can be easily added

following the format shown.

(3) Pointers:

Address indirection is supported with the following limitations:

(a) Strings are fully supported

(b) Unidimentional arrays, such as x[10] are fully supported.

(c) Assignment by indirection is also fully supported.

(d) Pointer math, like *(x + 10) is NOT supported, and may never be.

(4) Flow control: (if else do while for switch ()?: goto break continue)

All normal C flow statements are supported.

(5) Math:

All C math functions are supported except for (>> >>= << <<=), and those could be easily added to cssmath.c if one wishes. Pre and Post ++/-- are fully supported. Internally, the functions are stored in the arrays:

P_F_dataEl CSSOpAsgnArray[][MAX_MATH_TYPE];

P_F_dataEl CSSpOpAsgnArray[][MAX_MATH_TYPE]; // p = pointers


P_F_VOID CSSpMathArray[][MAX_MATH_TYPE]; // p = pointers

so that all math ops are rapidly accessed by calling the function indicated by the data type and math operation indices.

(6) Cast conversions: (fully supported). CORTEXCC parses all numbers as either LONG or FLOAT. Constants are cast to the proper destination type at compile time. In situations where variables of different types are assigned, or math operations must be done, casts are done at run-time.

(7) Functions: (System,User)

(A) Reserved functions: (main,pre-main,post-main,interrupt) Each of these functions takes no arguments, and returns nothing, so instead of listing a prototype like:

void main(void) { }

these are accessed as

main() { }

CORTEX does not currently use the other functions, although they are correctly coded by CORTEXCC. They are envisioned for use in future flexibility in graphics manipulations, especially if we eventually use a separate computer for graphics display.

(B) System functions: specified in the array SYSTEM_FN SystemFns[]; in ufnlst.c. The third entry is the syntax, so that proper syntax checking can be done, and so that incorrect syntax or number of arguments will generate error messages.

(C) SYNTAX: Values include (cilfCILF0123.). The first entry in the string is the return type. This, and only this, position can also take the value 'v', for void. The following entries indicate the type for each subsequent arg. Functions with no arguments take a syntax string of length one, such as "v", NOT "vv". As to the subsequent args, lower case letter indicate values, uppercase letters mean pointers to that data type, and numbers are double pointers to that data type. For example, the function:

void MessageString(int which, pchar message) -> "viC".

The '.' is used for variable length arglists, such as void printf(pchar fstring, ...) -> "vC.". Thus, when '.' is the last entry in a syntax string, that arg is optional, and any number of args of any type may follow the last required arg. CSS ensures that the proper number of args are POPed. Only the last entry in a syntax string may be '.'.

(D) User Functions: fully supported. Prototypes must be specified before these functions are used. Again, void is not specified except as a return type, so functions which take no arguments are specified like:

int print_hello();


float power(float x, float y);

3.8. Using the ITEMS, CONDITIONS, and TIMING files in a study

By modifying different ITEM, CONDITION, and TIMING files, either alone or in combination, you can create many different experiments. For example, once you have set up an experiment using a particular set of ITEM, CONDITION, and TIMING files, you could change the stimuli alone by importing a different ITEMS file into Cortex. Another way to achieve the same end would be to import a single large ITEM file that held a large "menu" of stimuli to use, and then switch the items to use for a particular experiment by importing a new

CONDITIONS file. The new CONDITIONS file will select new items from the ITEMS File. Alternatively, your primary interest might be in different experimental paradigms rather than different stimuli. In this case, you could keep the

ITEMS and CONDITIONS files constant, but import different timing files for

different experiments.

3.9. Configuration file

Cortex is organized hierarchically (much like the real Cortex!) and is menu driven (not like the real Cortex). You run Cortex by entering Cortex -n, where 'n' is the name of the disk drive where you would like temporary files stored. This drive can be a RAM disk, to speed up performance. The temporary files do not contain the experimental data but rather hold information used internally by Cortex, such as binary representations of the items.

3.9.1. Configuration file

Within your current directory, you should also have a Cortex configuration file, which must be named 'Cortex.cfg'. This file contains information about the type and address of the hardware devices you are using as well as information about the size and location of graphic windows on the user's display (e.g. the the histograms, rasters, eye position display, etc) [NOTE - the latter is true only for versions of Cortex starting with version 2.5.]. The file is an ascii file that can be edited both to conform to your current hardware configuration and to customize your display. An example configuration file should have been included with your original distribution disk. One critical piece of information to check when you first run the program is the graphics device, which should be either Hercules, VGA, or None [NOTE: Again, this only applies to versions 2.5 and greater]. If this is not set correctly, your screen may go haywire when the program loads. In general, we find that there are fewer compatibility problems with the pepper card if you use the hercules card as the user's display device. However, vga should work. If Cortex finds the configuration file, it will display its contents and ask you to type 's' to procede. If it cannot find the file, it will ask you if it should assume a set of default values. Otherwise, cortex will abort.

A second critical piece of information to check is the name and address of the analog/digital io board in your system.

If you have a Metrabyte_Dash-16, list that as the 'primary device' and give its base and register address (add 9 to the base address to get the register address).

If you have only a Metrabyte_PIO24_or_PIO24, list PIO24 as the primary device, giving its base address (the register address is ignored). In this case, Cortex will only collect digital data.

If you have both a Metrabyte_PIO24/PIO24 and Metrabyte_Dash-16, list the Dash-16 as the primary device and the PIO24 as the secondary device. The Dash-16 will be used for analog (eye movement) and spike input, and the PIO24 will be used for additional digital input/output.

If you have a Compuboard_CIO_AD-16, treat it as a combination of a Metrabyte Dash-16 and PIO24 (i.e. list the Dash-16 and PIO24 as the primary and secondary devices).

There is preliminary and rudimentary support for the Metrabyte_PIO96. If you have a PIO96 (which is the equivalent of 3 PIO24s), list the PIO24 as the primary device and the PIO96 as the secondary device. Give both the same base address.

An example configuration file is as follows (as in C, double slashes indicate



Primary_device Compuboard 0x300 0x309 //address = 300 Hex

Secondary_device Pio24 0x310 0x310 //address = 310 Hex

Graphics_specs 8 Never




// Locations of the various data windows on the user's screen.

// (window corners are in virtual device coordinates, with a max of 10,000)

STATUS_RECT 3250 8500 10000 10000 3

Eog_display 3250 0 10000 8300

Histogram1 0 1000 3000 4700 // has 5% boundary

Raster_table1 150 300 2850 900

Raster1 150 0 2850 200 // since 5% boundary

Histogram2 0 6200 3000 9900

Raster_table2 150 5500 2850 6100

Raster2 150 5200 2850 5400

//the following are functions that put information on the user's screen

GETScond_num "cond# " 5

GETStrial_num "trial " 5

GETSnothing "" 5

GETScollect_data "GetData" 3

GETSbin_width "bwidth " 6

GETSsaccade "saccade" 6

GETSwindowX "windowX" 6

GETSwindowY "windowY" 6

GETShtg_scale "scale " 6

GETSfile_name "file " 15

GETScond_pct_correct "correct" 8

GETStrial_status "status " 15

TEXT_COLORS 0x07 0x0f 0x70 0x87

// end of configuration file

// below are examples of other configurarion options you might use

Primary_device RANDOM_SPIKE_DEVICE 0 0

Graphics_specs 4 Bitmap

Graphics_specs 8 Never

Graphics_specs 2 Movie

Graphics_specs 1 Both

Primary_device No_device 0 0

Primary_device Pio24 0x310 0x310

Primary_device Compuboard 0x300 0x309

Primary_device Metrabyte 0x310 0x319

Primary_device No_device 0 0

Secondary_device Pio24 0x310 0x310



4. Running Cortex

After Cortex loads, you will be confronted by the following menu:

Run Play Item Condition Timing Disk Analysis

To choose a command from the menu, either type its first character or highlight it with the cursor keys, followed by the "enter" key. Normally, you will want to import the items, conditions, and timing files first. Note: you can temporarily branch out of the program into Dos by hitting the F2 key. This is useful if you need to check a directory or edit a file. Type 'exit' to get back into Cortex at the point you left off.

ITEMS. To import the items file, type 'i' for items, and Cortex will display the items menu. Then choose 'i' for import, and give the file name. If the items file contains bitmaps and you need to import a color lookup table, type 'c' for the CLT command, then give the filename for the table. To actually see one of the items on the graphics screen, type 'd' for Display, and give the item number when prompted. If you simply want to see a listing of the item's properties (item type, length, width, etc.) type 'v' for view and give the item number when prompted.

Reference_point. Another items menu choice is the reference point. If you type 'r' for reference, Cortex will tell you what is the current reference point and will then ask if you want to change it. The three possible reference points are 'fixspot', 'receptive field', and 'other'. The reference point is the location that Cortex defines to be the center of its coordinate system for placing items on the screen. For some studies in fixating monkeys, the fixation spot is the most natural reference point. In you make the fixation point the reference point, then the location of all the stimulus items will necessarily be in retinal coordinates. Alternatively, you might want your items displayed at locations relative to some other point on the screen, such as the center of the receptive field you are studying. If so, choose 'receptive field' or 'other' as the reference. Note that, unlike other items, the fixation point location is always specified in screen coordinates, i.e. its location is not referred to any reference point other than itself.

After you choose a reference point, Cortex will ask you for its x and y screen coordinates, in degrees.

The reference point and its location can also be changed within the "play" routine, which is described a little later.

Conditions. From the main menu you can access the Conditions menu just like you did the Items menu. Like with items, you can ask Cortex to display a condition. One difference from the items display is that when you ask Cortex to display the items listed on a particular test screen of a particular condition, it will show you all of the items that you linked together on the test screen, including the background item. Thus, you can see how the stimulus screens will actually appear during a trial.

Timing. Like the Items and Conditions, you access the timing menu from the main menu. Unlike the case for Items and Conditions, you can import multiple timing files. To import the first timing structure, type 'm' for modify and specify timing structure number 1. After importing the first file, type 'a' for Add, to add more timing files to the list. If you make a mistake, you can go back and reenter a state file, using the modify option. Each condition in the Conditions file must specify the number of a valid timing structure.

Disk. Rather than importing the same files each time you run an experiment, you can save all of the files and other parameters in a single disk save file, which you can import the next time you run the program. Go to the disk menu and type 's' for Save. Normally, you would not 'save' the files and parameters until you have entered all of the information required by Cortex, which is described below. To retrieve the file at a later time, type 'g' for


BRANCH TO DOS. From any menu you can hit the F1 key, which will put you into DOS while retaining Cortex in memory. From DOS, you can run your favorite editor to change any items, conditions, or timing files you want, or you might execute a Dos command such as a directory listing, or whatever. To return to the program, just type 'exit'. Everything you have entered into the program is saved when you return, as is any data you have collected. If you changed an items, conditions, or timing file while in DOS, Cortex will not know about your changes unless you re-import the file into Cortex.

Play. When you are collecting data, Cortex runs in a very automated fashion and does not allow you to change stimuli interactively. However, it is often desirable in mapping receptive fields to be able to change stimuli and their location "on the fly". In such a informal mode, you generally don't need to be collecting and storing data, since you are changing stimuli in an unpredictable way. Rather, you want to be able to hear the neuronal responses on a loudspeaker while you are playing around with stimuli. After you have decided on the receptive field, you want to be able to use the receptive field center as the reference point for all subsequent stimulus presentations when you collect data. You may even want the play stimulus you have created to become an item that is displayed automatically when data are collected again. All of this is what PLAY is designed for. Play has two modes, "alone" and with "conditions". The "alone" mode is appropriate for experiments in which there are no behavioral contingencies. In this mode, PLAY simply allows you to move and change stimuli constantly, using a mouse.

In the "conditions" mode, PLAY runs whatever experimental conditions you choose, and simultaneously allows you to change and move stimuli around with the mouse. The experimental conditions in this case normally have some behavioral component to them -- e.g. a fixation task. In this mode, PLAY will continue to reward the animal for performing the task correctly and will operate just as it does when you are collecting data. The difference is that you have control over your own mouse-driven stimulus.

When you have found a particular receptive field location that you want to save, PLAY allows you to save the play item parameters, including its location, to any item in your item file. The item you save the play item to can also be the reference item. Reference point items are described in the section on the ITEMS menu, in running Cortex. From within PLAY, you can select 'receptive field' or 'other' to be the reference point, and then save the location of the mapping stimulus to the reference point. Then, when you return to collecting data, the items will be displayed at locations relative to the center of the receptive field that you found. The reference point can also be changed from within the ITEMS menu, described previously.

Run. After importing the necessary files, you go from the main menu towards the menu for running trials and collecting data. The run menu looks as follows:

Histogram Resume Start Parameters Clear Error Display

HISTOGRAM. The most commonly used display is the "single histogram" display that displays the histogram (s) from a single histogram on the screen at one time, plus a display of eye position, if applicable. Single histogram is the default mode, and thus you do not need to enter this menu to set it.

PARAMETERS. The parameters menu has recently been changed. This needs to be rewritten.

Some of the more obscure parameters are as follows:

Blocking. Cortex allows the conditions to be grouped within blocks. Within each block, you can specify how many trials you want to run. This has great utility when you want conditions to be randomly chosen within a small subset of the conditions, but not across the full range of conditions. For example, you might set up one set of conditions in which the subject is to perform visual discriminations based on color and another set of conditions in which the subject is to perform discriminations based on orientation. By placing the color conditions within one block and the orientation conditions within another, you can prevent the orientation conditions from being interleaved with the color ones.

Selection_with/without_replacement. If you choose 'without replacement', then when Cortex picks a condition to run, it deletes that condition from the list of conditions that it will choose from for the next trial. All of the conditions within a block will be presented once (up to the maximum number of trials you specify) before the list of available conditions is regenerated, at which point a given condition may be chosen again. If you choose 'replacement', then Cortex puts the chosen condition back into the available list of conditions, so that condition might even be chosen again for the next trial.

Pick-type. In addition to selection without replacement, there are other blocking options that affect how conditions are chosen. Some of the possible condition-picking options are sequential, random, and "put-back-random". The last option is extremely useful, but it requires some explanation. When training an animal, it is often desirable to repeat a condition that it gets wrong, so that it doesn't decide to just skip all the difficult conditions or make just one kind of response. A common technique is to use "correction trials", which means to simply repeat a condition that is not performed correctly. The problem with conventional correction trials is that an animal who makes an error can learn to simply make the opposite response on the next trial, and it will then always get the next trial correct. A better way to do correction trials is to put the condition performed incorrectly back into the list of conditions to be chosen from randomly on the next trial. This option assumes random selection without replacement for all correctly performed trials. The incorrectly performed condition will keep coming up in the block ever more frequently (but not necessarily the very next trial) until the animal gets it correct.

Pixels/Deg. Because you specify all locations to Cortex in terms of degrees of visual angle, Cortex needs to know how many pixels equals one degree. You can calculate this based on the size and distance of your monitor, in conjunction with the fact that the Pepper SGT displays 640 pixels across the horizontal dimension of the screen. Cortex assumes that your monitor has the correct aspect ratio, so that pixels are square.

Window. For studies in fixating subjects, you need to specifiy the size of the fixation "window", or fixation criteria. If, in the timing file you wrote that the subject must start to fixate by some time point during the trial, then Cortex will use the window size to decide whether fixation has been achieved (and remains achieved). To open the window all the way up, give it a size of 30 degrees or more.

Gain. Cortex cannot actually measure degrees of eye position. It can only measure voltages from the A/D convertors. Cortex assumes that the full scale range of the A/D convertor is equivalent to an eye movement from one side of the display screen to the other. If this is not so, you should adjust the eye position gain either using the gain setting on your eye tracking hardware or the gain setting on the analog input board. Typically, you shift your fixation stimuli 5 degrees or so to the left, right, up, and down, and keep Adjusting the eye coil hardware gain and offset setting until cortex shows that the animal's eye position is on target (hopefully, your monkey knows how to fixate). The size, in degrees, of the "crosshair" on the eye display can be changed by setting the "saccade criterion" parameter in the eye menu, which is a convenient way to measure distance in degrees on the display screen. If for some reason you can't get your gain high enough or whatever, you can also enter an integer gain multiplcation value in the parameters menu, which will be used to multipy all incoming A/D values. The default is 1, which is the recommended setting.

5. Limitations of Cortex due to structure and hardware

5.1. Data storage

Although the details of the program will be discussed later, one important detail relevant to the capabilities of Cortex is the manner in which data are stored. Cortex currently keeps all of the raw data (time of occurrence of each spike etc.) for an experimental run in a disk file rather than in memory, except for a one-trial buffer. While this allows both for a virtually unlimited amount of data to be collected and for a simple data-saving module, it would be too time-consuming to sort through the disk file to resynchronize the histograms and rasters to a new time or event on-line during an experiment. Therefore, the synchronization of the on-line rasters and histograms is set and controlled by the state system and cannot be resynchronized during an experiment. You can, of course, do anything you want with the raw data outside of Cortex itself, after the data have been collected.

5.2. Data file format

Each trials worth of data is preceded in the disk file by a "header" that contains information such as the experimental condition number, the type of behavioral response that was expected of the animal, if any, the type of response the subject actually made, whether the trial was performed correctly, types of errors that were made, and the sizes of various arrays that follow. One array contains the codes for all events that occurred during the trial, including spikes, stimulus presentations, behavioral responses, etc. A second array contains the times of occurrence of each of the corresponding events. Time is measured from the beginning of the trial, and is represented by a 32 bit long integer. Additional arrays may contain eye movement or evoked potential analog data. The information in the header can be used to quickly sort trials for later data analysis. For example, suppose you want to know the individual firing rates to each of the stimuli presented on each experimental condition, for correctly performed trials only. This can easily be done using a combination of information in the header together with the information in the data arrays.

It is important to note that histograms are not stored in the disk data file. Averaged histograms can be recompiled off-line from the data in the arrays, using the GRAST program. GRAST also has the capability to synchronize the histograms on any arbitrary event.

5.3. Interrupt structure

Cortex currently runs with a single interupt routine, which both updates a software clock and samples all data lines. The interupt is initiated by the programmable system clock on the AT. This clock normally interupts every 55 milliseconds when MS-DOS is in control, but we re-program the clock to interupt every 1 millisecond (optionally 0.1 millisecond), and we set the interupt vector to point to our own service routine. We don't allow the MS-DOS clock interupt service routine to operate when Cortex is in control; thus, time stands still as far as MS-DOS is concerned. The reason we do this is that the AT has a primitive interupt handling system compared to the PDP-11, and we think it is unwise to allow any interupt routine to gain control of the CPU except for Cortex's own critical clock service routine. The control of behavioral contingencies and the presentation of stimuli all takes place outside of the service routine (see description of software).

5.4. Graphics display hardware (Pepper SGT)

While the Pepper SGT has been a workhorse graphics card for neurophysiology, Number Nine Computer Company decided in November, 1993 to no longer manufacturer it. We are currently working on support for a new board. The following is a description of the SGT's capabilities, which must be emulated in any future replacement.

The Pepper SGT has some very nice features for visual neurophysiology. For one, it operates independently of the graphics card that puts up the user's display, so that the user and experimental subject monkey can view different screens simultaneously. Secondly, it can simultaneously display 8 bits of color or grey scale from a 24 bit palette, which is marginally adequate for stimuli such as sine-wave gratings that must be "linearized" in intensity to correct for CRT displays, which are always highly non-linear. Third, you can have up to 4 megabytes of memory on the card (although 1 MB is sufficient for most purposes), which gives you plenty of space to store patterns and also room to "roam" over patterns. Fourth, it runs at 60 hz non-interlace, with 640 x 480 resolution, which, again, is marginally adequate for neurophysiological work in most cortical areas (the 60 hz might be a problem in the LGN). Fifth, the Pepper SGT informs the program of when it begins a vertical sweep of the screen, so that stimulus presentations can be timed to 1 millisecond accuracy (That is, you can know when a stimulus is presented to 1 millisecond accuracy. You have less control than this over the presentation time, because the SGT might be in the middle of a sweep at the precise time you want your stimulus to come on. Stimuli can be switched on only at the start of the sweep.) Finally, it has the Intel 82786 graphics chip which has hardware support for windows that have independent sizes and positions. Thus, for example, you can keep one stimulus stationary on the screen (such as a fixation stimulus) in one window while other stimuli move around in different windows (analogous to sprites), or you can scroll a grating inside one window in one direction while scrolling another grating in another window in a different direction. A major decision for us was to put the stimulus display card in the same machine that collects the data. The alternative would be to use two ATs, one for data and one for graphics, with communication between them. The current scheme seems to work well, and it is cheaper, but it would be fairly easy to tear the graphics out of the program and have it run in a separate machine in the future.

Because it is often necessary in visual neurophysiology to switch rapidly between two or more different images, each of which is large and complex, we decided that the stimuli to be presented on a given "trial" in an experiment would be drawn in different portions of the graphics memory before the trial begins. This allows us to change to a new image within one video frame (16 millisecs). Note, that we do not store all of the images for an experimental_session in graphics memory, because there is no limit on the number or size of stimuli in a session, and, thus, we could not always be sure that they would all fit in. Rather, we just load in the images for a single trial at a time. Optionally, you can instruct Cortex (within the state system) not to unload an image at the end of a trial.

5.5. Dual monitors

For animal neurophysiological studies, it is often convenient to have one stimulus display monitor for the subject and a slave display for the experimenter, so that the experimenter can see what the animal is seeing even though the animal is in a box or on the other side of the room. We have found the VOPEX video splitter from NTI to work well in this regard. In addition to the splitter, you will also need a second VGA-compatible display adapter for the experimenter to look at, of course.

6. Bitmap File Format and Color Lookup Tables

6.1. The bitmap file

Images must be a binary file, arranged as successive rows of 8 bit pixel values (but only the first 7 bits are read -- the 8th bit is ignored), i.e. the first horizontal row of pixel values, followed by the second row, etc. Preceding the image data are 2 integers that specify the X by Y size of the image. At byte location 12 in the file you should put the X size, in bytes. At location 14 should be the Y size (number of rows). In addition, at byte location 16 should be the number of frames, for movie items only. The image data itself should begin at byte location 18 in the file.

This information is summarized as follows:

Byte-Location in File

0-11 12-13 14-15 16-17 18- to end


Movie_Items. For animation, such as dynamic motion displays, you may concatenate any number of images in the same disk file, and you can instruct Cortex to display them sequentially (or any order you like) up to 60 frames per second. To allow for simple concatentation of image files, each frame should be organized like the bitmap described above. That is, each frame should have a header, which sets the x and y size. The number of frames in the movie only needs to be set in the first frame's header.

6.2. Color Lookup Table

The value that is written into a given pixel location in the graphics card memory does not, in itself, specify what color or intensity should be displayed on the screen. The pixel value becomes associated with a color by virtue of a lookup table in the SGT memory. The color lookup table of the SGT maps each of the 256 possible values a pixel can have (i.e. 8 bits) to a 24 bit color value, 8 bits each for the red, green, and blue guns of the display monitor. Thus, there are 256 rows in the table, each of which requires a red, green, and blue intensity value:

Pixel Arbitrary_Gun_Values

Value Red Green Blue

0 100 150 200

1 50 75 10

2 90 87 100


Cortex uses the first 128 of the 256 rows in the lookup table for single-color items such as bars and circles, and the upper 128 rows to hold the color values for bitmapped patterns stored in disk files. Each of the bar and circle items (which are 1 bit "deep") in a given condition gets one row in the lookup table. The user enters the red, green, and blue values for each of these items (Note: the actual row assigned to the item in the lookup table is irrelevant to the user, and therefore is assigned by Cortex using a specific algorithm).

For the bitmapped items, which are necessarily 7 bits "deep", you must fill in the upper 128 rows of the lookup table. This table of values must be created off-line, and Cortex must be given the name of the file that holds the values. The file is organized as a series of four successive integers for each of the 128 table locations that will be used. The integers hold the red, green, and blue gun intensities, plus a dummy integer (this is the format required by NNIOS, the Pepper SGT software interface). The user has the option of specifying one "experiment-wide" lookup table that will be applied to all bitmapped items, or of assigning a unique lookup table to each experimental condition (see below). At the appropriate time, the lookup table values are read by Cortex into the SGT memory. These lookup tables can be full-color tables or grey-scale tables, depending on the needs of the user.

For achromatic sinewave gratings, the user will normally "linearize" the output on the display CRT by adjusting the lookup table values so that equal increments in pixel values will produce equal increments of intensity on the CRT screen. It should be noted that in the current version of the program, if the user wants two bitmapped full-color items on the screen at the same time, they must use the same lookup table.

Cortex can interleave trials from a large number of experimental conditions within a single experimental session. Each condition determines what happens on a particular type of trial. For example, in one experimental condition, you may want just a red item presented, and in a different condition you may want just a green item presented. Therefore, you would set up two conditions, each with a single item on the test_0 screen.

Why use two conditions rather than one in this example? Why not just present the red item on the test_0 screen and the green item on the test_1 screen of a single experimental condition? You could do this, but then the order of stimulus presentations would be the same throughout the experiment (i.e. red followed by green). If you want stimulus presentations randomly interleaved, you should assign stimuli to different experimental conditions because Cortex can only randomize the order of presentation of conditions, not the screens within a condition (actually, strictly speaking, ANYTHING can be done within the state system, including randomizing the order of presentation of screens within a condition, but this is tricky stuff). Another reason to use multiple conditions rather than a single condition with multiple screens is that there is a limit to how many screens can be presented sequentially within a single condition. You can only have 10 different screens of images (test_0 through test_9) within a single condition, but you can have any number of conditions.

O.K., you might ask, why do you ever need multiple image screens within a single condition? Why don't you just use one condition per screen image? One reason is that screens can be switched in 16 milliseconds, whereas switching between conditions typically requires at least half a second because of various disk operations that must be performed. For some experiments, you may want to switch between certain stimuli rapidly, in which case you should put the stimuli on different screens within the same condition. Another reason is that you may want pairs or groups of stimuli always presented in the same order. Thus, on one condition you may want to have a red bar always followed by a green bar, and on a second condition you may want to have a blue bar followed by a yellow bar. Finally, if there are any behavioral contingencies linked to your conditions, at present these contingencies cannot easily span different conditions (except in some limited circumstances, described later). Consequently, if, for example, you expect a specific behavioral response to a red bar only when it has been preceded by a green bar, these two stimuli should be placed in the appropriate order within the same experimental condition. If you expect a different behavioral response to a red bar preceded by a red bar, you should place these stimuli in the appropriate order in a different condition. Items, incidently, can be reused in many different conditions.

The concept of conditions is also related to how the data are stored. Recall that data are stored by trial and condition, with the data for each trial preceded by a header. Thus, one trial's worth of data in the data file would contain the data for condition one, the next trial's worth of data might contain the data for condition five (depending on the order of presentation), etc.. Within each trial's worth of data, the onset and offset of test_0, test_1, etc is marked. Suppose that a red bar item is presented only on the test_0 screen of condition 12. To locate the neuronal responses to the red bar in the data file, you would find all the trial data for condition 12 (the condition number is given in the header for each trial), and then find the time of occurrence of test_0 in each of the trials for that condition. The spikes that fall within test_0 onset and test_0 offset represent the neuronal response to the bar on that trial.

7. Source Code

7.1. Distribution

The Cortex program includes over 35,000 lines of source code in about 200 source files. The program has been compiled with Microsoft C version 5.1. The source distribution disk can have any of a number of different formats. Typically, the source is in a compressed form and must be "unarchived" by a recent version of the "Arc" program (e.g. arc6.exe), which should be included on the disk. Arc will create directories on the hard disk during the unarchiving process.

The user begins by creating a new directory. For Cortex V4.32, for example, the directory name might be "CORTEX4.32". This is the value used in the present example. Once the directory is created, the archieve file (e.g. CORTEX.ARC) and the archieve program (e.g. ARC6.EXE) are placed in that directory. Then the file is unarchieved with the command:

arc6 cortex

in this example. A listing of the cortex4.32 directory should then look something like this:

Volume in drive C is MS-DOS_5 Serial number is 1905:8841

Directory of c:\cortex4.32\*.*

. <DIR> 6-10-93 10:03a

.. <DIR> 6-10-93 10:03a

BISON <DIR> 6-10-93 10:05a

CORTEX <DIR> 6-10-93 10:05a

CORTEXCC <DIR> 6-10-93 10:05a

DEMOS <DIR> 6-10-93 10:05a

EXE <DIR> 6-10-93 10:05a

GDP <DIR> 6-10-93 10:05a

GRAPHICS <DIR> 6-10-93 10:05a

HISTOGRA <DIR> 6-10-93 10:05a

INCLUDE <DIR> 6-10-93 10:05a

INTERF <DIR> 6-10-93 10:05a

INTERRUP <DIR> 6-10-93 10:05a

LIB <DIR> 6-10-93 10:05a

MDP <DIR> 6-10-93 10:05a

MENUS <DIR> 6-10-93 10:05a

NEWGDP <DIR> 6-10-93 10:05a

NNIOS <DIR> 6-10-93 10:05a

PANTEST <DIR> 6-10-93 10:05a

PLAY <DIR> 6-10-93 10:05a

RTIME <DIR> 6-10-93 10:05a

SCREEN <DIR> 6-10-93 10:05a

TOMCFG <DIR> 6-10-93 10:05a

arc6.exe 65408 1-12-93 12:42p

ctxman2.doc 237749 6-17-93 11:51a

preppy.exe 26653 3-07-93 10:11p

cortexcc.exe 81008 3-07-93 10:30p

cortex.exe 385105 3-07-93 11:02p

872,902 bytes in 31 file(s) 917,504 bytes allocated

151,511,040 bytes free

7.2. Components

Most of the directories hold the source code and .obj files for the Cortex libraries. The \lib directory holds the completed libraries. However, some directories are not library directories:

\include               include files (*.h) for Cortex                
\bison                 parser (internal to cortex) for compiling     
                       state programs                                
\cortex                directory for final linking of cortex         
\cortexcc              source, object, and include files for state   
                       system compiler                               
\demos                 software to demonstrate the operation of      
\exe                   storage site for latest version of            
                       cortex.exe and preppy.exe                     
\tomcfg                configuration and setup example               
\lib                   libraries for final linking of Cortex         

7.3. Compiling libraries

Cortex is built up from quite a few libraries. Within the directory for each library is a "make" file, which has the commands for compiling and creating the library. (See the Micorsoft "Make" utility in the Microsoft C manuals.) Before running the make file you will need to set up your computer to run the make files. The following steps are necessary.

1) Make sure that paths and environmental variables for running Microsoft C 5.1 are set up properly. See the Microsoft manuals. In addition, the library environmental variable (LIB) must point to the cortex \lib directory. In our example setup, the DOS set command will look something like this:

set lib=<standard C5.1 lib list>;\cortex4.32\lib

This is most easily done in a batch file, typically the autoexec.bat file.

2) The Cortex make files assume that disk drive J: will be used for temporary files. You can edit the make files to change the temporary directory to anything you want, including a RAM disk. Alternatively, you can create a J drive on your system. This is done in two steps. First add a DOS lastdrive command to your CONFIG.SYS file:


Then, substitute the J: drive with the DOS subst command. If you have a large RAM drive, the RAM drive will be the best choice. Otherwise, use any directory on a hard drive. For example, if you have a RAM drive as drive E:, make a \temp directory on E:, then associate the \temp directory with the J: drive like this:

subst J: E:\temp

3) Set up the default commands for linking.

set link= /seg:256 /nod /F /PAC /ST:20000

This can be established from the DOS prompt or placed into the AUTOEXEC.BAT file. NOTE: this command may have to be modified if there is a problem with the stack size.

Once the environment is set up a library can be compiled and rebuilt. We will use the menus library to explain this process. The library that we want to update is \cortex4.32\lib\menus.lib. The make file will find this library and modify it, even though we will work from the \cortex4.32\menus directory. In that directory are all the C programs for the menuing system. Assume we have modified one of the menu programs (this can be simulated by simply adding a blank space to one of the comments in the program pmenu2.c). The make file in the menus directory is called makefile. (All the make files are called makefile.) The make file is run with the make command:

make makefile

The make file will do the following steps:

1) copy all the include files (*.h) from the \include directory to the J: drive.

2) copy menus.lib from the \lib directory on the default drive to the J: drive.

3) find out which C programs (or header files) have been changed, and:

a) recompile

b) update menus.lib

for each of them.

4) copy the updated menus.lib over the original one in the \lib directory.

5) delete the updated menus.lib from the J: drive.

6) delete all backup files (*.bak) from the J: drive.

Many of the C programs will compile properly, but give error messages. Most of the libraries give the following errors:

Warning foobar.obj: target does not exist

Warning refoobar.obj: target does not exist

Unfortunately, it is impossible to tell if the error messages or warnings are important. There is no documentation on which messages are to be expected.

7.4. Compiling "main.c"

The main part of the program is in the \cortex directory (e.g. \cortex4.32\cortex\main.c). The same command is used to make the main program as was used to make each library. The command is given from the \cortex directory.

make makefile

You may have to delete main.obj to force the makefile to recompile main. Recompilation of main is not necessary if only the libraries have been modified. Linking, however, is required.

7.5. Linking

Go into the \cortex subdirectory and link together the Cortex system with:

link < cortex

This will run the MicroSoft linker and use the commands in the file CORTEX (no extension) to link all parts of the cortex program. It will create (or update) the program CORTEX.EXE. Move this to the \cortex\bin subdirectory, or whatever subdirectory is in the path (for running Cortex). That directory must have: CORTEX.EXE, CORTEXCC.EXE, and PREPPY.EXE.

A problem can occur with the stack size when link or executing cortex. If cortex will not link due to memory limitations, try using a reduced stack size by reducting the value after "ST:" in the link set command.

set link= /seg:256 /nod /F /PAC /ST:15000

If cortex.exe will not run at all, try increasing the ST value and re-link.

7.6. Adding new C functions to Cortex

While functions can be written into a state file, it is sometimes more advantageous to incorporate the function into Cortex itself. For example, to implement the natural log function, it is easier to just use the built in C log function than to write a complete state file function to calculate logs. Using the built in C function will also run a lot faster.

The first step is to create the new function itself. The function can be added to the file userfn.c. It might be more useful to create a new file for all your new user functions, using userfn.c as a guide. After entering all the appropriate #include, extern, and #define statements (don't ask me which ones are essential, I don't know), a typical function entry is added like this:

/* implementation of c pow() function as a built-in state function */

float CSSpow(CSS_ARGS) { /* name always starts with CSS , arguments are always CSS_ARGS */

/* Grab the arguments of CSS_pow(). They are in the format: type,name,argument# */

CSS_ARG(float,x,0); /* argument #0: type=float, name=x, number=0 (1st) */

CSS_ARG(float,power,1); /* augument #1: type=float, name=power, number=1 (2nd) */

return pow(x,power); /* use the CSS_pow() arguments as the arguements of pow().*/


The second step is to add a function declaration to the statefns.h file. For the example above, the entry looks like this:

float CSSpow(CSS_ARGS)

The third step is to add an entry into the SystemFns[] array in the ufnlst.c file. The entries identify the function name for the state system, and set the parameter types.

SYSTEM_FN SystemFns[] = {


Note the "fff". This cryptic code identifies variable types for the function and its parameters. The first "f" is for the function return type (in this case "f" stands for float). This, and only this, position can also take the value 'v', for void.. The second "f" is for argument #0 (see above), the third "f" is for argument #1, etc. Functions with no arguments take a syntax string of length one, such as "v", NOT "vv". The other types are

c      character                                                     
i      integer                                                       
l      long                                                          
f      float                                                         
C      pointer to character                                          
I      pointer to integer                                            
L      pointer to long                                               
F      pointer to float                                              
0      double pointer to character                                   
1      double pointer to integer                                     
2      double pointer to long                                        
3      double pointer to float                                       
.      variable length argument list                                 
v      void                                                          


void MessageString(int which, pchar message) would be characterized by "viC"

The '.' is used for variable length arglists, such as:

void printf(pchar fstring, ...) characterized by "vC."

Thus, when '.' is the last entry in a syntax string, that arg is optional, and any number of args of any type may follow the last required arg. CSS ensures that the proper number of args are POPed. Only the last entry in a syntax string may be '.'.

After the other steps are complete, the library state.lib must be updated. It is probably best to modify makefile in the \cortexcc directory to include your new source file name, assuming you did not put the function code into userfn.c. If you used userfn.c, then makefile does not need modification. Update state.lib following the instructions in Compiling libraries (7.3. above). Then follow the steps for linking in Linking (7.5. above).

8. Data File Structure

As described earlier in this document, the disk file consists of the sequential data for each trial. Each trial's worth of data is preceded by a header. The structure of the header is given in the file disk.h, which is typically in the \user\include directory. The structure is as follows:

typedef struct {

unsigned short length; /*not currently used*/

COND_ID cond_no; /* the experimental condition numb.*/

unsigned short repeat_no, /*repeat # of entire block structure*/

block_no, /*block no., this condition*/

trial_no; /*trial # for this condition/

unsigned short isi_size, /*size of the following array of times*/

code_size, /*size of the following array of codes*/

eog_size, /*size of foll. array of eog values*/

epp_size; /*size of foll. array of erp values*/

unsigned char kHz_resolution; /*0/1 if 1 msec, 10 if 0.1 msec*/

unsigned char eye_storage_rate /* millisecs between eye samples */

EXPECTED_RESPONSE expected response; /*not currently used*/

RESPONSE response; /*not currently used*/

RESPONSE_ERROR response_error; /*see list of error codes*/


All but two of the above values are short integers. The types of response errors are given as an enumerated type in defs.h (see RESPONSE_ERROR). Following the header comes the time and code arrays, the eye movement array, and erp array. For each event code (such as the occurence of a spike, the onset of a stimulus, etc.,) there is an associated time. The time is measured in milliseconds from the start of the trial, and is given as a unsigned long integer (i.e. 32 bits). The event codes are listed in defs.h as an enumerated type (see CODE).

Note: for repeat #, block #, cond. #, etc, all values are base 0 (e.g. the first trial is trial 0).

repeat_no. The repeat # for the entire block structure. The user sets up how many conditions per block and how many trials per block. When all of the blocks have been run once, the repeat number is incremented. Used by the analysis program Procortex.

trial_no. Trial number for this condition. Potentially could be used by Procortex, but currently is not.

block_no. Conditions can be assigned to blocks. For each block, the user specifies the number of trials and conditions to run before moving on to the next block. This is the block in which this condition is located. A given condition typically appears only in a certain block; thus, this number should typically not change for this condition within the data file. Not currently used by anything.

8.1. Sample data file

Each data item occurs sequentially in the file.

The following data items each occupy two bytes (one integer) in the data file, except for saccade error and timing resolution, which each occupy one byte.


0 length (not used)

1 condition number

0 repeat_no, /*trial no, this condition*/

0 block number /*block no, this condition*/

0 trial_no /*same as repeat no.*/

2560 size of array of time codes (e.g. 2560 bytes, 4 bytes per code)

1280 size of array of event codes (e.g. 1280 bytes, 2 bytes per code)

0 size of array of eog values (2 bytes per value)

0 size of array of evoked potential values (2 bytes per value)

4 milliseconds between eye samples - e.g. 4 = 250 hz (BYTE)

0 time resolution of interupt routine (1 khz) (BYTE)

1 code for expected response (not currently used in data anal.)

2 code for actual response (not currently used in data anal.)

0 code for response error (see list of codes)

The following data items each occupy 4 bytes (long integer)

0 first time code (see associated event codes below)

100 second time code

500 third time code


the following data items each occupy 2 bytes (integer)

18 first event code

24 second event code

40 third event code


Following these data items would be the header for the next trial's worth of data. And so on until end of file.

12. Motion Processing in Cortex

The functions for controlling stimulus motion are described in the documentation for the state system. The general principles behind the use of motion in CORTEX as well as an overview of the operations required of the user are described in this section.

Theory_of_operation. To understand how motion is implemented on the SGT, you must distinguish between the graphic images drawn in the SGT memory and the image that is actually displayed on the CRT monitor (see the User's Guide for the Pepper SGT). The underlying graphic memory is termed the bitmap. The bitmap generally contains far more "pixel" elements than does the CRT display. The minimum bitmap size is 1000 x 1000 pixels (8 bits deep), for a 1 megabyte Pepper board, whereas only 640 x 480 pixels are actually displayed on the screen.

The portion of the bitmap that is displayed on the CRT screen is determined by the window, whose size can range from as large as the CRT display down to 1 pixel in size. Using the Pepper graphics software, the programmer can specify where on the CRT display the window should be placed. Because this location can be changed from one video frame to the next, window movement provides the means to move an image across the screen.

In addition to controlling where on the screen the window is displayed, the programmer can also specify which portion of the bitmap gets mapped to the window. You can imagine the window to be a small viewport into a large graphic space. For example, if the image of a car is drawn in the bitmap, the window might contain just the image of just the wheel or the door. Using the Pepper graphic software, the programmer can specify which portion of the bitmap is mapped to the window. Remapping the window to different portions of the bitmap causes the bitmap image to 'scroll', or move, inside the window, while the location of the window on the CRT display remains in place. Because the portion of the bitmap mapped to the window can be changed from one video frame to the next, this provides a means to show "interior motion" within a window on the CRT display, where the borders of the window do not move on the screen. We will distinguish between these two types of motion on the Pepper SGT using the terms "window motion", where the window moves across the screen, and "interior motion", where the interior of the window scrolls, but the window remains in place. The two "window motion" functions in CORTEX are sweep() and scroll_win(). "Interior motion" functions in CORTEX include pan_win() and movie(), which can be used to display a type of animation.


SWEEP. The sweep function will cause an image to sweep across the screen. Each time the sweep function is executed, the image will move one small increment of the total sweep. With each call, sweep() returns the amount of time remaining (in milliseconds) before the total movement will be finished. Thus, sweep() is typically executed within a loop, testing for 0 msec.

Before starting a sweep, you must execute two other functions. One is init_sweep(), which initializes the sweep parameters. You pass this function the workstation number, speed, direction, and total time of the motion. The details are summarized in the state system function description. The second function you must execute before sweep() is one of the sample or test_on functions to turn the stimulus on. Turning the stimulus on with these functions also causes the subsequent sweep to be synchronized with the video frames. Finally, you execute sweep() in a loop, testing for 0. When the sweep is terminated, you then call the sample or test_on functions, to turn the stimuli off.

Sweep() is meant to be used in situations where you want a bar, or similar stimulus, to sweep across the center of the receptive field from one of a number of different directions. The center of the sweep is always the center of the window location, which is, in turn, determined by the coordinates of the item inside the window. For example, if you specify as item #1 a bar with coordinates (3,3), and then use this item as the sample item for a sweep, the sweep will be centered over (3,3). The starting location for the sweep will be determined by the direction of motion and the length of time you will be sweeping. In this example, if (3,3) is the center of the receptive field, motion will be centered on the field. Alternatively, you could position all items at (0,0), and then give the receptive field center as the reference point. This would also cause the motion to be centered over the field.

SCROLL_WIN. The scroll_win() is similar to sweep, but the motion is_not centered at the item location. Rather, the motion starts at the item location. For example, if the item location is (3,3), the motion will start at (3,3) rather than centered at (3,3). Scroll_win is meant to be used in situations where you change the direction or speed of motion dynamically during the trial.

PAN_WIN. In pan_win, the window is stationary on the screen, but the bitmap appears to move underneath it. Obviously, this requires that your bitmap image be larger than your window, or else you will "run out" of image. This is handled in either of two ways. If you list your window item as "automatic" in the 'bitpannable?' field of the items file (i.e. you put a 1 in this field), then Cortex will replicate your item 3 times (for a total of 4 instances of your item), adjacent to one another in a rectangular grid. The window size is fixed at the size of your original (single) image. Cortex then handles moving the bitmap underneath the window for you (starting automatically at the lower left corner) and jumps the image back to the beginning when the edge of your large composite image reaches the edge of the window. For some purposes, this is sufficient. However, you will see sharp discontinuities between the 4 image replicates, and the point where the image jumps back within the window may not be the one you want. The problem is particularly severe with sine wave gratings.

Alternatively, you can define the size of your window and its starting location relative to your bitmap image yourself. This allows you to set up perfectly smooth motion within the window, with no discontinuities in the image underneath. To establish a window size, just enter a window size for the item in your item file. Set the absolute starting location of the window (in relation to your bitmap image) with the pan_winABS(wkst, x, y) function in your state system. The lower left hand corner of your image is 0,0 and positive x values are to the left. If your image is large enough relative to your window, you may never have to worry about reaching an edge. This is the optimal situation. Otherwise, calculate when you will reach an edge and reset the absolute location of the window accordingly at the correct time. To move the image inside the window, use either the pan_win() function or the pan_winREL(wkst, x, y). See the example state1.pan and state2.pan functions in the demos.

13. Copyright notice

Conditions for Using Cortex

Some portions of Cortex (and associated utility programs, including Procortex, STATS100, and GRAST) are copyrighted (see History of Cortex, below, for a list of its authors). The program is "freeware". That is, the program and its source code are provided free of charge and may be copied freely, provided that this notice is attached. However, its use is subject to certain restrictions, and, thus, it is not strictly speaking in the public domain. The primary restriction is that you must not sell Cortex or its utility programs or incorporate it into any program that is sold. Furthermore, the Cortex program itself includes code from the Bison compiler generator, which is copyrighted by the Free Software Foundation (FSF). The FSF requires as part of the license for Bison that no program incorporating Bison code be sold. The Bison license is included in the source distribution for Cortex, along with complete information on obtaining the source code for Bison.

The source code for Cortex is provided freely on request, and we encourage users to modify and add to it. Depending on the version of the code you receive, one or two object modules required for linking the source code may be commercial products. If you plan to work with the source code, you should purchase any necessary commercial products directly from the appropriate vendor. Hopefully, all commercial object modules will have been eliminated by the time you receive the code, at least for the Cortex program itself. If you improve the source code, add to it, or write any supplemental programs for creating stimuli or analyzing data from Cortex, we ask to be sent copies. If we distribute your code to new users, you will be acknowledged. The program is provided "as is", and we accept no liability whatsoever for damages or loss of any kind caused by its use. Use of the program implies acceptance of all of these conditions.