last updated 5 September, 1997
The CORTEX (version 5)
User's Manual
by:
Steve Macknik, Andrew Mitz, & Robert Desimone
Lab Neuropsychology, NIMH
TABLE OF CONTENTS
Data acquisition capabilities include (dependent on the available I/O hardware) up to 16 spike inputs (TTL +5v digital inputs), specialized functions for monitoring hand-bar and lever movements (TTL +5v digital inputs), eye-movement monitoring functions (A/D inputs), and keyboard inputs. Reward triggering for alert-animal experiments is accomplished through TTL +5v digital output. The user retains access to the full-array of possible uses for the I/O equipment on-board (such as D/A output, or specialized digital I/O) with easy-to-use C functions. Multiple I/O devices may be accessed by CORTEX, limited only by the number of slots available in the computer.
Graphical capabilities include a device-generalized graphics kernel that allows the user to run the same experiments with either a Number Nine Co. Sgt Pepper board, or the new #9GXi-TC board. Many of the low-level technical aspects of graphical programming are handled directly by CORTEX, thus the graphical programming environment is relatively simple to use.
CORTEX is designed to simplify combined behavioral and neurophysiological experimentation by directly handling trial randomization and simple staircasing. The user retains complete control of the trial staircasing, if desired, and can generate any staircasing paradigm needed. The user can even change the staircasing design on-line, in any way, based on behavioral responses.
Associated with Cortex are a number of utility programs. Procortex is the basic numerical analysis program for the Cortex data file, and it can format the output in a varieties of ways, including Lotus 123 ASCII format, text format, and formats compatible with various statistical programs. It can either be run by itself, through command line options, or it can be run under the control of STATS100. STATS100 is a master control program that can automate the running of Procortex and can create and run batch files for sending the output of Procortex to compatible statistical programs such as SYSTAT or BMDP. STATS100 can also be used to direct the statistical programs to perform any sort of analysis on the data, but to use STATS100 you must already know the ins and outs of the statistical program itself. Cortex files can also be analyzed using PCOFF, which was developed in the Laboratory of Neurophysiology (NIMH). PCOFF is a comprehensive event code sequence search program that supports graphical display of rasters, histograms, reciprocal interval plots, using a number of different schemes to align data on specific events. The program also does analog signal processing (e.g. integration, averaging, etc.) as well as non-parametric and descriptive statistics. RAST62 and GRAST are programs for drawing histograms, rasters, eye position records, etc. off line, using the data collected by Cortex. The output of GRAST and RAST62 can be either low-resolution screen dumps, 300 dpi high resolution plots for laser printers, or CGM-compatible files that can be imported into Lotus Freelance for customizing the figures. Within Freelance or similar software, you can add receptive field diagrams, arrows, charts, etc. to your histograms and rasters. Other associated programs include various utility programs for drawing sinewave gratings, etc.
2) For visual stimulation studies: Pepper SGT-plus graphics card with at least 1 MB of memory, or a #9GXi-TC graphics card with 4megs VRAM and at least 1 meg DRAM, for the subject's visual display. The Pepper SGT will work with any VGA or multiscan monitor and the #9GXi-TC can support multiple resolutions and scanrates. Estimated cost = $1400 for either the Pepper SGT (1 Meg), or the #9GXi-TC (4 megs VRAM/1 meg DRAM), and $300-2000 for a compatible monitor. Many labs use two subject monitors (one for the animal and one for the user....they may be in separate rooms) and use a video splitter such as the VOPEX-2V-H from NTI).
3) Metrabyte Dash-16 for analog and digital input/output ($1000), or Metrabyte PIO24 for digital input/output only ($200). If you don't need to monitor eye movements or deal with analog data, the PIO24 is an economical behavioral input device. A highly recommended Metrabyte Dash-16 clone is the Compuboard CIO-AD16/50K. It includes a DASH-16 equivalent as well as a PIO24 digital i/o on the same board, and it costs somewhat less than the Dash-16. There is also a Compuboard clone of the PIO24 board, for about $60.00, not including cable or interface box.
As to be expected, a memory size that once seemed luxurious eventually became ridiculously too small. After the introduction of the IBM-AT, Stan Schein and I decided to switch from the PDP-11, in part because of the great difference in price and in part because the C compilers for the AT allowed transparent access to 640K of memory, whereas it was difficult to use more than 64K on the PDP-11. A Pepper SGT graphics card from Number Nine Computer was eventually chosen to replace the Jupiter graphics system. Stan, who moved from NIH to Harvard Medical School, hired Thuan Tran, then a computer science student at MIT to port the program after I had naively (and innocently!) assured Stan that the porting would take at most six weeks, once we understood all of the new hardware. Because the old program was inflexible and filled with kludges to get around memory limitations, Thuan did not actually port it. Although he retained a few design aspects of the original PDP-11 program, he completely rewrote and greatly expanded the program over the course of the next couple of years after graduating from MIT, with direction on the design by Stan and myself and support from Stan's grant (R01-EYO6096).
In 1989, Thuan took a full time job with a computer company but continued to work part-time on the state-system interface to Cortex, on contract to NIMH. At about the same time, I hired Tom White at NIMH, and he significantly rewrote most of Cortex again, in order to make it more user friendly, to greatly upgrade its stimulus display capabilities, and to improve the maintainability of its source code. He also took over the development of GRAST, a histogram display program, and Procortex and STATS100, numerical analysis programs, greatly expanding and rewriting them over the course of the past year. Tom integrated STATS100 with SYSTAT, a commercial statistical program, and wrote a data analysis compiler that users can use to write their own analysis routines, without doing any actual programming. Tom is now at Cornell Medical School in an MD/PhD program, but continues to work on Cortex occasionally, out of the goodness (truly) of his heart. I myself work on Cortex code, in my spare milliseconds, and am reasonably familiar with the workings of most of the modules. Rosalyn Merrill (NIMH), Amir Geva, and Jeff Moran also worked on small pieces of code for either Cortex or GRAST at one time during their development. Most recently, Marcello Gattass, the brother of Ricardo Gattass, has taken over the development of GRAST for the time being. Marcello promises a MS Windows version of GRAST sometime in the future. Steve Wise at NIMH has contributed towards some of the considerable development expense of Cortex. Finally, Cortex has benefited from the suggestions (and, unfortunately, bug-finding) of many of its users, particularly Mark Wessinger, Lin Li, Sidney Lehky, John Duncan, Earl Miller, Jennifer Hart, Driss Boussaoud, Josef Rauschecker, Ricardo Gattass, Leonardo Chelazzi, Andy Mitz, Richard Jeo, Carl Olson, Rebecca Hoag, and Peter DeWeerd.
.
1) Address: 0x300 or 0x310
2) address switch settings (from left to right): OFF OFF ON ON ON OFF
3) set for bipolar, 16 analog inputs
4) gain = whatever is convenient
5) analog eye inputs: analog input channels 3 and 4
6) bar contact input: digital input 2
7) solenoid reward output: digital output 1 (puts out brief (< 1 ms) TTL pulse to trigger a solenoid control circuit)
8) spike inputs (2 or more separate spike channels): digital inputs 0 and 1 (for the first two inputs), which are fed by flip flops (provided by the user's electronics shop...see below). Typically one uses the "user 1 and 2" lugs on the Metrabyte interface connector board to access the flip flops.
9) output to clear flip flops: digital output 0
Typical Compuboard CIO A/D-16 settings:
The Compuboard CIO AD-16 can be considered to be a combination of a DASH-16 A/D board and a PIO24 digital I/O board, with the address space of the PIO24 following that of the DASH-16. Because of the PIO24 on board, it is usually necessary to use the Compuboard with a base address of 0x300 for the DASH-16 component rather than 0x310. If the DASH-16 component has an address of 0x300, the PIO24 component can be set with an address of 0x310, which is a safe address. Otherwise, if you set the base address for the DASH-16 component at 0x310, the address space of the PIO24 is automatically set at 0x320, which apparently can conflict with other devices on the AT bus.
Pepper SGT settings:
1) 640x480 resolution (default)
2) set for no emulation
3) put the following line in your config.sys file
device=nnios.sys /m=A000:64;
4) put the nnios.sys file in your root directory. When your computer boots up, it should tell you that it has loaded the Pepper SGT device driver successfully.
#9GXi-TC settings:
1) install the board and run the installation program (it comes with the board)
2) replace the line "c:\tiga2\tigacd.exe" in your autoexec.bat file with "c:\cortex5\util_bin\ticortcd.exe". Where "c:" is the drive where these directories reside.
3) when your computer boots up, it should tell you that it has loaded the TIGA device driver successfully.
Spike Flip Flop Circuitry:
The Dash-16 does not latch digital inputs (for purposes of spike acquisition, neither does the PIO24). This seems to be typical for digital inputs boards for the PC, except possibly for a handshaking line. So if you want to collect neuronal spikes, you must add flip flops between the TTL pulse of your window discriminator (a +5v digital pulse) and the digital input ports of the board. Otherwise, if your TTL pulse is less than 1 msec long, it may be "gone" by the time the Cortex program gets around to examining the port; if longer than 1 msec, Cortex will read more than one spike for each pulse. A circuit diagram for the flipflops can be obtained by contacting Bob Desimone at bobd@cog.nimh.nih.gov. The flipflops are cleared by digital output #0 (courtesy of the software and the flipflop circuit). The spike inputs go to user inputs 0 and 1. Behavioral inputs go to digital inputs 2 and 3 (bar up/down = 2, bar left = 2, bar right =3), which naturally are not latched.
Thalamus:
Although Cortex does not require any interface beyond the screw-terminal box or panel sold by Metrabyte or Compuboard (plus a couple of flip flops for the spikes), Andy Mitz developed a fabulous general-purpose interface called Thalamus. Thalamus is an interface and interconnect box that goes between the computer I/O boards and the outside world. It contains BNC connectors for up to 8 A/D channels and up to 8 spike inputs. It has input latches for all 8 spike inputs. Four of the 8 A/D inputs can have built-in anti-aliasing filters. Thalamus also has "D" type connectors for digital input and output. There are 8 "touch pad" circuits in the Thalamus unit to aid with touch switches or to condition other switches. There is also a one-shot and driver for delivering rewards. Thalamus was developed by the Laboratory of Neurophysiology, NIMH. Drawings for the Thalamus are available through the Research Services Branch of the NIMH.
The three files you must write are items, conditions, and timing (state system) files. For a given experimental session, you can import only a single item and condition file into Cortex, but multiple timing files.
For the items and conditions files, you will note that at the top of each file, there are headings such as ITEM, SIZE, X_POSITION, etc. These headings are special keywords that you should not alter. The first and last character of each keyword define the beginning and end of the data field columns located under the keywords. That is, you enter your specifications in the columns situated under each heading word, taking care not to extend any data beyond the first or last character of the heading word. Because "tabs" can mean any number of spaces, depending on your text editor, Cortex cannot interpret tabs in the files. Thus, DO NOT USE TABS. USE ONLY SPACES between values in the items and conditions files. If you use tabs, previous versions of Cortex would read in your file in a jumbled manner and become very confused. The most recent version at least warns you that tabs are present in your file. If you get a warning about tabs, re-edit your file to remove them and then reimport the file. The state system file is free-format, so tabs are fine. The format of the files is given next.
You must describe each item you want to use in a given experimental session in the ITEMS ascii file, which you import into Cortex at the start of the session. For example, you might want to compare the responses of a neuron to 20 bars of light that vary in color and orientation. In the items ascii file, you would describe the 20 items in the format required by Cortex, including the type of visual stimulus (in this case, bars of light), their height, width, orientation, color, and position relative to some reference point on the display screen (to be described later).
An example items file is shown below:
ITEM TYPE FILLED CENTERX CENTERY BITPAN WIN_WIDE WIN_TALL HEIGHT WIDTH ANGLE INNER OUTER -R- -G- -B- C ------FILENAME-------4 1 1 0.00 0.00 0 1.00 1.00 0.00 0 0 0 x
-3 1 1 0.00 0.00 0 0.30 0.30 0.00 255 255 255 x
-2 1 1 0.00 0.00 0 1.00 1.00 0.00 255 255 255 x
-1 1 1 0.00 0.00 0 1.00 1.00 0.00 255 255 255 x
0 1 0 0.00 0.00 0 3.00 1.00 0.00 50 50 50 x
1 1 0 0.00 0.00 1 3.00 1.00 45.00 100 100 100 x
2 8 0.00 0.00 2 1.8 1.8 230 0 230 x face.vb
3 1 1 0.00 0.00 1 3.00 1.00 45.00 250 250 250 x
Some of the columns you must fill in within the items file are as follows:
ITEM. The item refers to the item number, which you henceforth use in Cortex as the name or identifier of the stimulus you have created. Some items have negative numbers. These are "special" items that are used internally by cortex. Their meaning is as follows:
-4 Background item
-3 Fixation spot item
-2
-1
TYPE. Cortex can handle many different kinds of items, which generally fall into two classes. The first class Cortex "knows" how to create itelf. This class includes bars, circles, ellipses, annuli, and ascii characters. Each of these types has an associated code number, which you enter in this column (see below for a list of type codes). In addition, you must enter the appropriate information about the items in the other relevant columns in the file. For example, bars need a length and width, circles need a radius, etc. The second class is bitmaps and movies, which are items that you have created outside of Cortex and saved in a disk file. These also have a "type" number, which you enter in this column. For these bitmap and movie items, you must also give the name of the disk file that holds them in the FILENAME column. For the format of these bitmapped files, see Appendix A. All items, regardless of type also need an X_POSITION and a Y_POSITION, which you enter in the appropriate column.
Item type codes:
1 bar
2 circle
3 circular annulus (outer and inner diameter should be entered in the inner and outer columns)
4-6 reserved
7 ascii character
8 bitmap (name of file must be in FILENAME column)
9 ellipse
10 annular ellipse
11 movie (name of file must be in FILENAME column)
12 regular polygon
13 ascii string
FILLED. Objects such as bars, circles and annuli can either be filled or just an outline. If you put a '1' in this column the object will be filled, and if you put a '0' it will be an outline.
CENTERX, CENTERY. Each object must have a location on the screen. This sets the x and y axis location, in degrees, from the reference point.
BITPAN. Objects that are bitpannable means that you plan to pan them around inside a stationary window on the screen. A '0' in this field means that the object is not bitpannable, and cortex should treat it like any other object. A '1' in this field means that it is bitpannable, that cortex should utilize the window size you specify (see below), and that cortex should replicate the object 4 times, to give you more object to pan around with. A '2' in this field means the same as 1, but cortex won't replicate it. If you use option 1, there may be sharp transitions in the image at the boundaries of the four replications, but for some purposes, this is o.k.. Also, you may need to set the bitpan field to 2 if you have an extremely large object. The pepper card will not allow you to import a bitmap larger than the display screen (640 x 480), unless you specify a window around it that is smaller than 640 x 480.
WINWIDE, WINTALL. When you give a window length and width (in degrees), and the object is bitpannable, Cortex will create a window around your item. This will allow you to pan a large object around inside a smaller window.
HEIGHT, WIDTH, ANGLE. Bars require a length, width and orientation, in degrees.
INNER, OUTER. Annuli require an inner and outer diamter.
C (Color), -R-, -G-, -B-. For items that Cortex knows how to create, such as bars and circles, you must tell Cortex what color they should be. In the color column you may, if you wish, give a letter representing the color name you want. 'R', for example, stands for red. However, these default colors will probably not have the exact red, green, and blue values you want, so you should really list the actual rgb values you want in the file instead of just giving a color name. To give the actual rgb values, use 'X' as the color name in the color column, and then enter the appropriate values in the -R-, -G-, and -B- columns. These values range from 0 (off) to 255 (maximal brightness) for each red, green and blue gun on the CRT.
For items that are stored as bitmaps, the color you list in the items file is ignored. The colors are determined by the color lookup table, which is described in Appendix A. The name of a color lookup table file can be entered in a menu within Cortex or listed in the conditions file (see below).
FILENAME. For bitmap items, you must give the filename. The name can include a path, if it is not in the current directory.
An example of a conditions file is as follows:
COND# TEST0 TEST1 TEST2 TEST3 TEST4 TEST5 TEST6 TEST7 TEST8 TEST9 BCKGND TIMING TRIAL_TYPE FIX_ID ---COLOR-PALETTE---
1 2 2 1 2 0 -3 lookup.0
2 3 6 7 5 4 1 1 0 -3
5
3 4 4 1 2 0 -3 lookup.0
4 6 7 8 9 1 1 0 -3
5 10 9 3 5 1 1 0 -3
6 12 13 12 13 1 3 0 -3
7 13 12 13 12 1 3 0 -3
COND# The condition number is, as you might expect, the number of each experimental condition. Each line typically defines a condition, but if you need to list more than two items on a given test screen, you can list the additional items on the next line. In the above example, condition number 2 extends across two lines. Depending on the version of cortex you are using, the maximum number of items per test screen is either 4 or 8.
TEST0, TEST1, etc. For each test screen you list the items that you want to appear together. The item numbers refer the items listed in the items file. On a single trial, you can have up to ten successive screens worth of images presented. For example, in condition 2, items 3, 6, and 5 will appear together first on the display screen. Then, items 7 and 5 will appear together, followed by item 4 alone. The timing and sequence of the display screens is given in the state system. You can switch from one display screen to the next in one frame, i.e. 16.6 milliseconds.
BCKGND. For the background, you list an item that you would like to have as the background on each test screen. If the background item is a bar, only the color of the bar will be used. This is the typical situation. If you list an item number that is, for example, a bitmapped image of a forest, the forest will appear as the background on each test screen at the start of the trial.
TIMING. In this column, you list which timing structure, i.e. state system file, you want to run with this condition. The state system files are imported separately, and are refered to by their number.
TRIAL_TYPE. Currently not used.
FIX_ID. In this column, you list which item you want for the fixation stimulus. This is typically item -3, the item number that Cortex uses internally to hold the fixation stimulus. However, you may list any item.
---COLOR-PALETTE---. The color lookup table for bitmapped items can either be entered from within one of Cortex's menus, or listed on a line in the conditions file. This should be the name of a disk file that holds the lookup table. Each condition can have its own lookup table. Lookup tables can also be loaded from the state system, for dynamic changes in color during a trial.
We have neglected the issue of timing -- when and for how long the stimuli remain on the screen. Each experimental condition is associated with a timing file. The same timing file may be used by all conditions or each condition may have its own. The timing file is discussed in the next section.
Within a single experimental session, several timing (or state system) files may be used. From inside Cortex you import each timing file you will use in an experiment, and essentially assign each a sequential number 1, 2, 3, etc. These numbers are used as names for the timing structures. In the conditions file (see above) you specify which timing structure is associated with each condition. For example, you may want to use two different timing structures in an experiment. Each will have its own ascii file, with its own file name, and you will import (from within Cortex) the one timing file as timing structure #1 and the other as timing structure #2. In the conditions file, some conditions must list timing structure #1 as the necessary timing structure and other conditions must list timing structure #2.
Just as with the previous timing files, each state system is written as an ascii file and saved with a file name. After you have written and stored your state systems as disk files, you then import one or more of the files into CORTEX and give them each a number. As was true of the former timing files, each experimental condition (in the Conditions file) can call a state system, by its number.
do_this();
This statement will call the "do_this" function from your state system (this just illustrates the form, there is not actually a "do_this" function in CORTEX). Note that, as in C, every statement must end with a semicolon.
Comments
In the newest version of the state system, you can place comments in your state code to enhance its readibility and maintainability. Comments are delimited by: /* and */, as in C. Thus, a legal comment is:
/* this is a comment */
Functions_that_are_passed_parameters
Some functions must be passed one or more parameters. For example, CORTEX contains a function to initialize a timer, called set_timer. You must pass the function a time, in milliseconds. Parameters are passed within the parenthesis following the function name (note: all function names are followed by a pair of parenthesis, whether or not they are sent a value). Thus, to call the set_timer function with a time of 100 msec, you enter the following statement into your state system:
set_timer(100);
Unlike in C, only integer values can be passed to functions in CORTEX.
state iti(){
}
Within the curly brackets, you place the statements that belong to your iti state. For example, to include the set_timer initialization within your iti state, you would enter:
state iti(){
set_timer(100);
}
Incidently, indenting is optional but it often helps you to read and understand our state system. The curly brackets that delimit the insides of the state are not an option; they must be used and they must be pointed in the correct directions. You can use spaces (or not) liberally in your state system, but you can't use them to break up the name of any function or state, i.e. names cannot have spaces within them. Thus, 'goto_state' is legal but 'goto state' is not because the name of a special keyword has been broken up.
At the end of each state (and often within, as you will see later), you must specify which state you will go to next. For example, you might define two states, an iti state and wait_fixation state. To go from the iti state to the wait_fixation state, you enter the following:
state iti(){
do_some_functions(); /* put in legal functions here */
goto_state wait_fixation();
}
state wait_fixation(){
do_some_functions(); /*put in legal functions here*/
}
The beginning state is called by a special function called main(). Every state system must have a call to main(). Here is a typical example:
main(){
goto_state iti();
}
state iti(){
do_some_functions;
goto_state wait_fixation();
}
state wait_fixation(){
other_functions();
exit();
}
It doesn't matter where you place the main() function in your file as long as it is outside of any other state or function. Neither is the order in which you list the states within your file important, although it would be helpful for readability if they were in logical order (i.e. the same order in which they were executed). Note that main() is not preceded by the word 'state'.
== logical equal
!= logical not equal
The branching or testing keyword is 'if'. To execute a function depending on the value of a variable, you can use 'if' as follows:
if(x == y){
some_function();
}
Or, to implement a branch:
if(x == y){
goto_state iti();
}
if(x != y){
goto_state fixation();
}
In a real state system, 'x' in the above examples would be replaced by a function that returns a value, and y would be replaced by a number. For example, recall that the get_fixation_state function returns a 1 if the subject is fixating and a 0 if not. In the CORTEX state language, as in C, the value returned by a function is inserted into the place where the function name appears in the statement . Thus:
if(get_fixation_state() == 1){ /*if the subject is fixating */
goto_state sample(); /* go to state that turns on stim. */
}
if(get_fixation_state() == 0){ /*if subject is not fixating */
goto_state error(); /* go to an error state */
}
In this example, the program control goes to the sample() state if the subject is fixating, and to the error() state if not.
keyword.
For example:
while(x == y){
some_functions();
}
If the condition following the 'while' is true, then the statements in the brackets are executed and the condition is then retested. If true again, then the statements inside the curly brackets are executed again, and on and on until the condition finally tests false. When the condition is false, program control procedes to the statement following the end of the 'while' end-bracket. As a practical example, you may want to test for whether the subject achieves fixation during a certain time interval. Recall that the function timer_expired() returns a 1 if the timer has run out and a 0 otherwise. Thus:
state wait_fixation{
set_timer(1000);
while(timer_expired() == 0){
if(get_fixation_state() == 1){
goto_state success();
}
}
/* You only get to the next statement if fixation was not achieved */
/* before the timer ran out */
goto_state error();
}
In this example, the timer_expired function is called repeatedly at the start of the while loop. As long as it returns a 0, meaning that the timer has not expired, the statements inside the while loop are executed. When the timer_expired() function returns a 1, meaning the timer has expired, the loop is exited, and the statement following the loop is then executed (in this case, it is a branch to an error state). Note that it is also possible to get out of the loop if the get_fixation() function returns a 1, meaning that the subject has achieved fixation. In this case, the program branches to the 'success' state.
The above example illustrates nested 'while' and 'if' statements, which is
a common construction. Proper indenting of brackets helps you keep track of
the beginning and end of the various states, while loops, if statements, etc.
Actually, it is possible to eliminate one set of brackets in the above
example. If a 'while' or 'if' condition is followed by just a single
statement, you don't need the curly brackets. Thus, the following is a legal
construction:
if(get_fixation_state() == 1)
goto_state success();
state=\path\state.
You import the state system using the 'timing' menu, as in the previous version of CORTEX. For state system #1, choose 'modify', then choose sequence '1', then give the name of your state file. CORTEX will then immediately load in the file, compile, and assemble it. You can add additional state systems by choosing 'add' from the menu. If you get an error message (as opposed to a warning), the compilation failed and you should check your state system code. If, after you have entered your state systems, you want to change one of the timing sequences to a different state file, just choose 'modify', give the sequence number, and then the new file name.
Many state functions require that a special code or value be passed to them. Although these can be passed by value, it is convenient to give some common values special names, which have some meaning. For example, rather than pass the encode() function the value 23, which is the value for turning the sample stimulus on, it is convenient to define, say, TURN_SAMPLE_ON to mean 23, and then use TURN_SAMPLE_ON rather than 23 throughout the remainder of the file (it is a C convention to use capital letters for #defined names - it is desirable, but not essential that you use this convention). To set up this definition, you can add the line:
#define TURN_SAMPLE_ON 23
at the start of your state file.
If you have a large number of #defines at the start of each of your state files, and these #defines are the same for every file (as they almost certainly will be), it is convenient to stick them in one special file, and then attach this definition file to each of your state files. To do this, you put the line:
#include <filename.h>
at the start of each state file, where filename.h is whatever file name you have given your special "header" file. Typically, this is defs.h.
The second sort of functions that require some explanation are a set of behavioral error and success marking functions, such as missing_response(), success(), break_fixation(), etc. These functions do not affect the codes and times in the data file. Rather, they only affect the "header" for each trial's worth of data in the file. Like the codes and times, this header information is used by STATS and RASTHIST for analysing the data. Specifically, it is used for sorting trials, so that you can ask STATS, for example, to only analyze the "correct trials". STATS knows what is a correct trial from the information you place in the header for each trial. The header information is also used by STATS to give you the "behavioral report" of the number of correct trials, or trials with loss of fixation, or no response, or an incorrect response, etc. As with the codes, this information used to be placed in the header for you by CORTEX - now you get to do it yourself.
If you need any of these more advanced features of C, you should write a C function and incorporate it into the CORTEX program itself.
You may be interested in how the state system works. Briefly, the state system 'engine' consists of a preprocessor, compiler and an assembler. The preprocessor and compiler and stand-alone executables that are spawned by CORTEX, but the assembler is part of CORTEX itself. The preprocessor simply handles the #defines, #includes, and comments. Its output is an ascii file named 'preppy.out', which you can examine. The compiler has been written in LEX and YACC (actually, Bison and Flex), with the resulting C code modified to work inside CORTEX. The compiler produces something that looks like assembly language. You can examine its output by looking at the 'state.out' ascii file which is produced every time you import a state system. The output of the assembler is, in part, a table of function pointers. The assembler output is executed by an interpreter during the execution of the trial.
You can actually run the preprocessor and compiler on their own, outside of cortex, by executing them with the state system files on the command line:
preppy state.1 > preppy.out
state preppy.out > state.out
The output files can then be examined with any text editor.
IMPORTANT NOTE TO USERS OF 3.0: CORTEX 3.0 added the graphics kernel before CORTEXCC (CORTEX's new C Compiler) was develloped, and thus used the archaic calling convention. Since the kernel is designed to last CORTEX for years, I had to upgrade the calling conventions to take advantage of FLOATs and LONGs. Thus, the prototypes have been changed so that you can, and are now required to, pass the actual degrees of visual angle, instead of 100* them. It is hoped that few, if any people, will be inconvienced by this.
UPGRADING TIMING FILES FROM CORTEX 2.9 AND BEFORE:
The old state system was such a mess that it had to be completely scrapped. However, the files written for versions of CORTEX prior to 4.0 are easily upgraded:
(1) "goto_state sample();" -> "goto sample;"
(2) "state sample() {" -> "sample: {"
(3) "exit();" -> "return;"
(4) enclose all states within the braces of main() { }, since the old state system was really only one function, with a bunch of "goto"s.
If you use BRIEF, which I highly recommend, the following translate commands will do the above specified conversions:
(0) commands will be specified as Pattern: "????" -> Replacement: "???"
(1) P: "goto_state{*}()" -> R: "goto \0"
(2) P: "<state{*}()" -> R: "\0:" // assumes state at left edge
(3) P: "exit()" -> R: "return"
(4) move "main()" and its body to the top, and put its closing brace after all of the states.
III. USING CORTEXCC (CSS = CORTEX State System)
CSS is a subset of "C". With the help of BISON to do the parsing, the CSS simulator is able to do all of the standard looping operations, function calls, global and local variable declarations (so recursion is possible) of the types {char,int,long,float,pchar...,ppchar...} where "pchar" == "(char *)" and "ppchar" == "(char **)".
Because CSS is an interpreter, there are two types of functions available: those written in "C" and declared as system functions, and those written in CSS. Obviously, the "C" functions run a bit faster, but they require a "C" compiler and linker. Thus, CSS should suffice, but if you have some time critical functions, it is best to port them (fairly directly) over the appropriate "C" file and re-compile CORTEX.
The following are the incompatabilites with "C":
(1) "typedef", "union", "struct" and "enum" are not supported.
(2) Only unidimensional arrays are supported.
(3) Hex and octal numbers are parsed, but numbers with exponents aren't.
(4) In the logical operations && and ||, only those args necessary to determine the answer are evaluated (e.g. (A&&B) does not eval B if A==0, and (C||D) does not eval D if C==1). The only deviation from "C" is that this is implemented via conditional jumps, so that (5||7), (0||5), and (5&&3) all == 5 instead of 1. Therefore, when using these operators, it is best to ask "if (A&&B) stmt;" instead of "if (A&&B == 1) stmt;" since the later may not work, and the former is faster anyway.
(5) Variable length arg lists are only supported for System Fuctions
The following directives are supported:
(1) #include "filename" // but not #include <filename>
(2) #define NAME val // e.g. PI -> 3.14
(3) #A // stringize A's value: #PI -> "3.14"
(4) arg1##arg2 // fuse: "Pi = "##PI -> "Pi = "3.14
(5) "str1" "str2" // concatenate: "Pi = " #PI -> "Pi = 3.14"
(6) #define MACRO(a,b,...) val2
// #define MATH_ALU(class,op,op_name) CSS_##class##_##op_name(void)
// MATH_ALU(char,+,add) -> CSS_char_add(void)
(7) #define lines ended with '\\' append next line to #define value.
(8) #ifdef, #ifndef, #else, #endif // conditional compilation
(9) #elif not supported: replaced with #else #ifdef or #else #ifndef since all directives can be embedded, like #include in included files, nested #ifdef loops, and #define macros which reference other macros.
(1) Data types: (char,int,long,float,pchar,...,ppchar,...} where pchar = (char *) and ppchar = (char **). Data is stored internally in a the union:
union dataEL {
char _char; int _int; long _long; float _float;
pchar _pchar; pint _pint; plong _plong; pfloat _pfloat;
ppchar _ppchar; ppint _ppint; pplong _pplong; ppfloat _ppfloat;
};
Macros are heavily used in the code, as cssmath.c makes evident.
(2) Data scope: (static,extern,auto)
Only "static" requires a reserved word. If variables are declared globally (outside of functions), they are automatically static, which means that they are stored in the data segment. "Static" scoping within functions is fully supported.
Extern vars are special, since this implies access to variables
within CORTEX proper. Within the file "ufnlist.c" is an array
EXTERN_VAR ExternVars[], to which new externs can be easily added
following the format shown.
(3) Pointers:
Address indirection is supported with the following limitations:
(a) Strings are fully supported
(b) Unidimentional arrays, such as x[10] are fully supported.
(c) Assignment by indirection is also fully supported.
(d) Pointer math, like *(x + 10) is NOT supported, and may never be.
(4) Flow control: (if else do while for switch ()?: goto break continue)
All normal C flow statements are supported.
(5) Math:
All C math functions are supported except for (>> >>= << <<=), and those could be easily added to cssmath.c if one wishes. Pre and Post ++/-- are fully supported. Internally, the functions are stored in the arrays:
P_F_dataEl CSSOpAsgnArray[][MAX_MATH_TYPE];
P_F_dataEl CSSpOpAsgnArray[][MAX_MATH_TYPE]; // p = pointers
P_F_VOID CSSMathArray[][MAX_MATH_TYPE];
P_F_VOID CSSpMathArray[][MAX_MATH_TYPE]; // p = pointers
so that all math ops are rapidly accessed by calling the function indicated by the data type and math operation indices.
(6) Cast conversions: (fully supported). CORTEXCC parses all numbers as either LONG or FLOAT. Constants are cast to the proper destination type at compile time. In situations where variables of different types are assigned, or math operations must be done, casts are done at run-time.
(7) Functions: (System,User)
(A) Reserved functions: (main,pre-main,post-main,interrupt) Each of these functions takes no arguments, and returns nothing, so instead of listing a prototype like:
void main(void) { }
these are accessed as
main() { }
CORTEX does not currently use the other functions, although they are correctly coded by CORTEXCC. They are envisioned for use in future flexibility in graphics manipulations, especially if we eventually use a separate computer for graphics display.
(B) System functions: specified in the array SYSTEM_FN SystemFns[]; in ufnlst.c. The third entry is the syntax, so that proper syntax checking can be done, and so that incorrect syntax or number of arguments will generate error messages.
(C) SYNTAX: Values include (cilfCILF0123.). The first entry in the string is the return type. This, and only this, position can also take the value 'v', for void. The following entries indicate the type for each subsequent arg. Functions with no arguments take a syntax string of length one, such as "v", NOT "vv". As to the subsequent args, lower case letter indicate values, uppercase letters mean pointers to that data type, and numbers are double pointers to that data type. For example, the function:
void MessageString(int which, pchar message) -> "viC".
The '.' is used for variable length arglists, such as void printf(pchar fstring, ...) -> "vC.". Thus, when '.' is the last entry in a syntax string, that arg is optional, and any number of args of any type may follow the last required arg. CSS ensures that the proper number of args are POPed. Only the last entry in a syntax string may be '.'.
(D) User Functions: fully supported. Prototypes must be specified before these functions are used. Again, void is not specified except as a return type, so functions which take no arguments are specified like:
int print_hello();
or
float power(float x, float y);
CONDITIONS file. The new CONDITIONS file will select new items from the ITEMS File. Alternatively, your primary interest might be in different experimental paradigms rather than different stimuli. In this case, you could keep the
ITEMS and CONDITIONS files constant, but import different timing files for
different experiments.
A second critical piece of information to check is the name and address of the analog/digital io board in your system.
If you have a Metrabyte_Dash-16, list that as the 'primary device' and give its base and register address (add 9 to the base address to get the register address).
If you have only a Metrabyte_PIO24_or_PIO24, list PIO24 as the primary device, giving its base address (the register address is ignored). In this case, Cortex will only collect digital data.
If you have both a Metrabyte_PIO24/PIO24 and Metrabyte_Dash-16, list the Dash-16 as the primary device and the PIO24 as the secondary device. The Dash-16 will be used for analog (eye movement) and spike input, and the PIO24 will be used for additional digital input/output.
If you have a Compuboard_CIO_AD-16, treat it as a combination of a Metrabyte Dash-16 and PIO24 (i.e. list the Dash-16 and PIO24 as the primary and secondary devices).
There is preliminary and rudimentary support for the Metrabyte_PIO96. If you have a PIO96 (which is the equivalent of 3 PIO24s), list the PIO24 as the primary device and the PIO96 as the secondary device. Give both the same base address.
An example configuration file is as follows (as in C, double slashes indicate
comments):
// DEVICE INFO
Primary_device Compuboard 0x300 0x309 //address = 300 Hex
Secondary_device Pio24 0x310 0x310 //address = 310 Hex
Graphics_specs 8 Never
MONITOR_TYPE vga
TRIAL_CLOCK_RATE 1000
PLAY_CLOCK_RATE 1000
// Locations of the various data windows on the user's screen.
// (window corners are in virtual device coordinates, with a max of 10,000)
STATUS_RECT 3250 8500 10000 10000 3
Eog_display 3250 0 10000 8300
Histogram1 0 1000 3000 4700 // has 5% boundary
Raster_table1 150 300 2850 900
Raster1 150 0 2850 200 // since 5% boundary
Histogram2 0 6200 3000 9900
Raster_table2 150 5500 2850 6100
Raster2 150 5200 2850 5400
//the following are functions that put information on the user's screen
GETScond_num "cond# " 5
GETStrial_num "trial " 5
GETSnothing "" 5
GETScollect_data "GetData" 3
GETSbin_width "bwidth " 6
GETSsaccade "saccade" 6
GETSwindowX "windowX" 6
GETSwindowY "windowY" 6
GETShtg_scale "scale " 6
GETSfile_name "file " 15
GETScond_pct_correct "correct" 8
GETStrial_status "status " 15
TEXT_COLORS 0x07 0x0f 0x70 0x87
// end of configuration file
// below are examples of other configurarion options you might use
Primary_device RANDOM_SPIKE_DEVICE 0 0
Graphics_specs 4 Bitmap
Graphics_specs 8 Never
Graphics_specs 2 Movie
Graphics_specs 1 Both
Primary_device No_device 0 0
Primary_device Pio24 0x310 0x310
Primary_device Compuboard 0x300 0x309
Primary_device Metrabyte 0x310 0x319
Primary_device No_device 0 0
Secondary_device Pio24 0x310 0x310
MONITOR_TYPE vga
MONITOR_TYPE hercules
Run Play Item Condition Timing Disk Analysis
To choose a command from the menu, either type its first character or highlight it with the cursor keys, followed by the "enter" key. Normally, you will want to import the items, conditions, and timing files first. Note: you can temporarily branch out of the program into Dos by hitting the F2 key. This is useful if you need to check a directory or edit a file. Type 'exit' to get back into Cortex at the point you left off.
ITEMS. To import the items file, type 'i' for items, and Cortex will display the items menu. Then choose 'i' for import, and give the file name. If the items file contains bitmaps and you need to import a color lookup table, type 'c' for the CLT command, then give the filename for the table. To actually see one of the items on the graphics screen, type 'd' for Display, and give the item number when prompted. If you simply want to see a listing of the item's properties (item type, length, width, etc.) type 'v' for view and give the item number when prompted.
Reference_point. Another items menu choice is the reference point. If you type 'r' for reference, Cortex will tell you what is the current reference point and will then ask if you want to change it. The three possible reference points are 'fixspot', 'receptive field', and 'other'. The reference point is the location that Cortex defines to be the center of its coordinate system for placing items on the screen. For some studies in fixating monkeys, the fixation spot is the most natural reference point. In you make the fixation point the reference point, then the location of all the stimulus items will necessarily be in retinal coordinates. Alternatively, you might want your items displayed at locations relative to some other point on the screen, such as the center of the receptive field you are studying. If so, choose 'receptive field' or 'other' as the reference. Note that, unlike other items, the fixation point location is always specified in screen coordinates, i.e. its location is not referred to any reference point other than itself.
After you choose a reference point, Cortex will ask you for its x and y screen coordinates, in degrees.
The reference point and its location can also be changed within the "play" routine, which is described a little later.
Conditions. From the main menu you can access the Conditions menu just like you did the Items menu. Like with items, you can ask Cortex to display a condition. One difference from the items display is that when you ask Cortex to display the items listed on a particular test screen of a particular condition, it will show you all of the items that you linked together on the test screen, including the background item. Thus, you can see how the stimulus screens will actually appear during a trial.
Timing. Like the Items and Conditions, you access the timing menu from the main menu. Unlike the case for Items and Conditions, you can import multiple timing files. To import the first timing structure, type 'm' for modify and specify timing structure number 1. After importing the first file, type 'a' for Add, to add more timing files to the list. If you make a mistake, you can go back and reenter a state file, using the modify option. Each condition in the Conditions file must specify the number of a valid timing structure.
Disk. Rather than importing the same files each time you run an experiment, you can save all of the files and other parameters in a single disk save file, which you can import the next time you run the program. Go to the disk menu and type 's' for Save. Normally, you would not 'save' the files and parameters until you have entered all of the information required by Cortex, which is described below. To retrieve the file at a later time, type 'g' for
Get.
BRANCH TO DOS. From any menu you can hit the F1 key, which will put you into DOS while retaining Cortex in memory. From DOS, you can run your favorite editor to change any items, conditions, or timing files you want, or you might execute a Dos command such as a directory listing, or whatever. To return to the program, just type 'exit'. Everything you have entered into the program is saved when you return, as is any data you have collected. If you changed an items, conditions, or timing file while in DOS, Cortex will not know about your changes unless you re-import the file into Cortex.
Play. When you are collecting data, Cortex runs in a very automated fashion and does not allow you to change stimuli interactively. However, it is often desirable in mapping receptive fields to be able to change stimuli and their location "on the fly". In such a informal mode, you generally don't need to be collecting and storing data, since you are changing stimuli in an unpredictable way. Rather, you want to be able to hear the neuronal responses on a loudspeaker while you are playing around with stimuli. After you have decided on the receptive field, you want to be able to use the receptive field center as the reference point for all subsequent stimulus presentations when you collect data. You may even want the play stimulus you have created to become an item that is displayed automatically when data are collected again. All of this is what PLAY is designed for. Play has two modes, "alone" and with "conditions". The "alone" mode is appropriate for experiments in which there are no behavioral contingencies. In this mode, PLAY simply allows you to move and change stimuli constantly, using a mouse.
In the "conditions" mode, PLAY runs whatever experimental conditions you choose, and simultaneously allows you to change and move stimuli around with the mouse. The experimental conditions in this case normally have some behavioral component to them -- e.g. a fixation task. In this mode, PLAY will continue to reward the animal for performing the task correctly and will operate just as it does when you are collecting data. The difference is that you have control over your own mouse-driven stimulus.
When you have found a particular receptive field location that you want to save, PLAY allows you to save the play item parameters, including its location, to any item in your item file. The item you save the play item to can also be the reference item. Reference point items are described in the section on the ITEMS menu, in running Cortex. From within PLAY, you can select 'receptive field' or 'other' to be the reference point, and then save the location of the mapping stimulus to the reference point. Then, when you return to collecting data, the items will be displayed at locations relative to the center of the receptive field that you found. The reference point can also be changed from within the ITEMS menu, described previously.
Run. After importing the necessary files, you go from the main menu towards the menu for running trials and collecting data. The run menu looks as follows:
Histogram Resume Start Parameters Clear Error Display
HISTOGRAM. The most commonly used display is the "single histogram" display that displays the histogram (s) from a single histogram on the screen at one time, plus a display of eye position, if applicable. Single histogram is the default mode, and thus you do not need to enter this menu to set it.
PARAMETERS. The parameters menu has recently been changed. This needs to be rewritten.
Some of the more obscure parameters are as follows:
Blocking. Cortex allows the conditions to be grouped within blocks. Within each block, you can specify how many trials you want to run. This has great utility when you want conditions to be randomly chosen within a small subset of the conditions, but not across the full range of conditions. For example, you might set up one set of conditions in which the subject is to perform visual discriminations based on color and another set of conditions in which the subject is to perform discriminations based on orientation. By placing the color conditions within one block and the orientation conditions within another, you can prevent the orientation conditions from being interleaved with the color ones.
Selection_with/without_replacement. If you choose 'without replacement', then when Cortex picks a condition to run, it deletes that condition from the list of conditions that it will choose from for the next trial. All of the conditions within a block will be presented once (up to the maximum number of trials you specify) before the list of available conditions is regenerated, at which point a given condition may be chosen again. If you choose 'replacement', then Cortex puts the chosen condition back into the available list of conditions, so that condition might even be chosen again for the next trial.
Pick-type. In addition to selection without replacement, there are other blocking options that affect how conditions are chosen. Some of the possible condition-picking options are sequential, random, and "put-back-random". The last option is extremely useful, but it requires some explanation. When training an animal, it is often desirable to repeat a condition that it gets wrong, so that it doesn't decide to just skip all the difficult conditions or make just one kind of response. A common technique is to use "correction trials", which means to simply repeat a condition that is not performed correctly. The problem with conventional correction trials is that an animal who makes an error can learn to simply make the opposite response on the next trial, and it will then always get the next trial correct. A better way to do correction trials is to put the condition performed incorrectly back into the list of conditions to be chosen from randomly on the next trial. This option assumes random selection without replacement for all correctly performed trials. The incorrectly performed condition will keep coming up in the block ever more frequently (but not necessarily the very next trial) until the animal gets it correct.
Pixels/Deg. Because you specify all locations to Cortex in terms of degrees of visual angle, Cortex needs to know how many pixels equals one degree. You can calculate this based on the size and distance of your monitor, in conjunction with the fact that the Pepper SGT displays 640 pixels across the horizontal dimension of the screen. Cortex assumes that your monitor has the correct aspect ratio, so that pixels are square.
Window. For studies in fixating subjects, you need to specifiy the size of the fixation "window", or fixation criteria. If, in the timing file you wrote that the subject must start to fixate by some time point during the trial, then Cortex will use the window size to decide whether fixation has been achieved (and remains achieved). To open the window all the way up, give it a size of 30 degrees or more.
Gain. Cortex cannot actually measure degrees of eye position. It can only measure voltages from the A/D convertors. Cortex assumes that the full scale range of the A/D convertor is equivalent to an eye movement from one side of the display screen to the other. If this is not so, you should adjust the eye position gain either using the gain setting on your eye tracking hardware or the gain setting on the analog input board. Typically, you shift your fixation stimuli 5 degrees or so to the left, right, up, and down, and keep Adjusting the eye coil hardware gain and offset setting until cortex shows that the animal's eye position is on target (hopefully, your monkey knows how to fixate). The size, in degrees, of the "crosshair" on the eye display can be changed by setting the "saccade criterion" parameter in the eye menu, which is a convenient way to measure distance in degrees on the display screen. If for some reason you can't get your gain high enough or whatever, you can also enter an integer gain multiplcation value in the parameters menu, which will be used to multipy all incoming A/D values. The default is 1, which is the recommended setting.
It is important to note that histograms are not stored in the disk data file. Averaged histograms can be recompiled off-line from the data in the arrays, using the GRAST program. GRAST also has the capability to synchronize the histograms on any arbitrary event.
The Pepper SGT has some very nice features for visual neurophysiology. For one, it operates independently of the graphics card that puts up the user's display, so that the user and experimental subject monkey can view different screens simultaneously. Secondly, it can simultaneously display 8 bits of color or grey scale from a 24 bit palette, which is marginally adequate for stimuli such as sine-wave gratings that must be "linearized" in intensity to correct for CRT displays, which are always highly non-linear. Third, you can have up to 4 megabytes of memory on the card (although 1 MB is sufficient for most purposes), which gives you plenty of space to store patterns and also room to "roam" over patterns. Fourth, it runs at 60 hz non-interlace, with 640 x 480 resolution, which, again, is marginally adequate for neurophysiological work in most cortical areas (the 60 hz might be a problem in the LGN). Fifth, the Pepper SGT informs the program of when it begins a vertical sweep of the screen, so that stimulus presentations can be timed to 1 millisecond accuracy (That is, you can know when a stimulus is presented to 1 millisecond accuracy. You have less control than this over the presentation time, because the SGT might be in the middle of a sweep at the precise time you want your stimulus to come on. Stimuli can be switched on only at the start of the sweep.) Finally, it has the Intel 82786 graphics chip which has hardware support for windows that have independent sizes and positions. Thus, for example, you can keep one stimulus stationary on the screen (such as a fixation stimulus) in one window while other stimuli move around in different windows (analogous to sprites), or you can scroll a grating inside one window in one direction while scrolling another grating in another window in a different direction. A major decision for us was to put the stimulus display card in the same machine that collects the data. The alternative would be to use two ATs, one for data and one for graphics, with communication between them. The current scheme seems to work well, and it is cheaper, but it would be fairly easy to tear the graphics out of the program and have it run in a separate machine in the future.
Because it is often necessary in visual neurophysiology to switch rapidly between two or more different images, each of which is large and complex, we decided that the stimuli to be presented on a given "trial" in an experiment would be drawn in different portions of the graphics memory before the trial begins. This allows us to change to a new image within one video frame (16 millisecs). Note, that we do not store all of the images for an experimental_session in graphics memory, because there is no limit on the number or size of stimuli in a session, and, thus, we could not always be sure that they would all fit in. Rather, we just load in the images for a single trial at a time. Optionally, you can instruct Cortex (within the state system) not to unload an image at the end of a trial.
Images must be a binary file, arranged as successive rows of 8 bit pixel values (but only the first 7 bits are read -- the 8th bit is ignored), i.e. the first horizontal row of pixel values, followed by the second row, etc. Preceding the image data are 2 integers that specify the X by Y size of the image. At byte location 12 in the file you should put the X size, in bytes. At location 14 should be the Y size (number of rows). In addition, at byte location 16 should be the number of frames, for movie items only. The image data itself should begin at byte location 18 in the file.
This information is summarized as follows:
Byte-Location in File
0-11 12-13 14-15 16-17 18- to end
blank X-SIZE Y-SIZE NUMBER FRAMES IMAGE DATA
Movie_Items. For animation, such as dynamic motion displays, you may concatenate any number of images in the same disk file, and you can instruct Cortex to display them sequentially (or any order you like) up to 60 frames per second. To allow for simple concatentation of image files, each frame should be organized like the bitmap described above. That is, each frame should have a header, which sets the x and y size. The number of frames in the movie only needs to be set in the first frame's header.
The value that is written into a given pixel location in the graphics card memory does not, in itself, specify what color or intensity should be displayed on the screen. The pixel value becomes associated with a color by virtue of a lookup table in the SGT memory. The color lookup table of the SGT maps each of the 256 possible values a pixel can have (i.e. 8 bits) to a 24 bit color value, 8 bits each for the red, green, and blue guns of the display monitor. Thus, there are 256 rows in the table, each of which requires a red, green, and blue intensity value:
Pixel Arbitrary_Gun_Values
Value Red Green Blue
0 100 150 200
1 50 75 10
2 90 87 100
etc.
Cortex uses the first 128 of the 256 rows in the lookup table for single-color items such as bars and circles, and the upper 128 rows to hold the color values for bitmapped patterns stored in disk files. Each of the bar and circle items (which are 1 bit "deep") in a given condition gets one row in the lookup table. The user enters the red, green, and blue values for each of these items (Note: the actual row assigned to the item in the lookup table is irrelevant to the user, and therefore is assigned by Cortex using a specific algorithm).
For the bitmapped items, which are necessarily 7 bits "deep", you must fill in the upper 128 rows of the lookup table. This table of values must be created off-line, and Cortex must be given the name of the file that holds the values. The file is organized as a series of four successive integers for each of the 128 table locations that will be used. The integers hold the red, green, and blue gun intensities, plus a dummy integer (this is the format required by NNIOS, the Pepper SGT software interface). The user has the option of specifying one "experiment-wide" lookup table that will be applied to all bitmapped items, or of assigning a unique lookup table to each experimental condition (see below). At the appropriate time, the lookup table values are read by Cortex into the SGT memory. These lookup tables can be full-color tables or grey-scale tables, depending on the needs of the user.
For achromatic sinewave gratings, the user will normally "linearize" the output on the display CRT by adjusting the lookup table values so that equal increments in pixel values will produce equal increments of intensity on the CRT screen. It should be noted that in the current version of the program, if the user wants two bitmapped full-color items on the screen at the same time, they must use the same lookup table.
Cortex can interleave trials from a large number of experimental conditions within a single experimental session. Each condition determines what happens on a particular type of trial. For example, in one experimental condition, you may want just a red item presented, and in a different condition you may want just a green item presented. Therefore, you would set up two conditions, each with a single item on the test_0 screen.
Why use two conditions rather than one in this example? Why not just present the red item on the test_0 screen and the green item on the test_1 screen of a single experimental condition? You could do this, but then the order of stimulus presentations would be the same throughout the experiment (i.e. red followed by green). If you want stimulus presentations randomly interleaved, you should assign stimuli to different experimental conditions because Cortex can only randomize the order of presentation of conditions, not the screens within a condition (actually, strictly speaking, ANYTHING can be done within the state system, including randomizing the order of presentation of screens within a condition, but this is tricky stuff). Another reason to use multiple conditions rather than a single condition with multiple screens is that there is a limit to how many screens can be presented sequentially within a single condition. You can only have 10 different screens of images (test_0 through test_9) within a single condition, but you can have any number of conditions.
O.K., you might ask, why do you ever need multiple image screens within a single condition? Why don't you just use one condition per screen image? One reason is that screens can be switched in 16 milliseconds, whereas switching between conditions typically requires at least half a second because of various disk operations that must be performed. For some experiments, you may want to switch between certain stimuli rapidly, in which case you should put the stimuli on different screens within the same condition. Another reason is that you may want pairs or groups of stimuli always presented in the same order. Thus, on one condition you may want to have a red bar always followed by a green bar, and on a second condition you may want to have a blue bar followed by a yellow bar. Finally, if there are any behavioral contingencies linked to your conditions, at present these contingencies cannot easily span different conditions (except in some limited circumstances, described later). Consequently, if, for example, you expect a specific behavioral response to a red bar only when it has been preceded by a green bar, these two stimuli should be placed in the appropriate order within the same experimental condition. If you expect a different behavioral response to a red bar preceded by a red bar, you should place these stimuli in the appropriate order in a different condition. Items, incidently, can be reused in many different conditions.
The concept of conditions is also related to how the data are stored. Recall that data are stored by trial and condition, with the data for each trial preceded by a header. Thus, one trial's worth of data in the data file would contain the data for condition one, the next trial's worth of data might contain the data for condition five (depending on the order of presentation), etc.. Within each trial's worth of data, the onset and offset of test_0, test_1, etc is marked. Suppose that a red bar item is presented only on the test_0 screen of condition 12. To locate the neuronal responses to the red bar in the data file, you would find all the trial data for condition 12 (the condition number is given in the header for each trial), and then find the time of occurrence of test_0 in each of the trials for that condition. The spikes that fall within test_0 onset and test_0 offset represent the neuronal response to the bar on that trial.
The user begins by creating a new directory. For Cortex V4.32, for example, the directory name might be "CORTEX4.32". This is the value used in the present example. Once the directory is created, the archieve file (e.g. CORTEX.ARC) and the archieve program (e.g. ARC6.EXE) are placed in that directory. Then the file is unarchieved with the command:
arc6 cortex
in this example. A listing of the cortex4.32 directory should then look something like this:
Volume in drive C is MS-DOS_5 Serial number is 1905:8841
Directory of c:\cortex4.32\*.*
. <DIR> 6-10-93 10:03a
.. <DIR> 6-10-93 10:03a
BISON <DIR> 6-10-93 10:05a
CORTEX <DIR> 6-10-93 10:05a
CORTEXCC <DIR> 6-10-93 10:05a
DEMOS <DIR> 6-10-93 10:05a
EXE <DIR> 6-10-93 10:05a
GDP <DIR> 6-10-93 10:05a
GRAPHICS <DIR> 6-10-93 10:05a
HISTOGRA <DIR> 6-10-93 10:05a
INCLUDE <DIR> 6-10-93 10:05a
INTERF <DIR> 6-10-93 10:05a
INTERRUP <DIR> 6-10-93 10:05a
LIB <DIR> 6-10-93 10:05a
MDP <DIR> 6-10-93 10:05a
MENUS <DIR> 6-10-93 10:05a
NEWGDP <DIR> 6-10-93 10:05a
NNIOS <DIR> 6-10-93 10:05a
PANTEST <DIR> 6-10-93 10:05a
PLAY <DIR> 6-10-93 10:05a
RTIME <DIR> 6-10-93 10:05a
SCREEN <DIR> 6-10-93 10:05a
TOMCFG <DIR> 6-10-93 10:05a
arc6.exe 65408 1-12-93 12:42p
ctxman2.doc 237749 6-17-93 11:51a
preppy.exe 26653 3-07-93 10:11p
cortexcc.exe 81008 3-07-93 10:30p
cortex.exe 385105 3-07-93 11:02p
872,902 bytes in 31 file(s) 917,504 bytes allocated
151,511,040 bytes free
\include include files (*.h) for Cortex
\bison parser (internal to cortex) for compiling
state programs
\cortex directory for final linking of cortex
\cortexcc source, object, and include files for state
system compiler
\demos software to demonstrate the operation of
cortex
\exe storage site for latest version of
cortex.exe and preppy.exe
\tomcfg configuration and setup example
\lib libraries for final linking of Cortex
1) Make sure that paths and environmental variables for running Microsoft C 5.1 are set up properly. See the Microsoft manuals. In addition, the library environmental variable (LIB) must point to the cortex \lib directory. In our example setup, the DOS set command will look something like this:
set lib=<standard C5.1 lib list>;\cortex4.32\lib
This is most easily done in a batch file, typically the autoexec.bat file.
2) The Cortex make files assume that disk drive J: will be used for temporary files. You can edit the make files to change the temporary directory to anything you want, including a RAM disk. Alternatively, you can create a J drive on your system. This is done in two steps. First add a DOS lastdrive command to your CONFIG.SYS file:
lastdrive=j
Then, substitute the J: drive with the DOS subst command. If you have a large RAM drive, the RAM drive will be the best choice. Otherwise, use any directory on a hard drive. For example, if you have a RAM drive as drive E:, make a \temp directory on E:, then associate the \temp directory with the J: drive like this:
subst J: E:\temp
3) Set up the default commands for linking.
set link= /seg:256 /nod /F /PAC /ST:20000
This can be established from the DOS prompt or placed into the AUTOEXEC.BAT file. NOTE: this command may have to be modified if there is a problem with the stack size.
Once the environment is set up a library can be compiled and rebuilt. We will use the menus library to explain this process. The library that we want to update is \cortex4.32\lib\menus.lib. The make file will find this library and modify it, even though we will work from the \cortex4.32\menus directory. In that directory are all the C programs for the menuing system. Assume we have modified one of the menu programs (this can be simulated by simply adding a blank space to one of the comments in the program pmenu2.c). The make file in the menus directory is called makefile. (All the make files are called makefile.) The make file is run with the make command:
make makefile
The make file will do the following steps:
1) copy all the include files (*.h) from the \include directory to the J: drive.
2) copy menus.lib from the \lib directory on the default drive to the J: drive.
3) find out which C programs (or header files) have been changed, and:
a) recompile
b) update menus.lib
for each of them.
4) copy the updated menus.lib over the original one in the \lib directory.
5) delete the updated menus.lib from the J: drive.
6) delete all backup files (*.bak) from the J: drive.
Many of the C programs will compile properly, but give error messages. Most of the libraries give the following errors:
Warning foobar.obj: target does not exist
Warning refoobar.obj: target does not exist
Unfortunately, it is impossible to tell if the error messages or warnings are important. There is no documentation on which messages are to be expected.
make makefile
You may have to delete main.obj to force the makefile to recompile main. Recompilation of main is not necessary if only the libraries have been modified. Linking, however, is required.
link < cortex
This will run the MicroSoft linker and use the commands in the file CORTEX (no extension) to link all parts of the cortex program. It will create (or update) the program CORTEX.EXE. Move this to the \cortex\bin subdirectory, or whatever subdirectory is in the path (for running Cortex). That directory must have: CORTEX.EXE, CORTEXCC.EXE, and PREPPY.EXE.
A problem can occur with the stack size when link or executing cortex. If cortex will not link due to memory limitations, try using a reduced stack size by reducting the value after "ST:" in the link set command.
set link= /seg:256 /nod /F /PAC /ST:15000
If cortex.exe will not run at all, try increasing the ST value and re-link.
The first step is to create the new function itself. The function can be added to the file userfn.c. It might be more useful to create a new file for all your new user functions, using userfn.c as a guide. After entering all the appropriate #include, extern, and #define statements (don't ask me which ones are essential, I don't know), a typical function entry is added like this:
/* implementation of c pow() function as a built-in state function */
float CSSpow(CSS_ARGS) { /* name always starts with CSS , arguments are always CSS_ARGS */
/* Grab the arguments of CSS_pow(). They are in the format: type,name,argument# */
CSS_ARG(float,x,0); /* argument #0: type=float, name=x, number=0 (1st) */
CSS_ARG(float,power,1); /* augument #1: type=float, name=power, number=1 (2nd) */
return pow(x,power); /* use the CSS_pow() arguments as the arguements of pow().*/
}
The second step is to add a function declaration to the statefns.h file. For the example above, the entry looks like this:
float CSSpow(CSS_ARGS)
The third step is to add an entry into the SystemFns[] array in the ufnlst.c file. The entries identify the function name for the state system, and set the parameter types.
SYSTEM_FN SystemFns[] = {
"pow",0,"fff",CSS_pow,
Note the "fff". This cryptic code identifies variable types for the function and its parameters. The first "f" is for the function return type (in this case "f" stands for float). This, and only this, position can also take the value 'v', for void.. The second "f" is for argument #0 (see above), the third "f" is for argument #1, etc. Functions with no arguments take a syntax string of length one, such as "v", NOT "vv". The other types are
c character i integer l long f float C pointer to character I pointer to integer L pointer to long F pointer to float 0 double pointer to character 1 double pointer to integer 2 double pointer to long 3 double pointer to float . variable length argument list v void
Example:
void MessageString(int which, pchar message) would be characterized by "viC"
The '.' is used for variable length arglists, such as:
void printf(pchar fstring, ...) characterized by "vC."
Thus, when '.' is the last entry in a syntax string, that arg is optional, and any number of args of any type may follow the last required arg. CSS ensures that the proper number of args are POPed. Only the last entry in a syntax string may be '.'.
After the other steps are complete, the library state.lib must be updated. It is probably best to modify makefile in the \cortexcc directory to include your new source file name, assuming you did not put the function code into userfn.c. If you used userfn.c, then makefile does not need modification. Update state.lib following the instructions in Compiling libraries (7.3. above). Then follow the steps for linking in Linking (7.5. above).
typedef struct {
unsigned short length; /*not currently used*/
COND_ID cond_no; /* the experimental condition numb.*/
unsigned short repeat_no, /*repeat # of entire block structure*/
block_no, /*block no., this condition*/
trial_no; /*trial # for this condition/
unsigned short isi_size, /*size of the following array of times*/
code_size, /*size of the following array of codes*/
eog_size, /*size of foll. array of eog values*/
epp_size; /*size of foll. array of erp values*/
unsigned char kHz_resolution; /*0/1 if 1 msec, 10 if 0.1 msec*/
unsigned char eye_storage_rate /* millisecs between eye samples */
EXPECTED_RESPONSE expected response; /*not currently used*/
RESPONSE response; /*not currently used*/
RESPONSE_ERROR response_error; /*see list of error codes*/
} DIR_ENTRY;
All but two of the above values are short integers. The types of response errors are given as an enumerated type in defs.h (see RESPONSE_ERROR). Following the header comes the time and code arrays, the eye movement array, and erp array. For each event code (such as the occurence of a spike, the onset of a stimulus, etc.,) there is an associated time. The time is measured in milliseconds from the start of the trial, and is given as a unsigned long integer (i.e. 32 bits). The event codes are listed in defs.h as an enumerated type (see CODE).
Note: for repeat #, block #, cond. #, etc, all values are base 0 (e.g. the first trial is trial 0).
repeat_no. The repeat # for the entire block structure. The user sets up how many conditions per block and how many trials per block. When all of the blocks have been run once, the repeat number is incremented. Used by the analysis program Procortex.
trial_no. Trial number for this condition. Potentially could be used by Procortex, but currently is not.
block_no. Conditions can be assigned to blocks. For each block, the user specifies the number of trials and conditions to run before moving on to the next block. This is the block in which this condition is located. A given condition typically appears only in a certain block; thus, this number should typically not change for this condition within the data file. Not currently used by anything.
The following data items each occupy two bytes (one integer) in the data file, except for saccade error and timing resolution, which each occupy one byte.
DATA MEANING
0 length (not used)
1 condition number
0 repeat_no, /*trial no, this condition*/
0 block number /*block no, this condition*/
0 trial_no /*same as repeat no.*/
2560 size of array of time codes (e.g. 2560 bytes, 4 bytes per code)
1280 size of array of event codes (e.g. 1280 bytes, 2 bytes per code)
0 size of array of eog values (2 bytes per value)
0 size of array of evoked potential values (2 bytes per value)
4 milliseconds between eye samples - e.g. 4 = 250 hz (BYTE)
0 time resolution of interupt routine (1 khz) (BYTE)
1 code for expected response (not currently used in data anal.)
2 code for actual response (not currently used in data anal.)
0 code for response error (see list of codes)
The following data items each occupy 4 bytes (long integer)
0 first time code (see associated event codes below)
100 second time code
500 third time code
etc.
the following data items each occupy 2 bytes (integer)
18 first event code
24 second event code
40 third event code
etc.
Following these data items would be the header for the next trial's worth of data. And so on until end of file.
The functions for controlling stimulus motion are described in the documentation for the state system. The general principles behind the use of motion in CORTEX as well as an overview of the operations required of the user are described in this section.
Theory_of_operation. To understand how motion is implemented on the SGT, you must distinguish between the graphic images drawn in the SGT memory and the image that is actually displayed on the CRT monitor (see the User's Guide for the Pepper SGT). The underlying graphic memory is termed the bitmap. The bitmap generally contains far more "pixel" elements than does the CRT display. The minimum bitmap size is 1000 x 1000 pixels (8 bits deep), for a 1 megabyte Pepper board, whereas only 640 x 480 pixels are actually displayed on the screen.
The portion of the bitmap that is displayed on the CRT screen is determined by the window, whose size can range from as large as the CRT display down to 1 pixel in size. Using the Pepper graphics software, the programmer can specify where on the CRT display the window should be placed. Because this location can be changed from one video frame to the next, window movement provides the means to move an image across the screen.
In addition to controlling where on the screen the window is displayed, the programmer can also specify which portion of the bitmap gets mapped to the window. You can imagine the window to be a small viewport into a large graphic space. For example, if the image of a car is drawn in the bitmap, the window might contain just the image of just the wheel or the door. Using the Pepper graphic software, the programmer can specify which portion of the bitmap is mapped to the window. Remapping the window to different portions of the bitmap causes the bitmap image to 'scroll', or move, inside the window, while the location of the window on the CRT display remains in place. Because the portion of the bitmap mapped to the window can be changed from one video frame to the next, this provides a means to show "interior motion" within a window on the CRT display, where the borders of the window do not move on the screen. We will distinguish between these two types of motion on the Pepper SGT using the terms "window motion", where the window moves across the screen, and "interior motion", where the interior of the window scrolls, but the window remains in place. The two "window motion" functions in CORTEX are sweep() and scroll_win(). "Interior motion" functions in CORTEX include pan_win() and movie(), which can be used to display a type of animation.
Controlling_motion
SWEEP. The sweep function will cause an image to sweep across the screen. Each time the sweep function is executed, the image will move one small increment of the total sweep. With each call, sweep() returns the amount of time remaining (in milliseconds) before the total movement will be finished. Thus, sweep() is typically executed within a loop, testing for 0 msec.
Before starting a sweep, you must execute two other functions. One is init_sweep(), which initializes the sweep parameters. You pass this function the workstation number, speed, direction, and total time of the motion. The details are summarized in the state system function description. The second function you must execute before sweep() is one of the sample or test_on functions to turn the stimulus on. Turning the stimulus on with these functions also causes the subsequent sweep to be synchronized with the video frames. Finally, you execute sweep() in a loop, testing for 0. When the sweep is terminated, you then call the sample or test_on functions, to turn the stimuli off.
Sweep() is meant to be used in situations where you want a bar, or similar stimulus, to sweep across the center of the receptive field from one of a number of different directions. The center of the sweep is always the center of the window location, which is, in turn, determined by the coordinates of the item inside the window. For example, if you specify as item #1 a bar with coordinates (3,3), and then use this item as the sample item for a sweep, the sweep will be centered over (3,3). The starting location for the sweep will be determined by the direction of motion and the length of time you will be sweeping. In this example, if (3,3) is the center of the receptive field, motion will be centered on the field. Alternatively, you could position all items at (0,0), and then give the receptive field center as the reference point. This would also cause the motion to be centered over the field.
SCROLL_WIN. The scroll_win() is similar to sweep, but the motion is_not centered at the item location. Rather, the motion starts at the item location. For example, if the item location is (3,3), the motion will start at (3,3) rather than centered at (3,3). Scroll_win is meant to be used in situations where you change the direction or speed of motion dynamically during the trial.
PAN_WIN. In pan_win, the window is stationary on the screen, but the bitmap appears to move underneath it. Obviously, this requires that your bitmap image be larger than your window, or else you will "run out" of image. This is handled in either of two ways. If you list your window item as "automatic" in the 'bitpannable?' field of the items file (i.e. you put a 1 in this field), then Cortex will replicate your item 3 times (for a total of 4 instances of your item), adjacent to one another in a rectangular grid. The window size is fixed at the size of your original (single) image. Cortex then handles moving the bitmap underneath the window for you (starting automatically at the lower left corner) and jumps the image back to the beginning when the edge of your large composite image reaches the edge of the window. For some purposes, this is sufficient. However, you will see sharp discontinuities between the 4 image replicates, and the point where the image jumps back within the window may not be the one you want. The problem is particularly severe with sine wave gratings.
Alternatively, you can define the size of your window and its starting location relative to your bitmap image yourself. This allows you to set up perfectly smooth motion within the window, with no discontinuities in the image underneath. To establish a window size, just enter a window size for the item in your item file. Set the absolute starting location of the window (in relation to your bitmap image) with the pan_winABS(wkst, x, y) function in your state system. The lower left hand corner of your image is 0,0 and positive x values are to the left. If your image is large enough relative to your window, you may never have to worry about reaching an edge. This is the optimal situation. Otherwise, calculate when you will reach an edge and reset the absolute location of the window accordingly at the correct time. To move the image inside the window, use either the pan_win() function or the pan_winREL(wkst, x, y). See the example state1.pan and state2.pan functions in the demos.
Conditions for Using Cortex
Some portions of Cortex (and associated utility programs, including Procortex, STATS100, and GRAST) are copyrighted (see History of Cortex, below, for a list of its authors). The program is "freeware". That is, the program and its source code are provided free of charge and may be copied freely, provided that this notice is attached. However, its use is subject to certain restrictions, and, thus, it is not strictly speaking in the public domain. The primary restriction is that you must not sell Cortex or its utility programs or incorporate it into any program that is sold. Furthermore, the Cortex program itself includes code from the Bison compiler generator, which is copyrighted by the Free Software Foundation (FSF). The FSF requires as part of the license for Bison that no program incorporating Bison code be sold. The Bison license is included in the source distribution for Cortex, along with complete information on obtaining the source code for Bison.
The source code for Cortex is provided freely on request, and we encourage users to modify and add to it. Depending on the version of the code you receive, one or two object modules required for linking the source code may be commercial products. If you plan to work with the source code, you should purchase any necessary commercial products directly from the appropriate vendor. Hopefully, all commercial object modules will have been eliminated by the time you receive the code, at least for the Cortex program itself. If you improve the source code, add to it, or write any supplemental programs for creating stimuli or analyzing data from Cortex, we ask to be sent copies. If we distribute your code to new users, you will be acknowledged. The program is provided "as is", and we accept no liability whatsoever for damages or loss of any kind caused by its use. Use of the program implies acceptance of all of these conditions.