Project Ideas (PRELIMINARY)

The class project is intended to provide in-depth experience into reading the modeling and biological literature, as well as in the construction of models. In order to make clear progress on both fronts by the end of the semester, it is important to be very focused about what one selects for the project. The modeling component should be based on a core of about 3 papers (2 modeling, 1 experimental or 1 modeling, 2 experimental; one may be a paper that we have already read). You may end up reviewing a few additional papers before selecting these three. The model may be a re-implementation of an existing model, but needs involve some novel A number of ideas are included in this document. As with the homework assignments, students may work in pairs. Ideally, these pairs will be cross-departmental and different than for the homework assignments.

Schedule

Models of MI Production of Arm Movements

All of the models proposed by Georgopolous et al., Caminiti et al., Mussa-Ivaldi, Scott and Kalaksa, and Ajemian et al. presume that MI cells fire as some function of extrinsic and/or intrinsic variables. However, none of these models are actually asked to produce movements in the correct direction and magnitude. What are the implications when we instead formulate the problem in terms of producing the correct movement? In other words, we do not make assumptions about the behavior of MI neurons. Instead, we can place constraints as to the inputs (visual representation of target) and the outputs (getting the arm to the target), and apply some optimization technique to adjust the connections between the neural units so as to satisfy the constraints?

One possibility might be to utilize a backpropagation network, although some experimentation will be necessary in order to decide on the right types of non-linear units. It may also be necessary to make adjustments to the error criterion (perhaps adding a term that prefers neurons to be off if possible). Issues/questions/problems that are relevant to this problem include:

Note that a 2DOF muscle geometry model is available in matlab.

Extensions of the Shah Model of MI Recruitment

The Shah model only focuses on the use of extrinsic-like cells in the production of the correct muscle recruitment pattern. We would like to talk about the origin of the intrinsic cells without committing to a particular way in which they are wired. One way to approach this problem is to apply a learning algorithm (e.g., as above) to acquire a mapping between the sensory inputs and a muscle activation pattern. Similar questions apply about the selection of optimality criteria and how to talk about the behavior of the population as a whole. Furthermore, since the Kakei paper provides a nice description of the changes in preferred direction as a function of movement conditions, these experimental results can be compared to the behavior of the model (and it might be possible for us to get our hands on the raw data).

Differences Between Todorov and Shah

The Todorov (2001) paper claims to prove that muscles are optimally recruited in a truncated cosine fashion. However, with the Shah model (detailed in Fagg, Shah and Barto) we demonstrate a clear counterexample to this. What is different about these two approaches that results in this apparent contradiction?

Basal Ganglia and Reinforcement Learning

The Basal Ganglia consist of a set of subcortical regions which are heavily involved in (among other things) motor control and reinforcement-based learning. Although the details of the learning processes differ in subtle (but important) ways from the machine learning notion of reinforcement learning, some of the parallels are quite surprising. For example, neurons in part of the Striatum appear to correlate with the monkey's expectation of future reward. For example, if the monkey performs a task properly, and as a result expects a reward, a larger number of cells in this region will fire vigorously. Downstream from the Striatum, cells the Substantia Nigra pars compacta (SNpc) respond to situations in which the monkey suddenly moves from a state of not expecting a reward to one in which a reward is expected. These cells produce the neurotransmitter dopamine, which (in certain conditions) leads to synaptic changes in the target cells. Target regions of these axons include the Striatum and parts of cortex. In the machine learning terminology, we would classify these dopaminergic signals as being something like the "temporal difference" between two states.

One possible project would be to construct an explicit implementation of the conceptual model presented by Graybiel (1998).

Other possible references:

Development of Orientation-Selective Cells in Visual Cortex

Cells in the primary visual cortex are often selective for oriented edges. Studies have shown that this behavior is not hardwired, but instead is a result of a developmental process that requires the occurance of oriented edges in the visual stream. Models that attempt to explain this process often rely on unsupervised learning techniques in which cells compete with one-another to represent different types of stimuli.

Motor Pattern Generator Circuits

Motor pattern generators are circuits that produce some sequence of movements. Often these are repeating sequences that control activities such as walking or chewing. In some cases these generators act in isolation; in others their behavior is influenced by sensor (and other) inputs. Two possible directions of modeling include:

Hippocampus and Navigation

In rat, the Hippocampus is heavily involved in the formation of spatial maps. Many Hippocampal cells fire in response to the rat being in a specific location, with the activation level dropping off slowly as the rat moves away from the central "place" encoded by the cell. By combining the current activity of many cells, it is possible to estimate the rat's position. These cells can be activated by a variety of sources, including visual cues, auditory cues, proprioceptive/tactile cues (position body sense and touch sense), and even motor efference copy (a copy of the motor signals that are currently being sent to the muscles). Thus, the representation appears to be truely a "cognitive map" that is independent of the sensory information that was used to estimate the rat's current position.

How are these representations constructed? How are they updated with new inputs? And how are they learned in the first place?

Sliding Threshold Theory for Hebbian Learning (BCM Theory)

The Hebbian learning rule (if two neurons tend to fire together, then increase the connection strength between them) has long been seen as a possible mechanism for learning in the brain and has seen some use in the artificial neural network (ANN) community. In order to build an implementation of this learning rule, one must answer the question of how to keep the connection strengths from growing in an unbounded fashion. Typical approaches range from allowing a single weight to grow up to a fixed level, or normalizing the set of weights (e.g., by keeping the sum of the weights at 1).

A more biological approach can be found in Sliding Threshold or BCM Theory. In this approach high, short-term correlations between the postsynaptic and presynaptic cells leads to an increase in connection strength, whereas low (but still positive) correlations result in decreases in strength. Furthermore, the dividing point between this Hebbian and anti-Hebbian behavior is allowed to slide. Specifically, when the average, long-term activity of the postsynaptic grows above a certain level, the threshold increases (thus requiring higher levels of correlation in order to induce Hebbian learning). The opposite happens when the average activity drops below a critical level. Thus, the rule is structured such that the postsynaptic cell attempts to achieve a set of connections that lead to the right level of activity (sort of a Goldie Locks kind of a story).

How might these types of mechanisms be implemented biologically and what might some of the computational implications be?

fagg@cs.umass.edu

Last modified: Tue Oct 23 13:05:54 2001