Author: Dave Deriso
University of California, San Diego
BIPN 146: Computational Neurobiology
Professor: Dr. Terrence J. Sejnowski
Spring 2010
Before reading, watch this video of a monkey controlling a robot arm with his brain:
http://www.nature.com/nature/journal/v453/n7198/extref/nature06996-s2.wmv
Abstract
The present paper discusses how neurons in the motor cortex can be decoded through a population vector algorithm (PVA) to control a motorized prosthetic arm. The PVA was computed by extracting and summing the preferred directions of each unit within the recorded population weighted by its instantaneous firing rate and then characterized into four dimensions: the 3 degrees arm’s endpoint (x,y,z) and the aperture velocity between gripper fingers (fourth dimension). Although the model was the first to offer four dimensions of control and had a surprisingly short 150ms delay between spike signals and motor movement, it was shown to have only a modest 61% success rate. My assessment of the model is that it was a great first step, but the accuracy indicates that alternative algorithms, such as the bayesian probability model, may improve the design.
Introduction
Central Question
The present experiment addressed separate neuroscientific and mathematical questions. The neuroscientific question asked if activity patterns of populations of neurons in the motor cortex could be decoded efficiently enough to control a prosthetic arm with accuracy and speed at a near-natural, or `embodied,’ level. To this end, the authors constructed a robotic armature that was controlled by the decoded neural signals. The underlying mathematical inquiry tested the efficacy of a population vector algorithm for modeling these neural signals and the movement intentions they represent. The success rate of a monkey’s (macca mulata) feeding attempts (measured by the number of attempts to retrieve food from the end of a delivery device and maneuver it back to the mouth) was used to indicate the efficacy of the system and model.
The original population vector algorithm was proposed in a paper by Georgopoulos, Kettner, and Schwartz (1988) and described a code by which a population of motor cortical neurons could determine uniquely the direction of reaching movements in three-dimensional space. The experimenters recorded from a population consisting of several hundred directionally tuned cells, each of which were found to discharge at the highest rate with movements in its “preferred direction” and at progressively lower rates with movements in directions away from the preferred one. The model made the assumption that the cells in the population could be linearly summed as vectors, and the output would determine the intended direction of movement in space. Specifically, each cell’s preferred direction was multiplied by it’s normalized change in the firing rate, and then summed to form the “neuronal population vector” as described in the equation below:
Where M is a particular movement direction, N is the number of cells in the population, wi(M) is the weighting function computed by the number of spikes a cell will fire in a particular direction M, and Ci is the preferred direction of the ith cell.
Further work by Lukashin, A.V. and Amirikian, B.R. and Georgopoulos, A.P. (1996) showed that neural recordings in the motor cortex of monkeys could control a mechanical actuator that mimicked the movements of an arm. The experimental results indicated that the artificial interface could generate forces in close quantitative agreement with those exerted by trained monkeys, in both the temporal and spatial domains, and may be controlled by the impulse activity of as few as 15 motor cortical cells. These results were the first to show that raw neuronal signals could be used to drive artificial mechanical systems and paved the way for future research in motor based neural prosthetics.
Specific Aims
The model was designed to decode firing patterns in a population of motor cortex neurons into vectors representing intended movement in real-time. Additionally, the PVA was designed to decode the signal into four dimensions of arm control: the 3 degrees of the arm’s endpoint (x,y,z) and the velocity between the gripper fingers (fourth dimension)
Model Assumptions
The population decoding method described in Equation 1 is only effective at predicting intended movement direction when neural firing patterns fit a linear relationship such that each component of the vector can be linearly summed (see Equation 2) and enough neurons are used. Additionally, the model was simplified by assuming that firing rates were modulate equally above and below baseline rate, and the preferred directions of those neurons are uniformly distributed throughout the workspace (Dornhege et al., 2007; Georgopoulos, Kettner, & Schwartz, 1988).
Where R(t) is the neural firing rate; B0 is it’s mean firing rate; Mx(t), My(t), and Mz(t) are the x, y and z components of a unit vector pointing in the direction of movement at time t; and Bx, By, and Bz are regression coefficients that define a vector pointing in the cells preferred direction.
Lastly, a few data pre-processing steps were taken to simplify the between-unit comparisons. To decrease noise, the spike data were smoothed with a finite-impulse response filter algorithm and each neuron was normalized using each unit’s baseline rate and amplitude.
Testable Predictions
The success of the model was measured by the success rate of the monkey’s feeding behavior attempts. This was calculated as a count of how many times the monkey was able to retrieve and eat the presented food divided by the number of attempts made.
Analysis
Findings
The model was able to show that a PVA could be used decode movement intentions in multiple dimensions, while previous models have only tested two. Additionally, it indicated that greater mounts of accuracy could be obtain form this method than were previously found, thereby adding more credibility to the PVA method of neural signal decoding. Finally, the work demonstrated that a real-time and near-natural “embodied” control is possible to achieve with the cortical-prosthesis interface technology.
Strengths and Weaknesses
The strength of the model is that it is inherently simple, which allows the decoding to occur in real-time with accuracy was higher than in previous work.
However, many other studies have shown that neural firing activity has a much more complicated relationship to intended movement than is captured by the population vector algorithm. Neural rates have also been shown to relate to movement speed as well as direction (Moran & Schwartz, 1999) therefore indicating that the PVA may be oversimplified.
A limitation of the PVA, is the assumption that the basis function add linearly. This model cannot represent distributions that are sharper than the component distributions. For instance, a point in between two tuning curves will be misrepresented as being in both places. Additionally, while previous models have typically viewed the relationship between firing rates in MI and various aspects of hand motion (position, direction, speed, velocity, or acceleration) in isolation, Wu et al. (2006) found that decoding performance is improved when the encoding model simultaneously takes into account all these variables, suggesting decoding performance drops when cells are assumed to be conditionally independent.
A second class of models addresses this problem using a generative approach, where an encoding model (e.g., Poisson) is first assumed and a Bayesian decoding model is used to estimate the intended direction (or its distribution), given the state of the population (Doya, Ishii, Pouget, & Rao, 2007). This bayesian approach has been shown to be a more robust and accurate model for population vectors than the PVA (Wu, Gao, Bienenstock, Donoghue, & Black, 2006).
References
Dornhege, G., Mill´n, J., Hinterberger, T., McFarland, D., Sejnowski, T., & Müller, K. (2007). Toward brain-computer interfacing. Cambridge, Massachusetts: The MIT
Doya, K., Ishii, S., Pouget, A., & Rao, R. (2007). Bayesian brain: Probabilistic approaches to neural coding. Cambridge, Massachusetts: MIT Press.
Georgopoulos, A., Kettner, R., & Schwartz, A. (1988). Primate Motor Cortex and Free Arm Movements to Visual Targets in Three-Dimensional Space. II. Coding of the Direction of Movement by a Neuronal Population. Journal of Neuroscience, 8 (8),
Lukashin, A., Amirikian, B., & Georgopoulos, A. (1996). A simulated actuator driven by motor cortical signals. Neuroreport, 7 (15-17), 2597.
Moran, D., & Schwartz, A. (1999). Motor cortical representation of speed and direction during reaching. Journal of Neurophysiology, 82 (5), 2676.
Taylor, D., & Schwartz, A. (2004). Direct cortical control of 3d neuroprosthetic devices.
Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., & Schwartz, A. B. (2008, June). Cortical control of a prosthetic arm for self-feeding. Nature, 453 (7198), 1098–101.
Wu, W., Gao, Y., Bienenstock, E., Donoghue, J. P., & Black, M. J. (2006, January). Bayesian population decoding of motor cortical activity using a Kalman filter. Neural computation, 18 (1), 80–118.
cran at disneyland
[video]
As I write my paper on the Schwartz “Cortical control of a prosthetic arm for self-feeding” article in Nature (2008), IEEE blogs about the newest in Dr. Schwartz series of experiments. This time with a cooler looking arm. Hmm…needs some kind of tattoo
“Cortical activity patterns have been used in the new field of brain–machine interfaces to show how cursors on computer displays can be moved in two- and three- dimensional space. Although the ability to move a cursor can be useful in its own right, this technology could be applied to restore arm and hand function for amputees and paralysed persons.” —
Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., & Schwartz, A. B. (2008). Cortical control of a prosthetic arm for self-feeding. Nature, 453(7198), 1098-101. doi: 10.1038/nature06996.
My feelings exactly.
Does HTML5 Really Beat Flash? No.
the decision to block Flash is less of a technological one and more of a business-minded one. After all, if you could easily visit Hulu.com to stream TV shows and movies, then why would you need to buy them from the iTunes Store?
from proofmathisbeautiful, an awesome blog
Apparatus Section
A home-made hand dynamometer was constructed from masticated stapler parts and an assorted conglomeration of microcontrollers, sensors, and telephone cables packaged within a discarded altoids canister. A blue led indicated that the device was indeed functioning.
Steve Jobs just wrote an essay on why Flash is “unfit” for the iPhone/iPad architecture, illuminating a new type of format war between Apple and Adobe. In his essay, Jobs employs numerous rhetorical strategies that may mislead those outside the developer stratosphere. While I am an avid mac user, I am also a long-time flash developer. Here is my critical analysis of Job’s essay.
Adobe’s Flash products are 100% proprietary. They are only available from Adobe, and Adobe has sole authority as to their future enhancement, pricing, etc. While Adobe’s Flash products are widely available, this does not mean they are open, since they are controlled entirely by Adobe and available only from Adobe. By almost any definition, Flash is a closed system.
Very slick. This is in-fact a misleading statement. Jobs leads the the reader to focus on the fact that Flash products from Adobe are closed-source, while leaving out the fact that there are plenty of alternative open-source tools from the community that create Flash files. Therefore Job’s rhetorical strategy is to focus on Adobe’s products to argue that Flash is a closed-source that is only controlled by Adobe, while leaving out the fact that there are many other open-source ways to make Flash files and that these are created by the thriving open source community.
Apple even creates open standards for the web. For example, Apple began with a small open source project and created WebKit, a complete open-source HTML5 rendering engine that is the heart of the Safari web browser used in all our products.
Jobs is now arguing that Apple is open-source in contrast to Adobe. This is a strikingly obvious rhetorical strategy that is designed to make Apple, a notably proprietary software firm, look like the harbinger of open-source collaboration.
Another Adobe claim is that Apple devices cannot play Flash games. This is true. Fortunately, there are over 50,000 games and entertainment titles on the App Store, and many of them are free. There are more games and entertainment titles available for iPhone, iPod and iPad than for any other platform in the world.
This paragraph is interesting. It appears as though Jobs considers Flash applications to be an unnecessary addition to the overflowing selection within the app store. One perspective is that there is truly no need for additional applications. However, this is analogous to saying that we don’t need more movies at the video store, we already have enough. From this standpoint, its rather short-sighted for Jobs to negate the prospect of future developments or advances that Flash developers could offer and unethical for his to attempt to monopolize the app market by being anti-competitive with the Flash platform. The same logic could be used against app development all together —if there’s already enough software, why continue to develop?
Symantec recently highlighted Flash for having one of the worst security records in 2009. We also know first hand that Flash is the number one reason Macs crash. We have been working with Adobe to fix these problems, but they have persisted for several years now. We don’t want to reduce the reliability and security of our iPhones, iPods and iPads by adding Flash.
PCWorld contacted Symantec about this in 2008. Symantec said: “When we did our testing in the lab, the latest version completely fixes the issue: No crashes, no exploits, no nothing.” Jobs is focusing on a problem in the past and makes no mention of its solution or speed of response. Apple’s software crashes too, it is a rhetorical strategy for Jobs to focus on a single problem as if it were a continuous issue.
Although Flash has recently added support for H.264, the video on almost all Flash websites currently requires an older generation decoder that is not implemented in mobile chips and must be run in software. The difference is striking: on an iPhone, for example, H.264 videos play for up to 10 hours, while videos decoded in software play for less than 5 hours before the battery is fully drained.
Jobs is implying that it is Adobe’s fault that older Flash software is slower. However, he does not offer the possibility that Apple could block older versions of Flash to encourage web developers to update their files. Regardless, he does mention that Flash has indeed updated to the latest standard that does not shorten the battery life, a point that he downplays.
We know from painful experience that letting a third party layer of software come between the platform and the developer ultimately results in sub-standard apps and hinders the enhancement and progress of the platform. If developers grow dependent on third party development libraries and tools, they can only take advantage of platform enhancements if and when the third party chooses to adopt the new features. We cannot be at the mercy of a third party deciding if and when they will make our enhancements available to our developers. […] This becomes even worse if the third party is supplying a cross platform development tool. The third party may not adopt enhancements from one platform unless they are available on all of their supported platforms. Hence developers only have access to the lowest common denominator set of features. Again, we cannot accept an outcome where developers are blocked from using our innovations and enhancements because they are not available on our competitor’s platforms.
Jobs is arguing that there is no way that a Flash developer could make a great application without having access to the entire repertoire of tools offered by Apple. This negates the fact that Adobe is new to the market and may continually improve it’s software. Anyway, isn’t it up to the developer to choose what language to write their own app? It would be better to pick the most comfortable platform that allows the developer to create without having to learn a new language from the “Hello World!” level.
At the end of the day, the essay is very biased and carefully crafted to suit Job’s purpose of degrading Adobe and pushing developers towards Apple’s Xcode development platform at the expense of severely misleading the reader. While Xcode may be the choice with the most options, Adobe’s software opens the doors for the Flash development community to contribute to the growing selection of Apps and will not harm the iPhone or iPad platform standards as Jobs would very much like you to believe. I strongly recommend any reader to look at the Wikipedia entry on Anti-competitive Practices. His entire essay can be summed as “bad rhetoric, and worse ethics.”
As a further point, Apple has a history of forcing developers to develop with Apple’s development platform by preventing all other options. This goes with software, like Adobe’s, and hardware. In a previous post, I have brought to light that Apple requires third-party hardware to have a special “authorization chip” that raises the price and development headache for hardware developers. You can read more about this in a Popular Mechanics article here.
Because there is no simple tutorial for getting the apa package in LaTeX to work (all of the information is loosely spread out across the internet), I decided to make a tutorial for the absolute beginner having some familiarity with the OSX terminal.
What is LaTeX and why is it useful for my research?
LaTeX is a type of markup (like HTML) that will turn your text into a beautifully typeset document. The advantage of using a markup-based text generator over a word processor (like MS Word) is that I don’t have to think about the formatting. The LaTeX engine will autmatically move my text around and into the proper formatting for APA, Chicago, MLS, etc. This way I can focus on the content instead of the formatting.
I’ve sumed up the last 6 hours of my day figuring out how to implement LaTeX with OSX 10.6 for an APA style paper.
1. Download MacTex at http://www.tug.org/mactex/
2. Create a directory for packages
go to the terminal:
mkdir -p ~/Library/texmf/tex/latex/
mkdir -p ~/Library/texmf/bibtex/bst/
3. Download the LaTeX APA packages:
apa package (zip file) from http://tug.ctan.org/tex-archive/macros/latex/contrib/apa/
endfloat package (zip file) from http://tug.ctan.org/tex-archive/macros/latex/contrib/endfloat/
4. Unzip all of the above files and copy each directory to the ~/Library/texmf/tex/latex/ folder created in step 2
5. Download the BibTeX APA packages:
apacite package (zip file) from http://tug.ctan.org/tex-archive/biblio/bibtex/contrib/apacite/
6. Unzip apacite and copy the directory to the~/Library/texmf/bibtex/bst/ folder created in step 2
7. Generates the apacite files necessary for making bibliographies from the apacite.ins file.
Terminal:
cd ~/Library/texmf/bibtex/bst/apacite
tex apacite.ins
8. Go into the ~/Library/texmf/tex/latex/apa folder and copy
apaexample.tex
examplebib.bib
to some folder on your computer (ex. ~/Documents/my_folder/
9. Open the apaexample.tex file with TeXShop by right-clicking the file and selecting “Open With” -> “TexShop”
10. Click on the “Typeset” button (make sure “LaTeX” is visible from the drop-down menu.)
11. A screen should pop up. Ignore it. Go back and select ”BibTeX” from the drop-down menu. Click on the “Typeset” button.
The output pdf should now have 8 pages, but is missing the reference section.
12. Again, close the pop-up and then change the drop-down back to “LaTeX”. Click on the “Typeset” button.The output pdf should now have 9 pages, and includes the reference section.
13. Some sources say to repeat step 12 (re-run LaTeX) 2-3 times for good measure. I don’t know if that makes a difference, but I do it anyway for good measure :)
14. To use the jou feature, you need to download and install the base35 and mdwfonts packages from ctan
mkdir -p ~/Library/texmf/fonts/type1/ #(for base35)
mkdir -p ~/Library/texmf/fonts/tfm/ #(for mdwfonts)
mkdir -p ~/Library/texmf/fonts/vf/ #(for mdwfonts)
Troubleshooting:
If you are ever missing a file and your LaTeX source wont compile, open the log file output and find the missing file. For example, I was missing, “usyr.pfb”. All you need to do is open the terminal and type
kpsewhich —show-path usyr.pfb
Then, find the path that looks like
:/Users/davederiso/Library/texmf/fonts/type1//
Make this missing directory and then search for that missing file on ctan (link), download it and unzip it and copy the file to that directory.
Note: I also use the fantastic TextMate program for generating my LaTeX document, but you can use anything you like to make them (such as textedit). Also, Skim has been recommended by multiple sources as an excellent pdf viewer and can convert some of the LaTeX formats to pdf.
Useful Resource: http://www.let.uu.nl/~Hugo.Quene/personal/tools/latex.hqall.html