By Andrew Jeavons
“There’s no art to find the mind’s construction in the face.”
“Macbeth”, Shakespear.
King Duncan of Scotland said these words in the play Macbeth when he learnt of his betrayal by the Thane of Cawdor, one of his noblemen. He then appointed Macbeth to that position, who then murdered him. Words to live by indeed.
We are always seeking ways to look inside the mind by measuring something physical, for King Duncan it was the face, for modern science one method that has become very popular is fMRI. fMRI stands for “functional magnetic resonance imaging”. The central idea is that using very expensive hardware and very fancy software we can see variations in oxygen usage in the brain and hence see a physical change that can be associated with cognitive processing. We want to watch thought processes as they happen, to see the mind at work.
fMRI has therefore become very popular with neuroscientists, and with the growing field of neuromarketing. Identification of regions of the brain that we can associate with specific cognitive processes and hence consumer behavior is one application of fMRI in the research world. Recently a paper published in the Proceedings of the National Academy of Sciences by Elkund et al (link here) has challenged the validity of many fMRI studies by pointing out problems with the way fMRI data is analyzed and the identification of elevated oxygen usage in the brain.
The core of Elkunds et al objections to the current fMRI methods is that the statistical analyses used are flawed and can lead to seeing things that aren’t there. Elkunds et al conclusions throw into question 40,000 fMRI studies carried out over decades. This is a major issue within the fMRI field.
I don’t pretend to understand all of the paper, but I have a close friend who does understand it. He is a Professor of Radiation Oncology, he has a Ph.D. in nuclear physics and is an expert of all forms of MRI, including fMRI. I asked him what he thought of this paper. The answer was as I suspected, having had several conversations with him over the years about fMRI. He found the paper believable and understood the conclusions. He went on to say that he saw fMRI as a “black art” where you adjust the parameters of the analysis until you see something you think is valid. The data from a fMRI scan is noisy, like a radio signal with interference. So researchers use statistics to try to differentiate a “real” signal from one that is just interference. This is where the problems start. Deciding what is interference and what is a valid signal is the “black art”
Some commentators have interpreted Elkund et al as saying there was a “bug” in all the software packages used to analyze fMRI, this is not strictly true although a bug was found in one of the software systems. One of the main problems with the packages used for analysis is that they all made the same assumption about characteristics of the fMRI data, which turned out to be invalid. This assumption caused a lot of errors to be produced in the analyses of FMRI data.
Assumptions can be the source of all sorts of errors. In this case it throws in to question a huge number of fMRI studies that used the software packages investigated in Elkund et al.
Phrenology and fMRI
In Victorian times the “science” of Phrenology was popular. This movement claimed to be able to “read” the bumps on a person’s head and get information on how kind, brave, selfish etc a person was. Of course no one believes in Phrenology now, it is seen as laughable. But what if fMRI is really a black art as my friend implied? Have we re-invented Phrenology except this time we use very complex machines and very complicated analyses? Using measurements of oxygen usage and blood flow in parts of the human brain to impute information about human cognition is very crude. The human brain is the most complex machine we know of. It has at least 10,000 different types of neurons, over 100 billion neurons, many different types of neurotransmitters and a structure we have only just started to understand. Using fMRI to understand the human brain seems like measuring hot spots on a car engine and hoping this will tell you how a fuel injection system works.
The human brain is hyper complex. Simple mono-dimensional measurements such as fMRI can only give extremely limited information about how it works. When you couple this will faulty measurement technology which is looking for 5% variations of noise I think you truly have a black art.
Andrew Jeavons, Mass Cognition
