Wednesday, September 23, 2009

fMRI is a useful tool - you just have to use it right!

Some dudes at Dartmouth put a dead fish in an fMRI scanner, "instructed" it in a task, and (using crappy statistical methods) found a couple significant voxels in the fish's brain. This is used to demonstrate the importance of using good statistical methods in analyzing fMRI data so that you minimize the false positives. (There's a good summary over at Neuroskeptic.)

But for some reason, my friends who've forwarded me the Wired article about it are all like, "SEE- just like I've been telling you, imaging studies are silly and untrustworthy."

Grrrr.

See, here's the big thing that the dead-fish peeps point out. When you're making statistical comparisons, you shoot for a certain confidence level. Let's say we're shooting for 95% confidence. That means, even if we do everything completely right, about 5% of the time we're going to get wacky results - completely by chance.

In an fMRI, you're making thousands of statistical comparisons, one for each voxel. So even if you are shooting for 99.999% confidence, there's enough voxels that some of them are going to, completely by chance, show weird and unlikely things. Like "activity" in a dead fish's brain.

As the Wired article says:
Bennett’s point is that a suite of methods known as multiple comparisons correction can allow researchers to maintain most of their statistical power while keeping the danger of false positives at bay.
But people, even smart people, even psychologists, look at the fishy fMRI and go, "OH WOW FMRI = USELESS LOLOL U STOOPID IMAGING PPL".

Ugh.

1 comment:

  1. I saw that, and reading your blog entry affirms my news gathering habits (i.e. not Wired)...

    ReplyDelete