Journal Club: Is statistical significance always biological significance?

Edit: The journal club presenter stated that the moths lay ~6 to 8 eggs, so perhaps the 1 egg difference mentioned below is meaningful.

This weeks journal club article: Feeding-induced rearrangement of green leaf volatiles reduces moth oviposition from Allmann et al. (2013).

Today marks the last green leaf volatile paper for the semester.

This was the first time I've read a paper from the eLIFE journal. I think I liked the journal's layout. eLIFE has made a bit of a splash, and I think it's one of many that will gain popularity as the open access movement continues to gain momentum. I expect that the future of scientific literature will be more of an open-access, crowd-sourced effort where peer review is done in the comments section of a publication by hundreds of scientists; search engines will allow relevant papers to be searched for, and crowd review efforts will determine what is accurate and important and let the garbage filter to the bottom.

Will this be better than what we have now? I have no idea. I just don't believe that the traditional publication system is capable of disseminating the tremendous volume of information that the world's research institutions can now generate.

As for the paper, the authors reported many statistically significant differences for the differential production of the green leaf volatile (GLV) isoforms in response to the M. sexta oral secretions (OS), and they reported that a behavioral difference in oviposition occurred because of reception of one of the particular GLV isoforms. While I don't doubt the differences they found were statistically significant, I'm not convinced that they are actually of any biological significance.

In their field trials, the same experiments were not statistically significant in the first trial, but they were in the second (although the visual representation of the data aren't very convincing that a difference exists for the second trial).

The calcium activity in the antennal lobe data were more convincing that there is a difference in perception of the two isoforms, although it doesn't demonstrate that a difference in perception translates to a behavioral difference.

Finally, I think Figure 7 was supposed to demonstrate the key point of the paper: behavioral changes based on differential perception of the two isoforms. However, while they did again see significant differences, it looks like there was only ever an average difference of one egg oviposited between the two treatments. I think it's potentially misleading that they're testing that $\mu_1 - \mu_2 \neq 0$ rather than testing $\mu_1 \neq \mu_2$ since, as reported, I have no clue how many eggs these things lay. Is it a difference between 100 and 101? or 2 and 3? As such, I'm not very convinced that this paper has shown anything new.

 My comment:
"I thought this paper's hypothesis was fun, but I didn't find the data very convincing. If I'm reading it correctly, Figure 7 reports that there is an average difference of just one egg between the two isoform treatments. I have no idea how many eggs these moths lay, and I'm not convinced that the one egg difference is demonstrative of a behavioral difference as a result of differential interpretation of the two isoforms."


Popular Posts