On inferring causation from correlation

In reading research, a question of enormous practical (and theoretical) importance is: Why do some children of adequate language skills, intelligence and educational opportunities lag behind their peers in their reading ability, or more simply put: What causes reading problems? This question has been tackled for decades, and the answer has proved to be incredibly complex. (There is still no agreed-upon answer to date.)

Experimentally, finding a causal influence of a behavioural outcome is somewhat tricky. Paraphrasing my undergraduate statistics textbook: there are three points that you need to show before claiming causality. (1) There is a correlation between the outcome measure (e.g., reading ability) and performance on the task which is proposed to cause the variability therein (say, phonological awareness). (2) The causal influence precedes the skill that it’s supposed to test (e.g., phonological awareness at an earlier time point is associated with reading at a later time point). (3) Experimentally manipulating the causing variable should affect performance on the outcome measure (e.g., children become better readers if you train them on a phonological awareness task).

There are two statistical procedures which are commonly used, in reading research, to show a causal relationship, even though they completely ignore Point (3). An experimental manipulation is essential for making a causal claim: both Points (1) and (2) are susceptible to the alternative explanation that a third factor influences both measures. For example, phonological awareness, even if measured before the onset of reading instruction, may be linked to reading ability in Grade 3, but both of them may be caused by vocabulary knowledge, parental involvement in their children’s education, the child’s intelligence, or statistical learning ability, just to name a few possibilities.

Most researchers know that correlation ≠ causation, but many seem to succumb to the temptation of inferring causation from structural equation models (SEMs). Paraphrasing my undergraduate statistics lecturer: SEM is a way of rearranging correlations in such a way that makes it look like you can infer causality. Here, the outcome measure and predictors are represented as boxes, and the unique variance of the particular link, obtained by a regression analysis, is written next to each arrow going from a predictor to the outcome measure. Even if a predictor is measured at an earlier time than the outcome measure (thus showing precedence, as per Point 2), this fails to show a causal relationship, as a third, unmeasured factor could be causing both.

Having just returned from a selective summer school on literacy, I have counted a total of four statements inferring a causal relationship from SEMs during this meeting, one by a prominent professor. They are in good company. Just to pick one example, a recent paper has used SEMs to infer causation (Hulme, Bowyer-Crane, Carroll, Duff, & Snowling, 2012)¹.

While I’m at it, there is another methodological method that has been used to infer causation even though it can’t, namely reading age matched designs. The logic is as follows: if you compare poor readers to good readers, who are matched on age, on any task (say, phonological awareness), you can expect poor readers to perform worse than good readers. This could be because being skilled at this task facilitates learning to read, or performance on this task could be a result of more reading exposure among the good readers (because good readers tend to read in their free time, while poor readers don’t). In a reading age matched design, one compares a group of poor readers, given their age, to a group of younger readers, who are average or good given their age, but their absolute reading ability is equivalent to that of the poor readers. If poor readers perform worse on phonological awareness tasks than their younger controls, this suggests that the deficit in phonological awareness is not a result of a lack of reading exposure.

There are theoretical problems in matching children for their absolute reading ability, because older poor readers and younger average-good readers are unlikely to have identical performance on different aspects of reading (see Jackson & Coltheart, 2001): the control group could vary widely in their age and cognitive skill profiles, depending whether the task to match them measures their nonword reading accuracy, their word reading fluency, or text comprehension. Even if it was theoretically possible to match poor readers to younger controls in terms of their reading ability, the caveats from SEMs still apply: it is possible that poor phonological awareness and poor reading skills are both caused by a third underlying factor. Although I know of no peer-reviewed paper that explicitly makes a causal claim based on the reading age matched design, I have heard such claims at conference talks, and causality is often implied in published papers, without explicitly stating the alternative explanation. 

The TL;DR summary of this post is very simple: It is never OK to infer causality from correlations.

References

Hulme, C., Bowyer-Crane, C., Carroll, J. M., Duff, F. J., & Snowling, M. J. (2012). The Causal Role of Phoneme Awareness and Letter-Sound Knowledge in Learning to Read: Combining Intervention Studies With Mediation Analyses. Psychological Science, 23(6), 572-577. doi:10.1177/0956797611435921

Jackson, N., & Coltheart, M. (2001). Routes to reading success and failure: Toward an integrated cognitive psychology of atypical reading. New York, NY: Psychology Press.

Footnote

[1] To be fair, this study also has a training component. Whether the paper makes a convincing claim for a causal relationship is a different question, but either way someone who only has a quick read of the title and abstract may get the impression that SEMs are a tool for assessing causality.