Friday, August 28, 2015

On inferring causation from correlation

In reading research, a question of enormous practical (and theoretical) importance is: Why do some children of adequate language skills, intelligence and educational opportunities lag behind their peers in their reading ability, or more simply put: What causes reading problems? This question has been tackled for decades, and the answer has proved to be incredibly complex. (There is still no agreed-upon answer to date.)

Experimentally, finding a causal influence of a behavioural outcome is somewhat tricky. Paraphrasing my undergraduate statistics textbook: there are three points that you need to show before claiming causality. (1) There is a correlation between the outcome measure (e.g., reading ability) and performance on the task which is proposed to cause the variability therein (say, phonological awareness). (2) The causal influence precedes the skill that it’s supposed to test (e.g., phonological awareness at an earlier time point is associated with reading at a later time point). (3) Experimentally manipulating the causing variable should affect performance on the outcome measure (e.g., children become better readers if you train them on a phonological awareness task).

There are two statistical procedures which are commonly used, in reading research, to show a causal relationship, even though they completely ignore Point (3). An experimental manipulation is essential for making a causal claim: both Points (1) and (2) are susceptible to the alternative explanation that a third factor influences both measures. For example, phonological awareness, even if measured before the onset of reading instruction, may be linked to reading ability in Grade 3, but both of them may be caused by vocabulary knowledge, parental involvement in their children’s education, the child’s intelligence, or statistical learning ability, just to name a few possibilities.

Most researchers know that correlation ≠ causation, but many seem to succumb to the temptation of inferring causation from structural equation models (SEMs). Paraphrasing my undergraduate statistics lecturer: SEM is a way of rearranging correlations in such a way that makes it look like you can infer causality. Here, the outcome measure and predictors are represented as boxes, and the unique variance of the particular link, obtained by a regression analysis, is written next to each arrow going from a predictor to the outcome measure. Even if a predictor is measured at an earlier time than the outcome measure (thus showing precedence, as per Point 2), this fails to show a causal relationship, as a third, unmeasured factor could be causing both.

Having just returned from a selective summer school on literacy, I have counted a total of four statements inferring a causal relationship from SEMs during this meeting, one by a prominent professor. They are in good company. Just to pick one example, a recent paper has used SEMs to infer causation (Hulme, Bowyer-Crane, Carroll, Duff, & Snowling, 2012)1.