Inconsistencies with formulas for the standard error of the standardized mean difference of repeated measures experiments.

Abstract

There are inconsistencies between the formulas for the variance of standardized mean difference (SMD) in the Cochrane Handbook for Systematic Reviews and the variance reported in other sources. Instead of the variance appropriate for the SMD of a crossover experiment, the Cochrane Handbook uses the variance appropriate for a pre-test post-test experiment. This means that if there is a non-negligible time period effect, the formula reported by the Handbook will underestimate both the effect size and its variance. In addition, the formula for the standard error of SMD reported in the Cochrane Handbook (in section 23.2.7.2) is inconsistent with the variance derived from the variance of the related t-test. Even if the period effect is negligible, the Cochrane Handbook formula is biased toward underestimates. The difference between the estimates from the two formulas will be small if either the correlation between the repeated measures, or the magnitude of the SMD estimate, is small, or if the sample size is large. However, it can be can be quite substantial in other circumstances.