Partlett, C and Riley, RD (2016) Random effects meta-analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation. Statistics in Medicine, 36 (2). pp. 301-317. ISSN 1097-0258

[thumbnail of R Riley - Random effects meta-analysis - Coverage performance of 95% confidence and prediction intervals following REML estimation.pdf]
R Riley - Random effects meta-analysis - Coverage performance of 95% confidence and prediction intervals following REML estimation.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

Download (1MB) | Preview


A random effects meta-analysis combines the results of several independent studies to summarise the evidence about a particular measure of interest, such as a treatment effect. The approach allows for unexplained between-study heterogeneity in the true treatment effect by incorporating random study effects about the overall mean. The variance of the mean effect estimate is conventionally calculated by assuming that the between study variance is known; however, it has been demonstrated that this approach may be inappropriate, especially when there are few studies. Alternative methods that aim to account for this uncertainty, such as Hartung-Knapp, Sidik-Jonkman and Kenward-Roger, have been proposed and shown to improve upon the conventional approach in some situations. In this paper, we use a simulation study to examine the performance of several of these methods in terms of the coverage of the 95% confidence and prediction intervals derived from a random effects meta-analysis estimated using restricted maximum likelihood. We show that, in terms of the confidence intervals, the Hartung-Knapp correction performs well across a wide-range of scenarios and outperforms other methods when heterogeneity was large and/or study sizes were similar. However, the coverage of the Hartung-Knapp method is slightly too low when the heterogeneity is low (I2 < 30%) and the study sizes are quite varied. In terms of prediction intervals, the conventional approach is only valid when heterogeneity is large (I2 > 30%) and study sizes are similar. In other situations, especially when heterogeneity is small and the study sizes are quite varied, the coverage is far too low and could not be consistently improved by either increasing the number of studies, altering the degrees of freedom or using variance inflation methods. Therefore, researchers should be cautious in deriving 95% prediction intervals following a frequentist random-effects meta-analysis until a more reliable solution is identified

Item Type: Article
Additional Information: This work is made available online in accordance with the publishers policies. This is the author created, accepted version manuscript following peer review and may differ slightly from the final published version.
Uncontrolled Keywords: REML; coverage; meta-analysis; random effects; simulation
Subjects: R Medicine > RA Public aspects of medicine
Divisions: Faculty of Medicine and Health Sciences > Primary Care Health Sciences
Depositing User: Symplectic
Date Deposited: 11 Oct 2016 15:26
Last Modified: 08 Apr 2019 10:19

Actions (login required)

View Item
View Item