Snell, KI, Hua, H, Debray, TP, Ensor, J, Look, MP, Moons, KG and Riley, RD (2015) Multivariate meta-analysis of individual participant data helped externally validate the performance and implementation of a prediction model. Journal of Clinical Epidemiology, 69. pp. 40-50. ISSN 1878-5921

[thumbnail of K Snell - Multivariate meta-analysis of individual participant data....pdf]
K Snell - Multivariate meta-analysis of individual participant data....pdf - Published Version
Available under License Creative Commons Attribution.

Download (888kB) | Preview


OBJECTIVES: Our aim was to improve meta-analysis methods for summarizing a prediction model's performance when individual participant data are available from multiple studies for external validation. STUDY DESIGN AND SETTING: We suggest multivariate meta-analysis for jointly synthesizing calibration and discrimination performance, while accounting for their correlation. The approach estimates a prediction model's average performance, the heterogeneity in performance across populations, and the probability of "good" performance in new populations. This allows different implementation strategies (e.g., recalibration) to be compared. Application is made to a diagnostic model for deep vein thrombosis (DVT) and a prognostic model for breast cancer mortality. RESULTS: In both examples, multivariate meta-analysis reveals that calibration performance is excellent on average but highly heterogeneous across populations unless the model's intercept (baseline hazard) is recalibrated. For the cancer model, the probability of "good" performance (defined by C statistic ≥0.7 and calibration slope between 0.9 and 1.1) in a new population was 0.67 with recalibration but 0.22 without recalibration. For the DVT model, even with recalibration, there was only a 0.03 probability of "good" performance. CONCLUSION: Multivariate meta-analysis can be used to externally validate a prediction model's calibration and discrimination performance across multiple populations and to evaluate different implementation strategies.

Item Type: Article
Uncontrolled Keywords: Risk prediction; Prognostic model; Individual participant data (IPD); Multivariate meta-analysis; External validation; Calibration; Discrimination; Heterogeneity; Model comparison
Subjects: R Medicine > RA Public aspects of medicine
Divisions: Faculty of Medicine and Health Sciences > Primary Care Health Sciences
Related URLs:
Depositing User: Symplectic
Date Deposited: 13 Aug 2015 14:05
Last Modified: 13 Aug 2018 10:27

Actions (login required)

View Item
View Item