Ensor, J, Snell, KIE, Debray, TPA, Lambert, PC, Look, MP, Mamas, MA, Moons, KGM and Riley, RD (2021) Individual participant data meta-analysis for external validation, recalibration, and updating of a flexible parametric prognostic model. Statistics in Medicine. ISSN 0277-6715

[thumbnail of Individual participant data meta-analysis for external validation....pdf]
Individual participant data meta-analysis for external validation....pdf - Published Version
Available under License Creative Commons Attribution.

Download (2MB) | Preview


Individual participant data (IPD) from multiple sources allows external validation of a prognostic model across multiple populations. Often this reveals poor calibration, potentially causing poor predictive performance in some populations. However, rather than discarding the model outright, it may be possible to modify the model to improve performance using recalibration techniques. We use IPD meta-analysis to identify the simplest method to achieve good model performance. We examine four options for recalibrating an existing time-to-event model across multiple populations: (i) shifting the baseline hazard by a constant, (ii) re-estimating the shape of the baseline hazard, (iii) adjusting the prognostic index as a whole, and (iv) adjusting individual predictor effects. For each strategy, IPD meta-analysis examines (heterogeneity in) model performance across populations. Additionally, the probability of achieving good performance in a new population can be calculated allowing ranking of recalibration methods. In an applied example, IPD meta-analysis reveals that the existing model had poor calibration in some populations, and large heterogeneity across populations. However, re-estimation of the intercept substantially improved the expected calibration in new populations, and reduced between-population heterogeneity. Comparing recalibration strategies showed that re-estimating both the magnitude and shape of the baseline hazard gave the highest predicted probability of good performance in a new population. In conclusion, IPD meta-analysis allows a prognostic model to be externally validated in multiple settings, and enables recalibration strategies to be compared and ranked to decide on the least aggressive recalibration strategy to achieve acceptable external model performance without discarding existing model information.

Item Type: Article
Additional Information: © 2021 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
Uncontrolled Keywords: external validation; IPD meta-analysis; model recalibration; model updating; time-to-event models
Subjects: R Medicine > R Medicine (General)
R Medicine > RA Public aspects of medicine
Divisions: Faculty of Medicine and Health Sciences > Primary Care Health Sciences
Related URLs:
Depositing User: Symplectic
Date Deposited: 07 Apr 2021 14:48
Last Modified: 07 Apr 2021 14:48
URI: https://eprints.keele.ac.uk/id/eprint/9325

Actions (login required)

View Item
View Item