Keele Research Repository
Explore the Repository
Snell, KIE, Archer, L, Ensor, J, Bonnett, LJ, Debray, TP, Phillips, B, Collins, GS and Riley, RD (2021) External validation of clinical prediction models: simulation-based sample size calculations were more reliable than rules-of-thumb. Journal of Clinical Epidemiology. ISSN 1878-5921
1-s2.0-S0895435621000482-main.pdf - Accepted Version
Download (977kB) | Preview
Abstract
INTRODUCTION: Sample size 'rules-of-thumb' for external validation of clinical prediction models suggest at least 100 events and 100 non-events. Such blanket guidance is imprecise, and not specific to the model or validation setting. We investigate factors affecting precision of model performance estimates upon external validation, and propose a more tailored sample size approach. METHODS: Simulation of logistic regression prediction models to investigate factors associated with precision of performance estimates. Then, explanation and illustration of a simulation-based approach to calculate the minimum sample size required to precisely estimate a model's calibration, discrimination and clinical utility. RESULTS: Precision is affected by the model's linear predictor (LP) distribution, in addition to number of events and total sample size. Sample sizes of 100 (or even 200) events and non-events can give imprecise estimates, especially for calibration. The simulation-based calculation accounts for the LP distribution and (mis)calibration in the validation sample. Application identifies 2430 required participants (531 events) for external validation of a deep vein thrombosis diagnostic model. CONCLUSION: Where researchers can anticipate the distribution of the model's LP (e.g. based on development sample, or a pilot study), a simulation-based approach for calculating sample size for external validation offers more flexibility and reliability than rules-of-thumb.
Item Type: | Article |
---|---|
Additional Information: | The final version of this article and all relevant information can be found online at; https://www.sciencedirect.com/science/article/pii/S0895435621000482?via%3Dihub |
Subjects: | R Medicine > R Medicine (General) R Medicine > RA Public aspects of medicine R Medicine > RA Public aspects of medicine > RA0421 Public health. Hygiene. Preventive Medicine |
Divisions: | Faculty of Medicine and Health Sciences > School of Primary, Community and Social Care |
Related URLs: | |
Depositing User: | Symplectic |
Date Deposited: | 04 Mar 2021 13:20 |
Last Modified: | 04 Mar 2021 13:20 |
URI: | https://eprints.keele.ac.uk/id/eprint/9209 |