Skip to main content

Research Repository

Advanced Search

Minimum sample size for external validation of a clinical prediction model with a continuous outcome

Riley, Richard D.; Debray, Thomas P.A.; Collins, Gary S.; Archer, Lucinda; Ensor, Joie; van Smeden, Maarten; Snell, Kym I.E.

Minimum sample size for external validation of a clinical prediction model with a continuous outcome Thumbnail


Authors

Richard D. Riley

Thomas P.A. Debray

Gary S. Collins

Lucinda Archer

Joie Ensor

Maarten van Smeden

Kym I.E. Snell



Abstract

Clinical prediction models provide individualised outcome predictions to inform patient counselling and clinical decision making. External validation is the process of examining a prediction model’s performance in data independent to that used for model development. Current external validation studies often suffer from small sample sizes, and subsequently imprecise estimates of a model’s predictive performance. To address this, we propose how to determine the minimum sample size needed for external validation of a clinical prediction model with a continuous outcome. Four criteria are proposed, that target precise estimates of (i) R_^2 (the proportion of variance explained), (ii) calibration-in-the-large (agreement between predicted and observed outcome values on average), (iii) calibration slope (agreement between predicted and observed values across the range of predicted values), and (iv) the variance of observed outcome values. Closed-form sample size solutions are derived for each criterion, which require the user to specify anticipated values of the model’s performance (in particular R_^2) and the outcome variance in the external validation dataset. A sensible starting point is to base values on those for the model development study, as obtained from the publication or study authors. The largest sample size required to meet all four criteria is the recommended minimum sample size needed in the external validation dataset. The calculations can also be applied to estimate expected precision when an existing dataset with a fixed sample size is available, to help gauge if it is adequate. We illustrate the proposed methods on a case-study predicting fat-free mass in children.

Journal Article Type Article
Acceptance Date Mar 22, 2021
Online Publication Date May 24, 2021
Publication Date Aug 30, 2021
Publicly Available Date May 26, 2023
Journal Statistics in Medicine
Print ISSN 0277-6715
Electronic ISSN 1097-0258
Publisher Wiley
Peer Reviewed Peer Reviewed
Volume 40
Issue 19
Pages 4230-4251
DOI https://doi.org/10.1002/sim.9025
Publisher URL http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1097-0258

Files





Downloadable Citations