Wynants, L, Riley, RD ORCID: https://orcid.org/0000-0001-8699-0735, Timmerman, D and Van Calster, B (2018) Random-effects meta-analysis of the clinical utility of tests and prediction models. Statistics in Medicine, 37 (12). pp. 2034-2052.

R Riley - Random effects meta analysis of the clinical utility of tests and predicition models.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

Download (776kB) | Preview


The use of data from multiple studies or centers for the validation of a clinical test or a multivariable prediction model allows researchers to investigate the test's/model's performance in multiple settings and populations. Recently, meta-analytic techniques have been proposed to summarize discrimination and calibration across study populations. Here, we rather consider performance in terms of net benefit, which is a measure of clinical utility that weighs the benefits of true positive classifications against the harms of false positives. We posit that it is important to examine clinical utility across multiple settings of interest. This requires a suitable meta-analysis method, and we propose a Bayesian trivariate random-effects meta-analysis of sensitivity, specificity, and prevalence. Across a range of chosen harm-to-benefit ratios, this provides a summary measure of net benefit, a prediction interval, and an estimate of the probability that the test/model is clinically useful in a new setting. In addition, the prediction interval and probability of usefulness can be calculated conditional on the known prevalence in a new setting. The proposed methods are illustrated by 2 case studies: one on the meta-analysis of published studies on ear thermometry to diagnose fever in children and one on the validation of a multivariable clinical risk prediction model for the diagnosis of ovarian cancer in a multicenter dataset. Crucially, in both case studies the clinical utility of the test/model was heterogeneous across settings, limiting its usefulness in practice. This emphasizes that heterogeneity in clinical utility should be assessed before a test/model is routinely implemented.

Item Type: Article
Additional Information: This is the peer reviewed version of the following article: Random-effects meta-analysis of the clinical utility of tests and prediction models, which has been published in final form at 10.1002/sim.7653. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.
Uncontrolled Keywords: decision curves, diagnostic, meta-analysis, net benefit, test accuracy
Subjects: R Medicine > R Medicine (General)
R Medicine > RA Public aspects of medicine
Divisions: Faculty of Medicine and Health Sciences > Primary Care Health Sciences
Depositing User: Symplectic
Date Deposited: 22 Mar 2018 09:23
Last Modified: 25 Mar 2019 01:30
URI: https://eprints.keele.ac.uk/id/eprint/4635

Actions (login required)

View Item View Item