Julius Sim j.sim@keele.ac.uk
Should treatment effects be estimated in pilot and feasibility studies?
Sim
Authors
Abstract
Background
Feasibility studies and external pilot studies are used increasingly to inform planning decisions related to a definitive randomized controlled trial. These studies can provide information on process measures, such as consent rates, treatment fidelity and compliance, and methods of outcome measurement. Additionally, they can provide initial parameter estimates for a sample size calculation, such as a standard deviation or the ‘success’ rate for a binary outcome in the control group. However, the issue of estimating treatment effects in pilot or feasibility studies is controversial.
Methodological discussion
Between-group estimates of treatment effect from pilot studies are sometimes used to calculate the sample size for a main trial, alongside estimated standard deviations. However, whilst estimating a standard deviation is an empirical matter, a targeted treatment effect should be established in terms of clinical judgement, as a minimum important difference (MID), not through analysis of pilot data. Secondly, between-group effects measured in pilot studies are sometimes used to indicate the magnitude of an effect that might be obtained in a main trial, and a decision on progression made with reference to the associated confidence interval. Such estimates will be imprecise in typically small pilot studies and therefore do not allow a robust decision on a main trial; both a decision to proceed and a decision not to proceed may be made too readily. Thirdly, a within-group change might be estimated from a pilot or a feasibility study in a desire to assess the potential efficacy of a novel intervention prior to testing it in a main trial, but again such estimates are liable to be imprecise and do not allow sound causal inferences.
Conclusion
Treatment effects calculated from pilot or feasibility studies should not be the basis of a sample size calculation for a main trial, as the MID to be detected should be based primarily on clinical judgement rather than statistics. Deciding on progression to a main trial based on these treatment effects is also misguided, as they will normally be imprecise, and may be biased if the pilot or feasibility study is unrepresentative of the main trial.
Journal Article Type | Article |
---|---|
Acceptance Date | Aug 21, 2019 |
Publication Date | Aug 28, 2019 |
Publicly Available Date | Mar 29, 2024 |
Journal | Pilot and Feasibility Studies |
Print ISSN | 2055-5784 |
Publisher | BioMed Central |
Peer Reviewed | Peer Reviewed |
Volume | 5 |
Article Number | 107 |
DOI | https://doi.org/10.1186/s40814-019-0493-7 |
Keywords | Pilot studies, Feasibility studies, Treatment effects, Randomized controlled trials |
Publisher URL | http://doi.org/10.1186/s40814-019-0493-7 |
Files
Sim 2019 Pilot studies.pdf
(774 Kb)
PDF
Publisher Licence URL
https://creativecommons.org/licenses/by/4.0/
You might also like
Downloadable Citations
About Keele Repository
Administrator e-mail: research.openaccess@keele.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search