Skip to main content

Research Repository

Advanced Search

Should treatment effects be estimated in pilot and feasibility studies?

Sim

Should treatment effects be estimated in pilot and feasibility studies? Thumbnail


Authors



Abstract

Background
Feasibility studies and external pilot studies are used increasingly to inform planning decisions related to a definitive randomized controlled trial. These studies can provide information on process measures, such as consent rates, treatment fidelity and compliance, and methods of outcome measurement. Additionally, they can provide initial parameter estimates for a sample size calculation, such as a standard deviation or the ‘success’ rate for a binary outcome in the control group. However, the issue of estimating treatment effects in pilot or feasibility studies is controversial.

Methodological discussion
Between-group estimates of treatment effect from pilot studies are sometimes used to calculate the sample size for a main trial, alongside estimated standard deviations. However, whilst estimating a standard deviation is an empirical matter, a targeted treatment effect should be established in terms of clinical judgement, as a minimum important difference (MID), not through analysis of pilot data. Secondly, between-group effects measured in pilot studies are sometimes used to indicate the magnitude of an effect that might be obtained in a main trial, and a decision on progression made with reference to the associated confidence interval. Such estimates will be imprecise in typically small pilot studies and therefore do not allow a robust decision on a main trial; both a decision to proceed and a decision not to proceed may be made too readily. Thirdly, a within-group change might be estimated from a pilot or a feasibility study in a desire to assess the potential efficacy of a novel intervention prior to testing it in a main trial, but again such estimates are liable to be imprecise and do not allow sound causal inferences.

Conclusion
Treatment effects calculated from pilot or feasibility studies should not be the basis of a sample size calculation for a main trial, as the MID to be detected should be based primarily on clinical judgement rather than statistics. Deciding on progression to a main trial based on these treatment effects is also misguided, as they will normally be imprecise, and may be biased if the pilot or feasibility study is unrepresentative of the main trial.

Journal Article Type Article
Acceptance Date Aug 21, 2019
Publication Date Aug 28, 2019
Publicly Available Date Mar 29, 2024
Journal Pilot and Feasibility Studies
Print ISSN 2055-5784
Publisher BioMed Central
Peer Reviewed Peer Reviewed
Volume 5
Article Number 107
DOI https://doi.org/10.1186/s40814-019-0493-7
Keywords Pilot studies, Feasibility studies, Treatment effects, Randomized controlled trials
Publisher URL http://doi.org/10.1186/s40814-019-0493-7

Files




You might also like



Downloadable Citations