Sim, J (2019) Should treatment effects be estimated in pilot and feasibility studies? Pilot and Feasibility Studies, 5 (1). ISSN 2055-5784

[thumbnail of Sim 2019 Pilot studies.pdf]
Sim 2019 Pilot studies.pdf - Published Version
Available under License Creative Commons Attribution.

Download (774kB) | Preview


Feasibility studies and external pilot studies are used increasingly to inform planning decisions related to a definitive randomized controlled trial. These studies can provide information on process measures, such as consent rates, treatment fidelity and compliance, and methods of outcome measurement. Additionally, they can provide initial parameter estimates for a sample size calculation, such as a standard deviation or the ‘success’ rate for a binary outcome in the control group. However, the issue of estimating treatment effects in pilot or feasibility studies is controversial.

Methodological discussion
Between-group estimates of treatment effect from pilot studies are sometimes used to calculate the sample size for a main trial, alongside estimated standard deviations. However, whilst estimating a standard deviation is an empirical matter, a targeted treatment effect should be established in terms of clinical judgement, as a minimum important difference (MID), not through analysis of pilot data. Secondly, between-group effects measured in pilot studies are sometimes used to indicate the magnitude of an effect that might be obtained in a main trial, and a decision on progression made with reference to the associated confidence interval. Such estimates will be imprecise in typically small pilot studies and therefore do not allow a robust decision on a main trial; both a decision to proceed and a decision not to proceed may be made too readily. Thirdly, a within-group change might be estimated from a pilot or a feasibility study in a desire to assess the potential efficacy of a novel intervention prior to testing it in a main trial, but again such estimates are liable to be imprecise and do not allow sound causal inferences.

Treatment effects calculated from pilot or feasibility studies should not be the basis of a sample size calculation for a main trial, as the MID to be detected should be based primarily on clinical judgement rather than statistics. Deciding on progression to a main trial based on these treatment effects is also misguided, as they will normally be imprecise, and may be biased if the pilot or feasibility study is unrepresentative of the main trial.

Item Type: Article
Additional Information: This is the final published version of the article (version of record). It first appeared online via BioMed Central at - please refer to any applicable terms of use of the publisher.
Uncontrolled Keywords: Pilot studies, Feasibility studies, Treatment effects, Randomized controlled trials
Subjects: R Medicine > R Medicine (General) > R735 Medical education. Medical schools. Research
Divisions: Faculty of Medicine and Health Sciences > Primary Care Health Sciences
Depositing User: Symplectic
Date Deposited: 02 Sep 2019 10:31
Last Modified: 02 Sep 2019 10:31

Actions (login required)

View Item
View Item