Statistical analysis of randomised controlled trials: a simulation and empirical study of methods of covariate adjustment

Abstract

Randomised controlled trials (RCTs) are widely accepted as the optimum design for comparing two or more medical therapies. Chance baseline imbalance (BI) through randomization opens the estimate of effect to bias. Statistical methods such as change score analysis (CSA) and analysis of covariance (ANCOVA) - are commonly used to deal with BI. However, unadjusted analysis by analysis of variance (ANOVA) is still common.

This study examined precision, power, efficiency and bias of estimates of effect associated with ANOVA, CSA and ANCOVA in RCTs with a single post-treatment assessment of a continuous outcome variable. A total of 210 hypothetical trial scenarios were evaluated; each was simulated (1000 iterations per scenario) using combinations of specific levels of treatment effect, covariate-outcome correlation, direction and level of BI. Evaluation was also performed on three empirical trial datasets, which showed different baseline-outcome correlations.

Precision and efficiency of CSA were not better than those of ANOVA unless baseline-outcome correlation exceeded 0.5. Depending on the level of baseline-outcome correlation, the sample size required at a given level of nominal power can be reduced by up to 80% using ANCOVA. Conditionally, both ANOVA and CSA are prone to false-negative or false-positive error. When BI exists in the same direction as treatment effect, the conditional power to detect the unbiased effect by ANCOVA falls below the nominal power in most trial scenarios. Also the review of current practices regarding covariate adjustment shows that whereas over 60% adopt appropriate (modelling and stratified analysis) statistical adjustment, the most widely used single analytical approach is change. Overall, minimum covariate-outcome correlation of 0.3 is necessary but not sufficient to consider a covariate for inclusion in the model for adjustment. Appropriateness of CSA depends largely on baseline-outcome correlation and direction of imbalance. ANOVA is reasonable if the prognostic strength of the covariate is low. ANCOVA is the optimum statistical strategy regardless of BI.