Key Principles for Conducting a Good Randomized Controlled Trial

Multi-Center Versus Single-Center

One of the obvious differences between the 2 RCTs in study design is the number of sites involved in the trial.^2,3 The study by Grieco and colleagues is a multi-center RCT with 4 ICUs, whereas the study by Nair and colleagues is a single-center RCT. In contrast to multi-center trials, single-center trials are generally simpler and easier to implement. However, one of the most serious drawbacks of a single-center trial is lack of external validation, which reduces the power to support its implementation in practice in a wider clinical setting. For example, in a review of RCTs in critical care, Bellomo and colleagues⁶ demonstrated contradictory results or active harm in subsequent definitive multi-centered trials compared to what was recommended by the previous single-center trials. Another limitation of single-center RCTs is the small-scale population, which potentially increases the risk of drawing an inaccurate conclusion. A meta-epidemiological study⁷ compared 26 meta-analyses including 292 RCTs (177 single-center vs 115 multi-center) and found that single-center trials overestimated a significantly larger treatment effect than multi-center trials. Nevertheless, limitations of single-center trials should not be interpreted as reasons to prevent them from being conducted. Single-center trials are useful for testing efficacy of new treatments (internal validity) in the early-phase patient studies due to their simplicity, scalability, flexibility, and cost-effectiveness.⁸

Study Protocol

An RCT protocol usually includes rationale/background, methods, outcome measures, and proposed statistical analyses. It allows standardized and consistent trial conduct across investigators, trial personnel, and participants to reduce biases and improve data analyses adherence and outcome reporting.⁹ A study reviewing protocols of 48 trials and their subsequent reports in journal publications found that 31% of outcomes were incompletely reported in each trial. Forty percent of trials had major discrepancies in primary outcomes between protocols and publications and a higher ratio of fully reported outcomes in statistically significant outcomes than that in nonsignificant outcomes.¹⁰ Although guidelines for RCT protocol content vary,¹¹ the Consolidated Standards of Reporting Trials (CONSORT) Statement is well known as standard criteria for reporting of RCT findings.¹² CONSORT is a checklist for quality assessment covering trial design, sample size calculation, data analyses, outcome reporting, and interpretation, which has been shown to improve quality of reporting.¹³ Comparing the 2 example RCTs used for this discussion,^2,3 a detailed protocol is accessible in the supplementary files for the study by Grieco and colleagues, whereas a brief registration protocol is available for the study by Nair and colleagues. Although there are no biases in outcome reporting between protocol and publication, a detailed protocol generally provides more information and transparency in trial conduct.

Trial Registration

Under the umbrella of the CONSORT Statement, the prospective registration of a trial before enrollment is recommended by the International Committee of Medical Journal Editors (ICMJE) Statement¹⁴ and required for publication in all journals. A proper trial registration ensures a transparent overview of the process so that publication and selective reporting biases are reduced.^14,15 In regard to participants, public accessibility to a registered trial provides information that facilitates enrollment and fulfillment of ethical obligations.^14,16 Although no specific registries are advocated by ICMJE, ClinicalTrials.gov, operated by the National Library of Medicine of the National Institutes of Health, is one of the most well-known registries that meets all criteria for acceptable registry, which includes access to the public for free, open to all prospective registrants, managed by a nonprofit organizations, able to validate registration data, and is electronically searchable.¹⁷ A list of primary registries that are recognized by both ICMJE and the World Health Organization International Clinical Trials Registry Platform (ICTRP) can be found in ICTRP Registry Network (https://www.who.int/clinical-trials-registry-platform/network/primary-registries Accessed July 26, 2022). Multiple registrations in more than one registry sometimes happen for practical purposes, for example, a multinational/multi-center trial that involves participants from multiple countries/institutions. A systematic review of 197 RCTs registered in more than one registry found significant inconsistency across trial registries, including 22% for primary outcomes and 37% for target sample size,¹⁸ which highlights potential biases and unreliability and indicates the need for future guideline updates.

Blinded Versus Unblinded

Blinding is a commonly used methodology in RCTs, which is believed to prevent performance bias between treatment groups and ascertainment bias during assessment of outcomes.¹⁹ It is believed that unblinded trials can result in exaggerated estimates of treatment effects.²⁰ However, this belief has been challenged by a recent systematic review²¹ of 142 meta-analyses including 1,153 trials that found no significant differences in estimated treatment effects between trials with and without blinding subjects, caregivers, and outcome assessors. Furthermore, it has also been reported²² that bias effects of unblinding cannot be generally applied or deemed in all clinical areas. In unblinded trials, subjective outcome measures are reported to be associated with a significant exaggeration of intervention effects, but not objectively measured outcomes.²³ Therefore, additional efforts and measures should be taken to minimize performance bias if a trial is designed as unblinded to assess subjective outcomes. In the 2 example RCTs,^2,3 participants and clinicians were not blinded and one of the outcome measures was intubation, which is a subjective decision based on specific criteria. This potential performance bias was properly considered in the trial by Grieco and colleagues and adjusted by introducing an adjudication committee of 2 blinded experts to verify the decision of intubation.

Baseline Imbalance (Confounders)

An imbalance covariate, also called a confounder or confounding factor, is a bias that mixes effects of an additional element with effects of treatment exposure on a given outcome, which eventually distorts the true relationship between treatment exposure and outcome.²⁴ In other words, the observed differences in outcomes between groups might be, by chance, due to the confounders if they occur.²⁵ Therefore, confounders could either mask real effects of treatment intervention or lead to false-positive effects. For example, in both RCTs, awake prone positioning was predominantly applied in HFNC group (60% and 100% in the study by Grieco and Nair, respectively), whereas no patient on NIV was compliant to awake prone positioning.^2,3 Here, introduction of awake prone positioning is an obvious confounder because it has been demonstrated to be associated with a significantly lower risk of intubation in patients with COVID-19–related hypoxemic respiratory failure.²⁶ Hence, the significant difference in intubation between HFNC and NIV might be due to use of awake prone positioning in HFNC group in the study by Nair and colleagues. To control confounding factors, apart from the randomization procedures at study design, multivariate analysis or stratification can be used to adjust for them during analysis after completion of a study.^24,27

RCTs are the most scientifically rigorous design in methodology with considerably low biases; however, they are only clinically informative and beneficial for developing evidence-based clinical practices if they are well designed and properly executed. Besides the factors mentioned above, other components are also imperative for a good RCT, including but not limited to an appropriate end point, sample size, statistical analyses, safety measurements, fulfillment of subject recruitment, and a collaborative team with full engagement.²⁸ When planning an RCT, potential biases should be considered in every aspect of the trial, and appropriate measures need to be implemented to minimize the influence of these biases on the final results.