Applying results of randomised trials to patients
BMJ 1998; 317 doi: https://doi.org/10.1136/bmj.317.7157.537 (Published 22 August 1998) Cite this as: BMJ 1998;317:537
N of 1 trials are needed
- Stephen Senn, Professor of pharmaceutical and health statistics. (stephens{at}public-health.ucl.ac.uk)
- Department of Epidemiology and Public Health, Department of Statistical Science, University College London, London WC1E 6BT
- Department of Clinical Epidemiology, McMaster University, Hamilton, Ontario, Canada L8N 3Z5
- Division of Respiratory Medicine, University of Toronto, Toronto, Ontario, Canada
EDITOR—Guyatt et al's proposal for analysing randomised trials1 is misguided, flies in the face of elementary statistical theory, and should be resisted. There are three obvious sources of variability in clinical trials.Firstly, pure differences occur between patients:some are more seriously ill than others. Secondly, there is variability within patients: even given the same treatment they, or their measurements, may vary from time to time. Thirdly, some patients may react more favourably to a given treatment than other patients. The parallel group trial does not and cannot distinguish between the three types of variability unless we can find meaningful ways of classifying subgroups.2The standard crossover trial will distinguish between the first type of variability and the other two but not easily between the second and third,4and certainly not in the form of analysis suggested by Guyatt et al.
Guyatt et al have implicitly assumed that which of two treatments is better for a patient can be determined by comparing one period of treatment on each. This is at complete variance to advice Guyatt and coworkers have given elsewhere.5 They have previously suggested that to establish efficacy for individual patients the patients should be randomised to repeated periods of treatment and control: the so called “n of l” method.
Nothing from the two clinical trials presented by Guyatt et al is inconsistent with the theory that all patients benefitted equally. If we wish to establish what proportion of patients benefit from treatments, rather than merely being satisfied with average effects, then we need random effect models and sequences of n of 1 trials.3Since the method which they propose does not correctly partition the sources of random variability, it will simply produce random results.
Authors' reply
- Gordon Guyatt, Professor,
- Elizabeth F Juniper, Professor,
- Roger S Goldstein, Professor
- Department of Epidemiology and Public Health, Department of Statistical Science, University College London, London WC1E 6BT
- Department of Clinical Epidemiology, McMaster University, Hamilton, Ontario, Canada L8N 3Z5
- Division of Respiratory Medicine, University of Toronto, Toronto, Ontario, Canada
EDITOR—Contrary to Senn's interpretation, we do not propose deciding on which individual patients in the trial benefit but rather the overall proportion who obtained a particular magnitude of benefit. Senn identifies three sources of variation in individual patient's responses—differences between patients, differences within patients due to random variation, and differences due to a systematic treatment by patient interaction. To that we may add a fourth—the overall main effect of treatment.
Senn's logic fails when he argues that nothing from the two clinical trials is inconsistent with the theory that all patients benefitted equally. Quite the contrary, the key is that randomising patients to treatment and control and aggregating results across patients permits independent estimates of the main effect of treatment and the other three sources of variance. The sources of variance are confounded in a parallel group design; patient variance is separable in a crossover design; but all are separable from the overall treatment effect by virtue of the use of a sample of patients and random allocation.
In our previous work on n of 1 clinical trials we recommend multiple periods of treatment and control in order to establish the efficacy for individual patients1The principle is the same—multiple observations, whether from a single patient or multiple patients, permit separate estimation of the main treatment effect and other sources of variation. Rather than “flying in the face of elementary statistical theory” this experimental approach follows directly from such theory.
Our paper showed how we can make the results of trials examining quality of life more easily interpretable by estimating the proportion of patients who benefit from a treatment. Senn's letter highlights a question clinicians may legitimately ask: “Ah, but which patients?” N of 1 randomised trials will still be required to address this question.