Evaluating complementary medicine: methodological challenges of randomised controlled trials

Sampling

Separation of organisations —Complementary care within European countries is generally separate to conventional medicine. Complementary medicine is not usually integrated into national health systems, and individuals pay for treatment. Funding for trials should include the additional costs of complementary treatment. Additional complexities require attention; these include legal indemnity for the practitioners taking part and assurances of their professional and technical competence.

Generalisability —Because complementary and conventional care are separate, patients recruited to a trial comparing the two may differ from typical complementary patients since they themselves did not decide which treatment they would receive. For example, belief systems or coping mechanisms might differ, affecting treatment outcomes. Incorporating preference in the design of the trial might help tackle and investigate this problem.

Acupuncture is a popular complementary treatment

Therapeutic expectation—Choosing or preferring a particular complementary discipline carries an associated implicit belief in benefit. Such credibility and expectation of therapeutic gain can bias the results of trials, especially if they cannot be blinded (as in a trial of yoga practice or tai chi). Expectation and perceived credibility should be acknowledged and assessed at randomisation. Several controlled trials have used a credibility scale to check the equivalence of psychological treatments. 10 11

Wide range of symptoms—Complementary medicine and subsequent diagnosis are often not used because of a specific disease. Instead, users may feel tired, lack energy, or be unhappy. A trial about the effectiveness of an intervention based on a disease may not be relevant to complementary treatments. Trials may need to focus on problem areas which are independent of categories of disease. Broad and appropriate outcome measures should capture a wide range of symptoms.

Treatment

Standardisation—Within a complementary discipline, treatment is individualised and includes care associated with an individual's “diagnosis” and the specific practitioner (skills, competence, manner, etc). The practitioner is explicitly recognised as a component of or contributor to the treatment. This may conflict with ideas of standardisation in conventional trials. Complex treatment interventions can be investigated pragmatically. Standardised treatment can be interpreted as treating to a standard regimen, defined in a protocol, rather than exactly the same treatment. Randomisation should include stratification by individual practitioner.

Influence of practitioner and user —Practitioners often claim that they do not just carry out a set of techniques—for example, the remedy in herbalism or the needles in acupuncture—on the user. They claim to understand how to improve the healing process, the user's role in the healing process, and the therapeutic relationship between the user and the practitioner. All aspects of complementary care should be assessed, and the human experience should be retained as central (not marginalised, as often occurs in reductionist methods).12 If a trial design includes modified therapies so that the holistic aspect is excluded, then this must be explicitly stated.

Controls —Choosing an appropriate comparison group in trials of complementary medicine trials is often problematic. Sham controls—for example, dummy acupuncture to create a placebo controlled comparison—should be treated with caution as they are artificial experiences and distort practice. Any benefit over standard medical treatment should be assessed and appropriate controls adopted. For some complementary treatments—for example, homoeopathy and herbalism—the traditional blinded randomised controlled trial is appropriate. For treatments in which blinding is not possible—for example, meditation—emphasis should be on pragmatic trials with conventional medical practice as controls.

Understanding complementary processes—It is difficult to define precisely which aspect of a complementary treatment provides benefit. During spiritual healing, is it the intent to heal, the preparation, the channelling of “energy,” the accompanying non-judgmental listening to the user, the relationship with the healer, the healing environment, users' expectations or attitudes, or all of these which are important? Complex interventions in conventional medicine result in similar problems.13 Too often trials provide both limited detail on the actual process and little insight into what component of the intervention leads to the effect. Qualitative methods should be used to investigate the process. The design of the trial should address the therapeutic relationship and non-specific effects, in addition to any specific treatment effects.14 At the same time, from the user's perspective, it is the beneficial effect itself that matters, not how it was brought about.

Outcome measures

Appropriate outcome measures —Capturing the essence of how complementary medicine might help people is an important challenge. Noteworthy changes after treatment, such as spiritual change or personal growth, might not be measurable. Choice of outcome criteria in the evaluation of complementary therapies “may make a substantial difference to conclusions as to whether a particular intervention or therapy is effective or not.”14 Holism, which is central to the philosophy and practice of much complementary medicine, needs to be better understood. Detailed qualitative studies, accompanying a trial, could elicit these more elusive outcomes and aim to take account of experiential outcome criteria, such as increased self reflection.

Illness role —According to some complementary philosophies, illness indicates an imbalance in the body and symptoms of illness draw attention to this. For example, individuals develop colds, backache, or eczema when they are under stress or run down. If complementary medicine helps people recognise imbalance and adverse lifestyle factors that contribute to illness and promotes change and resultant improvements in health, then research needs to capture these changes.

Chronic illness —Complementary treatment is often sought by people with chronic illnesses who may not have responded to conventional treatment or may want to explore different forms of care. Any change is likely to be gradual and subtle and to have a relapsing and remitting pattern. As complementary interventions tend to aim for long term healing, and not necessarily cure, they are well suited to help people with chronic illness. Funding bodies must understand that long term follow up of outcomes (specific and non-specific) is needed for results to be meaningful. Randomised controlled trials work best for exploring short term effects. Over time, factors other than the trial treatments could bring about the observed effects.

Variations in experience —One person can have different experiences of complementary medicine at different times; different people can have widely different experiences. Designing a trial to investigate a treatment that might be effective immediately for one person but take months to be effective for another is a problem. A “:healing moment”: may occur at the time of treatment or some time after. Appropriate experiential outcome measures should take variations into account along with adequately long follow up periods.

Sick role —Some users do not respond to treatment. There are those who adapt to the sick role because they have an investment in staying ill. Others may be satisfied with the short fix, of feeling peaceful and relaxed, that is offered by many complementary treatments and may not take responsibility to make the necessary changes to improve their health. In principle, randomised controlled trials provide an efficient way to deal with this problem. People adopting the sick role should be distributed equally throughout the arms of a trial through the randomisation process.