Equipoise, design bias, and randomized controlled trials: the elusive ethics of new drug development
© Fries and Krishnan, licensee BioMed Central Ltd. 2004
Received: 4 February 2004
Accepted: 3 March 2004
Published: 18 March 2004
The concept of 'equipoise', or the 'uncertainty principle', has been represented as a central ethical principle, and holds that a subject may be enrolled in a randomized controlled trial (RCT) only if there is true uncertainty about which of the trial arms is most likely to benefit the patient. We sought to estimate the frequency with which equipoise conditions were met in industry-sponsored RCTs in rheumatology, to explore the reasons for any deviations from equipoise, to examine the concept of 'design bias', and to consider alternative ethical formulations that might improve subject safety and autonomy. We studied abstracts accepted for the 2001 American College of Rheumatology meetings that reported RCTs, acknowledged industry sponsorship, and had clinical end-points (n = 45), and examined the proportion of studies that favored the registration or marketing of the sponsor's drug. In every trial (45/45) results were favorable to the sponsor, indicating that results could have been predicted in advance solely by knowledge of sponsorship (P < 0.0001). Equipoise clearly was being systematically violated. Publication bias appeared to be an incomplete explanation for this dramatic result; this bias occurs after a study is completed. Rather, we hypothesize that 'design bias', in which extensive preliminary data are used to design studies with a high likelihood of being positive, is the major cause of the asymmetric results. Design 'bias' occurs before the trial is begun and is inconsistent with the equipoise principle. However, design bias increases scientific efficiency, decreases drug development costs, and limits the number of subjects required, probably reducing aggregate risks to participants. Conceptual and ethical issues were found with the equipoise principle, which encourages performance of negative studies; ignores patient values, patient autonomy, and social benefits; is applied at a conceptually inappropriate decision point (after randomization rather than before); and is in conflict with the Belmont, Nuremberg, and other sets of ethical principles, as well as with US Food and Drug Administration procedures. We propose a principle of 'positive expected outcomes', which informs the assessment that a trial is ethical, together with a restatement of the priority of personal autonomy.
Keywordsdesign bias ethical principles equipoise expected outcomes randomized controlled trials
The concept of 'equipoise', the 'uncertainty principle', has been represented as a central ethical principle of human experimentation [1–15]. Equipoise holds that a patient may be enrolled ethically into a randomized controlled trial (RCT) only when substantial uncertainty surrounds which of the trial treatments would most likely benefit them. More formally, "to be in equipoise between two treatments A and B is to be cognitively indifferent between the statement 'A is strictly more effective than B' and its negation ... equipoise regarding A and B is necessary for randomized assignment to treatments A and B to be beneficent and non-maleficent and is sufficient for such an assignment to be fair" . In the strictest mathematical formulations, equipoise has been held to require that the prior probability for either treatment A or treatment B being superior to the other is 0.5.
Such definitions of equipoise, if literally interpreted, would substantially impede RCT-based clinical research. Indeed, when is nothing known before a trial that would permit inference of the 'better' arm with an accuracy exceeding 50%? Few, if any, RCTs would be 'ethical' according to such rigid definitions. Such concerns have led to a softening of definitions. Equipoise, it has been suggested, could represent uncertainty in the collective conventional scientific wisdom, in the patient community, or in the mind of the potential participant or of an individual physician [2, 3]. 'Zones of indifference' could be constructed around the equipoise (50:50) point. In one study of patient opinions, one half believed a trial to be unethical if equipoise was disturbed beyond 70:30, and 97% believed it unethical if equipoise were disturbed beyond 80:20 . It was recently argued that perceptions of basic fairness (not necessarily equipoise) are important to the decision to accept a trial . Much of the medical ethics based equipoise literature has attempted to harmonize the uncertainty principle with the need for medical progress. The ethical basis of equipoise itself has seldom been challenged.
The term 'equipoise', in the context of RCTs, means the following : participants will not suffer relative harm from random assignment to a particular treatment arm; the results of a study cannot be predicted consistently in advance; and over a number of RCTs those proving or failing to prove an hypothesis will be approximately equal in number.
Industry-sponsored RCT abstracts accepted for the 2001 American College of Rheumatology (ACR) meetings  were studied. All abstracts (n = 45) reporting RCTs, acknowledging pharmaceutical company sponsorships as required by the ACR, containing a study arm with a drug from that sponsor, and having clinical end-points were analyzed. Abstracts were categorized as 'favorable' or 'unfavorable' to the sponsor's drug. 'Favorable' required superiority in efficacy to placebo or comparator.
Note that the analysis comprised abstracts numbered 88, 89, 90, 149, 151, 152, 179, 190, 221, 224, 226, 429, 432, 492, 494, 495, 496, 497, 498, 500, 502, 595, 599, 603, 604, 631, 727, 802, 809, 810, 965, 968, 976, 1032, 1327, 1550, 1595, 1808, 1809, 1896, 1897, 1904, 1915, 1917, and 1921 from the ACR report .
The results from every RCT (45 out of 45) favored the drug of the sponsor (P < 0.0001).
Clearly, the results of these trials could have been accurately predicted based on knowledge of sponsorship alone. The studies appeared to be well executed in general, most had academic authorship, and they were intended to survive stringent US Food and Drug Administration (FDA) review. Hence, we considered that poor scientific quality was unlikely to be a major contributor to this result. It actually has been suggested that 'quality scores' of industry-sponsored RCTs may be better than those of publicly funded RCTs , strengthening our belief that these studies were generally carefully and accurately performed.
How, then, could these results have been so predictable? There are at least four nonexclusive potential explanations. First, a placebo control was employed as an arm in 32 of the 45 studies, and prestudy evidence (e.g. phase I, II, and prior phase III studies) is likely to have suggested that the new drug would be found to be superior to placebo. Second, of the 22 studies in which an active comparator arm was used, three were derived from previously reported RCTs, essentially reporting the same trial using different end-points such as cost or quality of life. Third, 'publication bias', discussed below, might have played a role. Finally, we suggest that a relatively undiscussed bias, 'design bias', is likely to have contributed substantially to the predictability of these results.
In the present study we only studied abstracts for one meeting in one medical specialty, and we classified studies using only information available in the abstract. This might limit the ability to generalize from our findings, and certainly if one looked at enough trials some exceptions would be found. However, our results are confirmed by many other studies. In a study of 61 industry-sponsored trials of nonsteroidal anti-inflammatory drugs , 100% of trials found the sponsor's drugs comparable or superior to the comparator; no studies favored the comparison treatment. In multiple myeloma trials, equipoise was generally met in publicly funded RCTs, but 74% of commercial RCTs favored the new product, suggesting differences related to funding source . Bekelman and coworkers  found financial conflicts of interest to be widespread. Als-Nielsen and colleagues  found that funding source was related to reported conclusions in drug trials. On the other hand, many publicly sponsored RCTs address issues with uncertain outcomes (see O'Dell and coworkers ). However, a large majority of industry-sponsored trials must violate the principle of equipoise.
Publication bias, where positive studies are more likely to be published than negative ones, is an obvious potential explanation for these findings. However, the magnitude of publication bias remains controversial, and a number of studies suggest that this type of bias has only small effects [21–24], whereas our observed effects are huge. Recently, publication bias in oncology trials was noted to include more specific causes of 'lack of time or resources', 'incomplete study', and others that are not necessarily related to study outcome. Only 10% of unpublished papers were so categorized because of insufficient priority to warrant publication; 81% of positive trials and 70% of negative ones were published, suggesting a publication bias of perhaps 13% . Publication bias is thus unlikely to account for the unanimity of the results reported here, although it is likely to have played some role. Publication bias occurs after an RCT has been completed.
Design bias, on the other hand, occurs before the RCT is begun, when the trial parameters are determined and before the final decision has been made to initiate the study . Major deviations from equipoise appear to result from the design stage, particularly during the drug development process.
From an industry perspective the drug development process must involve 'designing for success' (Fig. 1). In a well established set of procedures company consultants and staff debate what is known about the drug, its competitors, its potential advantages in terms of toxicity or efficacy, and the potential disease indications. One of us (JFF) has frequently been involved in this process. Then, trials are designed that include the patients, dosages, study duration, end-points, and comparators that are likely to provide a positive result for the sponsor and one that is acceptable to the US FDA. These design decisions are intended in part to identify the most appropriate clinical niche for the product, using all prior information. A funding commitment by a for-profit entity to an RCT that may cost hundreds of millions of dollars simply will not be made unless a positive outcome may be predicted with considerable certainty.
Is 'design bias' a bad thing? At first it appears so. After reflection, however, we would suggest 'not necessarily'. How else should studies be designed? Should we study drugs without promise – study drugs that are not thought to be superior to placebo or drugs with no known potential advantages over existing drugs? Should we conduct studies that fail to identify an appropriate, perhaps narrow, therapeutic niche for the drug? From a trial participant's perspective, the current design process limits the chance of exposure to an ineffective or unduly toxic drug. From a social perspective, violation of equipoise is essential to efficient medical progress. To enroll humans in large RCTs without preliminary studies might pose truly major risks to participants, but after preliminary studies have been conducted true uncertainty no longer exists. The principle of equipoise becomes the paradox of equipoise.
Ethical issues with equipoise
The equipoise principle, upon examination, actually contributes to ethical problems, in part because it embodies an unreasonably paternalistic attitude. When we, as clinicians, ask a patient to consider participation in a trial, the typical responses are 'Might this study help others?' and 'Are the risks reasonable?' In stark contrast, the equipoise principle does not allow consideration of potential social benefits or consideration of the magnitude of the (often very small) risk to the patient. Contrary to the altruism expressed by many patients, equipoise gives weight neither to personal autonomy nor to personal satisfaction.
Of substantial importance, the authoritative Belmont Report , the US National Institutes of Health, the US FDA, and the Nuremberg Code  do not require equipoise. The Belmont Report enunciates three principles: autonomy (respect for persons), beneficence, and justice. Autonomy refers to the patient who, unless of diminished mental capacity or under coercion, is capable of self-determination and must be granted its exercise. Beneficence refers to study designs that to the greatest degree possible maximize benefits and minimize risks to the patient. Justice refers to fairness in distributing the benefits and burdens of research. The concluding section of the Belmont Report notes a standard of 'the reasonable volunteer' and that 'no undisclosed risks may be more than minimal'. Even more importantly, 'risks to subjects must be outweighed by the sum of the anticipated benefit to the subject, if any, and the anticipated benefit to society' and 'beneficence requires that we protect against risk of harm to subjects and also that we be concerned about the loss of substantial benefits that might be gained from research' .
Equipoise fails as an ethical principle because it directly contradicts the principle of personal autonomy and fails to consider the balance between the individual and the societal good. It implicitly encourages performance of trials that will prove uninformative by limiting the expected differences between the arms of the trial. It has been suggested, with similar reasoning, that underpowered RCTs are unethical because the resulting negative results may not benefit society [28, 29].
We propose two principles that more appropriately underlie the ethics of enrollment of experimental subjects into an RCT: positive expected value and exercise of personal autonomy.
First principle: positive expected value of participation
For example, consider an RCT of a new drug that is believed likely to reduce osteoarthritis pain by 40% versus a clinical standard known to reduce pain by 20%. Before randomization, each participant has a pooled expected value of a pain reduction of 30%, which is 10% more pain relief than under usual care. The expected value of participation is positive – it is of greater value to the participant than declining the RCT and accepting usual care, and the study is ethically sound – but it does not meet the weaker principle of 'equipoise' or 'uncertainty'.
Placebo-controlled RCTs will usually have positive pooled expected values when new drug and placebo alike are added to usual care. However, if the placebo and the new drug replace usual care a study may not have positive pooled expected values. If usual care is expected to yield 30% pain reduction, placebo 10%, and new drug 40%, then the pooled average of the arms of 25% is less beneficial than usual care. For the 'positive expected value' principal to be met in this instance, the expected pooled positive effects from the placebo and new drug must exceed the expected negative effect from the loss of usual care.
The equipoise principle is replaced here by a new ethical standard of reasonable 'positive expected value' with a higher standard in protection of the participant. Unlike equipoise, this standard allows placebo-controlled RCTs with predictable results, where the net effect over all participants is expected to be a benefit. It does away with the charade of pretending not to know the likely outcome in advance.
Second principle: autonomy for patients to make decisions despite negative average expected value
The reasonable and autonomous prospective participant might wish to accept an RCT even though the pooled expected value is negative. Therefore, following full factual review of expected outcomes with the potential participant, the personal values of that patient need to be applied. A common circumstance is altruism, with the patient valuing social progress higher than personal risks. In other instances, minimal increases in risks, although real, may be of little importance to the patient.
Another circumstance is the choice of a decision strategy other than 'expected value' by the patient. 'Expected value' is not the only defensible decision strategy for an individual; decisions may be driven by a desire to have a chance at the best possible outcome or to avoid the possibility of the worst possible outcome. A reasonable person could choose to participate in an RCT because the best possible result could only come from acceptance, even if the pooled expected value of accepting the RCT is negative. For example, accepting an RCT might be the only possible way to gain access to a new treatment. Alternatively, another reasonable potential participant could decline a trial with a positive average expected value because of excessive fear about the described potential side effects of the new drug ('risk aversion').
Autonomy implies the ability to make decisions under a personal rather than an externally imposed value system. However, the standard is that of the 'reasonable person'. Autonomy also implies absence of coercion, full disclosure of significant risks, and protection of those with limited mental capacity, as well as other experimental subject protections described in the Belmont Report .
We do not propose that formal decision analysis (expert driven) and utility assignment (patient driven) should be required of prospective participants, although they could well be helpful to institutional review boards. The complexity of the discussion must be appropriate for the patient. However, formalizing the concepts helps to clarify the issues involved. In a similar vein, Lilford  recently noted that 'choices should be based both on probabilities of events (which experts might know) and on the value that a patient places on these events (which only the patient can know)'.
In a broader sense, we note that this same decision process should apply to all studies involving human participant, and not only RCTs. A forthright invitation, objective explanation of known facts, application of the patient's personal values, and a reasonable and autonomous decision are salient necessities for all such studies.
Equipoise is a false and diverting principle, and should be replaced by 'positive expected value' using the patient's values, with elimination of the internal contradictions that surround the concept of equipoise and with a return to the Belmont principles. In this context, should we worry that a large majority of commercial clinical trials are positive? The answer is this is 'yes and no'. Certainly, skepticism and vigilance are required with industry-sponsored trials because there are inevitably large financial interests, and omitted results, redundant presentations, and marketing 'spin' do sometimes occur [18, 19, 29, 31]. Studies designed to evaluate noninferiority with an US FDA approved comparator may represent 'me too' drugs and raise questions about a societal need for a drug. On the other hand, the ability to design clinical trials for success, and therefore to be able to predict outcomes in advance, implies the ability to cull weak drugs early in the approval process (and this frequently occurs), thus exposing fewer patients, improving efficiency in drug development, possibly reducing the cost of pharmaceuticals , and identifying those drugs that are most likely to be clinically useful and of positive social value.
The pseudo-ethical principle of equipoise, with its inappropriate decision point, must be abandoned. It disregards patient autonomy, fails to protect patients on aggregate, ignores potential benefits to society and impedes medical progress, and there are better alternatives.
American College of Rheumatology
Food and Drug Administration
randomized controlled trial.
This work was supported by a grant from the US National Institutes of Health (AR43584) to ARAMIS (Arthritis, Rheumatism, and Aging Medical Information System). The authors appreciate helpful review and criticism from Bonnie Bruce, Edward Harris, Halsted Holman, Mark Genovese, John Reiland, and Michael Ward.
- Ashcroft R: Equipoise, knowledge and ethics in clinical research and practice. Bioethics. 1999, 13: 314-326. 10.1111/1467-8519.00160.View ArticlePubMedGoogle Scholar
- Freedman B: Equipoise and the ethics of clinical research. N Engl J Med. 1987, 317: 141-145.View ArticlePubMedGoogle Scholar
- Loftus EF, Fries JF: Informed consent may be hazardous to health. Science. 1979, 204: 11-View ArticlePubMedGoogle Scholar
- Johnson N, Lilford RJ, Brazier W: At what level of collective equipoise does a clinical trial become ethical?. J Med Ethics. 1991, 17: 30-34.PubMed CentralView ArticlePubMedGoogle Scholar
- Mills N, Donovon JC, Smith M, Jacoby A, Neal DE, Handy FC: Perceptions of equipoise are crucial to trial participation: a qualitative study of men in the Protect study. Controlled Clin Trials. 2003, 24: 272-282. 10.1016/S0197-2456(03)00020-5.View ArticlePubMedGoogle Scholar
- Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, Kuderer NM, Lyman GH: The uncertainty principle and industry-sponsored research. Lancet. 2000, 356: 635-638. 10.1016/S0140-6736(00)02605-2.View ArticlePubMedGoogle Scholar
- Gifford R: Community-equipoise and the ethics of randomized clinical trials. Bioethics. 1995, 9: 127-148.View ArticlePubMedGoogle Scholar
- Djulbegovic B: Acknowledgement of uncertainty: a fundamental means to ensure scientific and ethical validity in clinical research. Curr Oncol Rep. 2001, 3: 389-395.View ArticlePubMedGoogle Scholar
- Avins AL: Can unequal be more fair? Ethics, subject allocation, and randomized clinical trials. J Med Ethics. 1998, 24: 401-408.PubMed CentralView ArticlePubMedGoogle Scholar
- Chard JA, Lilford RJ: The use of equipoise in clinical trials. Soc Sci Med. 1998, 47: 891-898. 10.1016/S0277-9536(98)00153-1.View ArticlePubMedGoogle Scholar
- Karlawish JH, Lantos J: Community equipoise and the architecture of clinical research. Camb Q Healthc Ethics. 1997, 6: 385-396.View ArticlePubMedGoogle Scholar
- Alderson P: Equipoise as a means of managing uncertainty: personal, communal and proxy. J Med Ethics. 1996, 22: 135-139.PubMed CentralView ArticlePubMedGoogle Scholar
- Elwyn G, Edwards A, Kinnersley P, Grol R: Shared decision making and the concept of equipoise: the competences of involving patients in healthcare choices. Br J Gen Pract. 2000, 50: 892-899.PubMed CentralPubMedGoogle Scholar
- Chaloner K, Rhame FS: Quantifying and documenting prior beliefs in clinical trials. Stat Med. 2001, 20: 581-600. 10.1002/sim.694.abs.View ArticlePubMedGoogle Scholar
- Schafer A: The ethics of the randomized clinical trial. N Engl J Med. 1982, 307: 719-724.View ArticlePubMedGoogle Scholar
- American College of Rheumatology: Arthritis and Rheumatism abstract supplement, 2001 Annual Scientific Meeting. Arthritis Rheum. 2001, 44: S1-S400.Google Scholar
- Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, Minaker KL, Chalmers TC: A study of manufacturer-supported trials of non-steroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med. 1994, 154: 157-163. 10.1001/archinte.154.2.157.View ArticlePubMedGoogle Scholar
- Bekelman JE, Li Y, Gross CP: Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003, 289: 454-465. 10.1001/jama.289.4.454.View ArticlePubMedGoogle Scholar
- Als-Nielsen B, Chen W, Gluud C, Kjaergard LL: Association of funding and conclusions in randomized drug trials. JAMA. 2003, 290: 921-928. 10.1001/jama.290.7.921.View ArticlePubMedGoogle Scholar
- O'Dell JR, Leff R, Paulsen G, Haire C, Mallek J, Eckhoff PJ, Fernandez A, Blakely K, Wees S, Stoner J, Hadley S, Felt J, Palmer W, Waytz P, Churchill M, Klassen L, Moore G: Treatment of rheumatoid arthritis with methotrexate and hydroxychloroquine, methotrexate and sulfasalazine, or a combination of the three medications: results of a two-year, randomized, double-blind, placebo-controlled trial. Arthritis Rheum. 2002, 46: 1164-1170. 10.1002/art.10228.View ArticlePubMedGoogle Scholar
- Montaner JSG, O'Shaughnessy MV, Schecter MT: Industry-sponsored clinical research: a double-edged sword. Lancet. 2001, 358: 1893-1895. 10.1016/S0140-6736(01)06891-X.View ArticlePubMedGoogle Scholar
- Olson CM, Rennie D, Cook D, Dickersin K, Flanigan A, Hogan JW, Zhu Q, Reiling J, Pace B: Publication bias in editorial decision making. JAMA. 2002, 287: 2825-2828. 10.1001/jama.287.21.2825.View ArticlePubMedGoogle Scholar
- Dickersin K, Min Y, Mienert C: Factors influencing publication of research results. JAMA. 1992, 267: 374-378. 10.1001/jama.267.3.374.View ArticlePubMedGoogle Scholar
- Abbot NC, Erst E: Publication bias: direction of outcome less important than scientific quality. Perfusion. 1998, 11: 182-184.Google Scholar
- Krzyzanowska MK, Pintilie M, Tannock IF: Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA. 2003, 290: 495-501. 10.1001/jama.290.4.495.View ArticlePubMedGoogle Scholar
- National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research: The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. 1978, DHEW publication no. (OS) 78-0012. Washington, DC: Government Printing OfficeGoogle Scholar
- Shuster E: The Nuremberg code: Hippocratic ethics and human rights. Lancet. 1998, 351: 974-977. 10.1016/S0140-6736(00)80046-X.View ArticlePubMedGoogle Scholar
- Halpern SD, Karlawish JHT, Berlin JA: The continuing unethical conduct of underpowered clinical trials. JAMA. 2002, 288: 358-362. 10.1001/jama.288.3.358.View ArticlePubMedGoogle Scholar
- Savulescu J, Chalmers I, Blunt J: Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability. BMJ. 1996, 313: 1390-1393.PubMed CentralView ArticlePubMedGoogle Scholar
- Lilford RJ: Ethics of clinical trials from a bayesian and decision analytic perspective: whose equipoise it is anyway?. BMJ. 2003, 326: 980-981. 10.1136/bmj.326.7396.980.PubMed CentralView ArticlePubMedGoogle Scholar
- Schulman KA, Seils DM, Timbie JW, Sugarman J, Dame LA, Weinfurt KP, Mark DB, Califf RM: A national survey of provisions in clinical trial agreements between medical schools and industry sponsors. N Engl J Med. 2002, 347: 1335-1341. 10.1056/NEJMsa020349.View ArticlePubMedGoogle Scholar
- Toroyan T, Roberts I, Oakley A: Randomisation and resource allocation: a missed opportunity for evaluating health care and social interventions. J Med Ethics. 2000, 26: 319-322. 10.1136/jme.26.5.319.PubMed CentralView ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.