Let’s be clear: a well-designed and well-conducted Phase III drug trial that has very few or no missing data and robust statistics but does not demonstrate positive efficacy and safety for a potential new therapeutic agent is not a “failed trial.” It’s a good trial with a negative outcome. On the other hand, a poorly designed and executed trial with weak statistics but a nominally positive outcome qualifies as a “failed trial” because it represents a waste of time and effort—it will not survive the scrutiny of regulatory or peer review.
Publication Bias
Largely because of a long tradition of publication bias, there are not a lot of data on negative-outcome trials. “Publication bias” encompasses both lack of enthusiasm for the time and effort of publishing the details of a negative or failed trial on the part of the sponsor and investigators and lack of interest in devoting page space to negative-outcome trials on the part of journal editors and publishers. Fortunately, support for publishing negative-outcome trials has expanded substantially in the past few years as the stakeholders in the drug development process realize the importance of sharing this information.1
Negative Outcomes in Earlier Trials
In Phase I-II trials, negative trial outcomes as high as 70 percent are to be expected. These early phase and proof-of-concept trials are truly “experimental,” and no one expects every new compound to progress to Phase III. In contrast, the current 50 percent negative outcome rate in Phase III trials adds substantially to the cost of drug development and may have important financial consequences for the sponsors.2
Inadequate Efficacy and/or Safety Issues
In a high level analysis, most drugs were discontinued in Phase III due to inadequate efficacy (44 percent) or safety issues (24 percent). These data suggest that the majority of negative outcomes likely reflect intrinsic properties of the study drug.1
Six Key Drivers of Failure
Looking in detail specifically at “failed trials,” Grignolo and Pretorius proposed six key “drivers of failure”2:
- Inadequate basic science
- Suboptimal dose selection (inadequate dose finding studies or narrow therapeutic index)
- Insufficient assessment of the current standard of care and disease area landscape
- Flawed study design
- Flawed data collection/analysis
- Study operations problems
Flawed Study Design
The first three drivers on this list clearly fall into the sponsor’s area of scientific responsibility in terms of the decision to embark on a Phase III study. The latter three, on the other hand, are critical issues for the clinical trialists. From the trialists’ viewpoint, the primary concern is generating data that accurately reflect the safety and efficacy of the study drug in the target population. With this in mind, there is a strong case to be made for flawed study design as the critical issue. Careful study design should mitigate potential flaws in data collection and operations, particularly with regard to missing data.
What are the potential flaws in study design that trialists should keep in mind and look for critically?
1. A weak statistical analysis of the Phase II results.
Remember that achieving nominal statistical significance (p = .05) leaves a 1 in 20 chance that the positive Phase II results are due to chance. How strong are the data, really?
2. The Phase II surrogate endpoint is not congruent with an acceptable approval endpoint.
For example, in cancer trials the “rate of stable disease” and the “clinical benefit rate,” two measures increasingly reported in early phase studies, have never been shown to correlate with outcomes yet are regarded by many as measures of efficacy. Inaccurate assessments of limited data and reliance on endpoints that have not been validated are likely more important than overoptimism.3
In the cardiovascular/metabolic area, a recent FDA advisory committee stressed that low-density lipoprotein (LDL) cholesterol levels were not a reliable surrogate for cardiovascular benefit, and that timely completion of outcome trials was imperative. Among cardiovascular drugs for which the development was halted in Phase 3, most were discontinued due to inadequate efficacy, underscoring the importance of rigorous trials in elucidating the risk–benefit balance of new treatments.4
3. The protocol allows inclusion of non-responders in Phase III.
Trialists are under pressure to design trials that are as large as possible, enroll as quickly as possible, and will achieve the broadest possible indication for the drug. This can lead to overly broad inclusion criteria, allowing subjects with a low probability of favorable responses to sink the trial. The recent TOPCAT trial is an excellent example of this problem.
4. There is potential for “reversion to the mean.”
Phase III trials that require “hard clinical endpoints” like survival often include subjects with multiple serious comorbidities. A positive short-term therapeutic response may not be detectable in longer-term studies due to overall reversion to the mean in the entire study population. (See chapter 11 of Nesiritide: The Rise and Fall of Scios for a detailed example.)
In summary, good trial design cannot turn a poor drug into a good one. However, attention to good trial design does mean that a potential therapy will undergo a meticulous evaluation that clarifies its risks and benefits in the targeted disease state.
References
- Hwang TJ, Carpenter D, Lauffenburger JC, et al. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med. 2016 Oct 10. doi: 10.1001/jamainternmed.2016.6008. [Epub ahead of print].
- Grignolo, A, Pretorius S. Phase III Trial Failures: Costly, But Preventable. Applied Clinical Trials 2016; 25: 36-42.
- Amiri-Kordestani L, Fojo T. Why do phase III clinical trials in oncology fail so often? J Natl Cancer Inst. 2012;104:568-9.
- Hwang TJ, Lauffenberger JC, Franklin JM, Kesselheim AS. Temporal Trends and Factors Associated With Cardiovascular Drug Development, 1990 to 2012. JACC: Basic to Translational Science 2016; 1:301-308.
About the Author
Roger M. Mills, MD, is a retired cardiologist and an independent writer in the pharmaceutical industry.
Follow us on Twitter and Facebook for updates on the latest pharmaceutical and biopharmaceutical manufacturing news!
Filed Under: Drug Discovery
