With the FDA’s introduction of new guidelines surrounding the use of real-world evidence (RWE) in medical device regulatory decisions, FDA Commissioner Scott Gottlieb advances the argument for the utility of RWE. In fact, the FDA is currently considering the role of RWE in evaluating pharmaceutical treatments. Despite much debate over what part RWE should play in regulatory approval and payer coverage decisions, many of us are left asking two fundamental questions: 1) Why do we need RWE?; and 2) If RWE is needed, how can we use it?
Why do we need real-world evidence?
Randomized, controlled trials (RCTs) are the bedrock of measuring treatment efficacy and safety. By randomizing participants to two or more separate treatment arms, researchers can identify key causal parameters, such as treatment efficacy and safety. Findings from large RCTs are considered the highest level of evidence by most scientific organizations.
Yet RCTs are not without their flaws. First, the findings from RCTs—while valid within the constructs of the trial—may not be relevant to real-world practice. The issue of external validity occurs when real-world treatment practice differs from that of the RCT, or when the patient population most likely to use the treatment in the real world differs from the RCT population. For instance, treatment efficacy in RCTs may be overstated if real-world adherence is significantly worse than adherence in the trial. Second, for rare disease treatments and narrowly targeted precision medicine interventions, recruiting a sufficient sample size to randomize across multiple treatment arms may be problematic. More generally, collecting data prospectively within a clinical trial may be cost-prohibitive in many cases.
Additionally, RWE can supplement the evidence base beyond the “limited and rigidly constructed circumstance” of the clinical trial. One area where RWE is already extensively used is in measuring drug safety. The FDA, for instance, already uses real-world data to measure pharmaceutical safety as part of its Sentinel Initiative.
Quality measurement is another area where RWE is often used. Payers and life sciences firms will use RWE to inform outcomes-based pricing contracts, such as the outcomes-based contract between Novartis and CMS for the new CAR-T therapy Kymriah. Under value-based purchasing agreements, provider reimbursement rates are often linked to quality-of-care metrics collected from real-world data.
How can we use real-world evidence?
A recent report from Duke University identified three categories to consider when implementing an RWE study: clinical context, data selection, and study method.
Clinical context. Clearly, use of RWE is not appropriate to measure treatment efficacy for an initial indication for the vast majority of treatments. However, RWE may be sufficient for securing a new drug application (NDA) for certain rare diseases where sample sizes are insufficient to power two arms of a trial. Clinical trials are often relatively short in duration; Phase 4 clinical trials using real-world, observational data can help stakeholders understand a treatment’s long-term efficacy and safety. RWE could also be used to evaluate effectiveness and safety of an approved drug for a new indication.
Data selection. Once life sciences firms understand the clinical context and key research questions to answer, the next step requires identification of relevant data sources. Commonly used real-world data sources include administrative data (e.g., health insurance claims), electronic health records (EHRs), and registry data. The increasing use of remote patient monitoring and smartphone-enabled technologies offers the promise of even more detailed real-world information.
Each of these data sources has advantages and limitations. Administrative data comprises detailed patient information, but may be incomplete if patients receive care from other provider networks or health plans. EHRs offer rich, real-world clinical information, but data quality—while improving rapidly—is often lacking due to missing data or the use of non-standardized free text fields. Further, syncing EHR data across different systems has proven difficult without universal standards. While registry data can be designed prospectively to collect a wide variety of real-world data, enrollment in registries may not be representative of the wider group of patients with the disease; patient attrition is also a concern.
Study method. A robust study design must be applied to real-world data. Many researchers are skeptical of RWE due to a lack of randomization in a controlled setting. Randomization, however, can be incorporated into real-world study designs. Pragmatic trial designs blend randomization with real-world treatment practices. Other studies use cluster randomization—where different provider sites are randomized to receive the intervention—to assess treatment efficacy in a real-world setting. When randomization is not possible (for instance, when using retrospectively rather than prospectively collected data) advanced statistical techniques such as instrumental variables and regression discontinuity designs can help solve the problem of causal inference.
Where do we go from here?
Use of RWE should not be seen as entirely good or entirely bad. RWE has its uses, advantages, and limitations, which this article details briefly. While RWE will not replace clinical trial data for many purposes, RWE can supplement clinical trial evidence to become a useful tool for healthcare stakeholders to make better regulatory, policy, coverage, and treatment decisions.
Jason Shafrin, PhD, is a senior research economist with Precision Health Economics and the director of research at the Innovation and Value Initiative.
Filed Under: Drug Discovery
