Real-world evidence and real-world data
The use of real-world evidence (RWE) to support regulatory and reimbursement decision-making received increasing attention over the past decades. The US FDA provides a definition of RWE as “the clinical evidence about the usage and potential benefits and risks of a medical product”1. RWE can be used across the entire drug life cycle to support regulatory and reimbursement filings, as well as the commercialization of drugs. Real world data (RWD) is the data that is collected routinely about the health of patients and is required to generate RWE. RWD is collected outside of controlled clinical trial research settings, and includes electronic health records (EHRs), registries, PRO data, claims data, laboratory and genomics data. In the generation of RWE, the RWD is typically used for purposes other than those they were originally collected for. It is also not necessarily structured for the RWE research purpose, thus there is an inherent risk for quality issues. Some examples include incomplete data on follow up, lack of the right clinical endpoints and large sets of unstructured data.
The evolution of real-world data frameworks
As a result of the growing interest in RWE, regulatory bodies, such as the FDA and the EMA, as well as health technology assessment (HTA) agencies have released guidance on the use of RWE for decision making. These RWE frameworks provide guiding principles that should be followed to generate high-quality evidence. In 2018, the FDA initiated the development of its series of frameworks with the intent to support the inclusion of RWE in regulatory decision-making about the effectiveness of drugs. Similarly, in 2024, the EMA released its framework to support the use of RWD for regulatory decision-making. The UK HTA organization, NICE, encourages the use of RWD to fill gaps in knowledge to drive forward access to innovations by patients. The NICE real world evidence framework provides guidance on when RWD can be used, and to describe best practices for the execution and reporting of RWE studies. It was released in 2022.
Data quality definitions
The RWE frameworks provide guiding principles that should be followed to generate high-quality evidence. A critical factor in the use of RWD is to ensure the data is of sufficient quality and of relevance. The data quality dimensions of the FDA, the EMA and the NICE frameworks are overlapping but do include slightly different levels of details.
The FDA RWE framework identifies two major areas of data quality: relevance and reliability2. Relevance includes the availability of key data elements and representative patients for the study, while reliability includes accuracy, completeness, provenance, and traceability.
The EMA data quality framework includes the following dimensions3:
- Reliability – does the data faithfully represent what it is meant to be?
- Extensiveness – is the data sufficient?
- Coherence – is the data analyzable?
- Timeliness – is the data available at the right time?
- Relevance – is the data of the right kind?
In the NICE real world evidence framework, data quality relates to the completeness and accuracy of key study variables4. Specifically, data relevance is determined by the data content, differences in patients, interventions and care settings between the data and the target population in the NHS, and characteristics of the data such as sample size and length of follow up.
New sources of RWD
In order to overcome RWD quality issues, other means of data collection have been pursued, for example, collecting new endpoints via wearables, longitudinal data linking and natural-language processing (NLP) for unstructured data.
Collection of health data digitally include wearables, mobile apps including fitness apps, and social media. These RWD sources enable collection of a continuous stream of data that is expansive in scope and scale that would not otherwise have been possible to collect in a controlled research environment. While fit-for-purpose data may be collected, these data sources can be subject to selection bias.
Machine learning (ML) is getting increasingly popular as a powerful tool to analyze large sets of unstructured data. It has primarily been used for predictive modeling (e.g. health outcomes) and variable selection rather than generating RWE on its own. While ML may alleviate some of the analytical challenges with traditional or new RWD sources, such as utilizing information in physicians’ notes in EHRs, it has the characteristics of being a “black box” analysis and may be unintuitive and hard to explain.
Leveraging the full value of supplementary RWD to support decision-making can be enhanced by linking clinical trial data with RWD. When patient identifiers are not available in each data source, or to preserve patient anonymity, tokenization of clinical trial data may be used to enable linking. Linking of data is not only relevant to supplement clinical trial data, linking different RWD creates a richer ecosystem of information that may overcome some quality concerns.
The future of RWE
The rapid development of new technology solutions, and the increased attention to RWE, has generated a plethora of new sources of RWD. The development of RWE frameworks has enabled better and more systematic assessment of data quality. RWE frameworks also continue to be developed in other jurisdictions than those mentioned here, and there might be a need for further harmonization in the future. Regardless, setting standards for data quality is essential to harness the full potential of RWD and RWE to inform decision-making and provide patients with rapid access to innovative healthcare technology.
Jennifer Eriksson, is Divisional Principal, Insights, Evidence & Value – Health Economics & Epidemiology, ICON Commercial Solutions.
References:
- Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory. Decision-Making for Drug and Biological Products. Guidance for Industry. U.S. Department of Health and Human Services. Food and Drug Administration. Center for Drug Evaluation and Research (CDER). Center for Biologics Evaluation and Research (CBER). Oncology Center of Excellence (OCE). August 2023.
- Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices Draft Guidance for Industry and Food and Drug Administration Staff. U.S. Department of Health and Human Services Food and Drug Administration Center for Devices and Radiological Health Center for Biologics Evaluation and Research. December 2023. Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices (fda.gov)
- Data Quality Framework for EU Medicines Regulation. EMA/326985/2023. https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/data-quality-framework-eu-medicines-regulation_en.pdf
- NICE real-world evidence framework. National Institute for Health and Care Excellence. June 2022. NICE real-world evidence framework
Filed Under: Data science, Regulatory affairs
