As stated in the International Conference on Harmonisation (ICH) E9 guideline, when developing a drug it is necessary to find out at which dose(s) and schedule it is both safe and effective. To achieve this usually requires a series of clinical studies, each with its own specific objective. Interpretation and assessment of the total evidence from the programme of studies involves synthesis of evidence from the individual studies via integrated analyses. In this paper we will discuss the planning, creation and timing of integrated analyses, along with common pitfalls and solutions.
Step 1: Complete the FDA-Required Efficacy and Safety Summaries and Integrated Assessments
As per ICH M4E, The Common Technical document, a Summary of Clinical Efficacy (SCE), Summary of Clinical Safety (SCS) and Integrated Summary of Efficacy (ISE) and Integrated Summary of Safety (ISS) are all required when a sponsor seeks marketing approval for a new drug. The intent of the Clinical Summary, SCE and SCS sections are to discuss study design and general efficacy and safety results, which the biostatistician needs to contribute to the writing of. The sections need to be mostly text with some in-line tables and figures. These sections are usually derived from the full ISE and ISS sections located in 5.3.5.3.1/2, which were designed to contain the more detailed in-depth analysis. That is, for most applications the sections listed above should all be distinct. The SCE/SCS and ISE/ISS meet separate and distinct requirements. For some indications, however, such as Orphan Drug with small studies, they may be very similar.
The clear relationship between the approach described in SCE/SCS and ISE/ISS have led to confusion that the sections can be combined. Sponsors have also adopted this approach in order to streamline their global regulatory submission efforts (particularly in the light of the fact that many regions do not require an ISE or ISS). However in order to expedite the review process and to minimize the risk of receiving a “refuse to file” letter, it is strongly recommended the all the above sections as described are completed.
Step 2: Plan for comprehensive detailed analyses of integrated summaries of efficacy and safety
Drug development spans many years and during the course of a development program significant data are accumulated. The ISE/ISS integrated analyses provide the opportunity to establish the robustness and consistency of findings that are observed in individual trials, and to further substantiate the benefits and risks of the drug. Integrated analyses enable sub groups gender, baseline disease severity) and other factors (exposure) to be explored that may influence outcomes. For example, the analysis of death data, serious adverse events and other significant adverse events that occur infrequently in individual studies need to be investigated by a variety of different factors to tease out important associations at the integrated level.
Potential associations need to be investigated from both an efficacy and safety perspective, and the findings ultimately used to argue the benefit-risk profile of the drug. As per individual studies, detailed integrated efficacy and safety analysis plans need to be prepared as early as possible in the phase III development phase. It is not uncommon for many sponsors to view the integrated analyses as a repeat of the study level analyses, and as such miss out on important aspects such as exploring the effects and risks in a variety of different patients with different characteristics. Furthermore, integrated analyses are always on the critical path, and as such there is increased reluctance to delay a submission whilst scientists explore the integrated data for potential issues, such as unexpected risks. However, the analyses are critical since they cannot be undertaken at the study level. The failure to inadequately explore the integrated efficacy and safety data often leads to more regulatory questions so cutting corners is ultimately often a fallacious time-saver. Another common mistake is to look at efficacy and safety in isolation. Benefit risk can only be assessed when the patient population is consistent across efficacy and safety. Thus it is imperative that subgroups analyses based on baseline characteristics are consistent and ultimately used jointly in the final assessment of benefit risk. Once the Integrated Efficacy and Safety Analyses work is completed and fully documented, the next task is to take the key findings and derive the SCE and SCE sections.
Issues to address to facilitate the smooth pooling of data at the end of a development program
- Do individual studies comply with CDISC requirements (SDTM and ADaM)?
- Has a single dictionary been used? If older dictionaries were used, do any terms need to be re coded to ensure consistency of terms?
- Have common algorithms been deployed across studies?
- Have appropriate Quality Control practices been applied across the program of studies, and any Integrated Summaries? Have Integrated Summaries been cross checked against individual studies?
- Have all clinical trials in the development program been considered for the integrated summaries? If any are excluded, are reasons defendable?
- Are the overall Integrated Summary conclusions supported by the individual CSR observations?
Step 3: Adopt CDISC standards for all studies
SDTM and ADaM implementation at the study level facilitates consistency in reporting, and easy aggregation at the integrated summary level. It is not uncommon to find legacy studies that need to be mapped for integration, since drug development spans many years. These need to be mapped as early as possible in the development program. Consideration also needs to be given to different dictionary versions during this process. Other aspects that need attention is the format of .xml files, etc. All too often legacy studies are an afterthought, and not given enough attention early in phase III. Conversion plans need to be developed up front, off the critical path, to ensure the smooth pooling of data.
Contents of a Master Analysis Plan
- Target Product Profile
- Development Plan objectives
- Trials covered
- Difference and similarities across studies
- Statistical methods, including
- Visit windowing
- Missing data assumptions
- Populations
- Primary and key secondary efficacy endpoints (including derivations, and analysis method)
- Study medication/compliance algorithms
- Subgroups interest
- In-text and post-text sample table, listing and figure shells
- Data and naming conventions, layout location, treatment ordering, etc.
Step 4: Build in quality by design
The approvability of a product depends on the data that are selected for collection during the planning of the clinical trials. Each trial needs to be designed with a view to pooling at the end of the development program. However, often integrated summaries of efficacy and safety are only considered as the last phase III study completes. Considerable resource, time and money can be saved by carefully planning how data will be structured and integrated at the beginning of each study and whether it fits in with the overall development program strategy. That is, ensuring each clinical trial collects the right data in the correct format and structure the first time to ensure smooth pooling at the end. Table 1 lists some of the key issues that commonly inhibit the smooth pooling of data at the end of a development program.
From a biostatistical perspective, to facilitate the smooth integration of data from individual studies it is important that a master analysis plan (MAP) is prepared and maintained throughout the development process. The intent of the MAP is to document project standards for statistical analysis and programming. The contents of a MAP are very similar to that of a statistical analysis plan (SAP) for an individual study. Table 2 summarizes the typical contents of a MAP. Individual CSR SAPs and Integrated Safety and Efficacy SAPs are then derived from the MAP.
To ensure the analysis strategy meets with regulatory approval, early discussions with the relevant agency concerning integration requirements are encouraged. These can be achieved through scientific review meetings within Europe or end of phase II meeting with the FDA. Early involvement of medical writing to shape the organization of the integrated reports for section 5.3.5.3 is recommended.
From a programming perspective it is also imperative that quality is built in by design when supporting the integrated analyses. Before tables are generated, integrated datasets based on ADaM need to be generated. As an example of how quality can be built in, one approach is to subset these integrated datasets by individual study and then electronically cross compare with the datasets used to for the individual CSRs. This reduces the risk of inconsistencies between the integrated results and those summarised in individual CSRs. Where differences exist, explanations should be provided. Failure to build in quality by design may delay the approval process and result in lost revenue.
Step 5: Assess and update benefit risk
In the past, regulatory authorities have approved drugs that are demonstrated to be safe and efficacious. Safe has historically been interpreted as “benefits” outweighing the risk. The benefit-risk assessment has often been variable and subjectively based. However, formal structured benefit-risk analyses are becoming increasingly important in regulatory submissions. One of the more commonly used frameworks is the BRAT (Benefit Risk Action Team) approach. (The steps involved are outlined in Figure 1). The framework is a set of processes and tools for selecting, organizing, summarizing and interpreting data that is relevant to decisions based on benefit-risks. It provides a standardized yet flexible platform for incorporating study outcome and preference weights as well as communicating the rationales for decisions.
BRAT methodology for assessing Benefit Risk
The M4E EWG is revising Section 2.5.6 “Benefits and Risks Conclusions” of the ICH M4E guideline to standardize the content and presentation of benefit risk in regulatory submissions. The statistician has a key role to play in working with the teams in selecting the appropriate methodology for assessing benefit–risk. As with ISE and ISS, planning for any formal benefit risk analysis starts at the end of phase II, and should be continuously updated throughout phase III. Benefit-risk assessments should be updated when new study results are available.
Bringing a new drug to market spans many years and involves significant resources. Individual study reports are supplemented by integrated analyses and summaries. In fact it arguably that these higher level reports are the most critical since they provide assessors with the succinct overview of the key findings associated with the development program. Subsequently it is imperative that integrated analyses and the reporting of such findings are comprehensive and in line with regulatory requirements. Understanding the challenges associated with integrated analyses and summaries and how to mitigate them will reduce delays in the review process and move effective drugs to market faster.
Filed Under: Drug Discovery
