In July, the life sciences services company Eversana revealed it was partnering with AWS to tap generative AI (gen AI) for pharma and other life science customers. It aimed to “pharmatize” gen AI, as the company put it then. Now, the company has revealed the first technology from that partnership – a regulatory review automation system using natural language processing and gen AI services on the AWS platform.
To arrive at its first area of focus, the company considered use cases that were “really practical,” offering “great opportunities for efficiency gains,” but without high risk to patients or healthcare professionals, said Scott Snyder, Eversana’s chief digital officer. The company used a version of the three R’s — responsibility, reliability and ROI. The first chiefly involves “making sure you really understand this is something you’re going to take ownership of, be accountable for, and do it in the right way,” Snyder said. Eversana has its own responsible development guidelines for AI in general.
Scott Snyder
The second R, reliability, is also critical. “You have to make sure that the generative AI for the use case you’ve picked, with the datasets and content you have, can deliver the reliability or performance you need. Can you live with a 2% error rate, or can you handle a certain amount of false positives or hallucinations, especially if you have a human in the loop trained to catch them?”
Then third is ROI is also pivotal in a often-hyped field. “I think a lot of people are chasing tons of pilots like shiny objects, but the question is, will it actually deliver the payback? It’s easy to say, ‘Well, you know, we’re going to be able to do software coding 50% faster,’ but if you then have to add three QA people to check what the AI came up with, are you really offsetting your gains?” Snyder said.
Where AI tools come in is “taking those digital documents, going through them, and annotating them, saying like, ‘Here’s where there’s a potential claim in this piece of content.’ So, automation could really happen in the manual identification and linking, as well as building the claims database, linking to potential claims with a risk score, and doing the annotating automatically. All these steps could potentially take out a massive amount of hours,” Snyder said.
Inside Eversana’s AWS partnership to slash regulatory red tape
With those considerations in mind for its partnership with AWS, the company homed in on medical regulatory review. “It’s such a time-consuming effort on the agency side. It’s thousands of hours, in some cases for certain brands, especially for that initial review of the set of content or library of content that you’re using around the brand. We got a lot of great feedback from our client base, saying, ‘Yeah, that’s a really important problem,’” Snyder said.
Examples of the initial areas of focus include AI-based approaches to streamline time-consuming medical and regulatory review processes. Specifically, the company aims to tap Amazon Bedrock and Textract to help automate these traditionally labor-intensive and error-prone compliance operations. Processes such as verification of clinical trial documents, scientific references, labeling and package compliance, and adverse event report analysis can take thousands of hours when done manually.
“The regulatory review process involves not just the medical team, but also other teams, because you’ve got the marketing team assembling the content library, which needs to be structured in a certain format for the reviewers,” Snyder explained. Added to that are employees assembling data around the scientific claims and references.
AI’s role in document annotation and training human reviewers
AI can annotate the documents, indicating potential claims within the content. This capability facilitates the manual identification and linking process, as well as the creation of a claims database, assigning risk scores to potential claims, and automating the annotation process. Implementing these steps could significantly reduce the number of hours required for these tasks,” Snyder said.
“You’re never going to eliminate the need for human reviewers, at least in the foreseeable future. But training the reviewer to be more attuned to where these models could go awry is going to be really important,” he added. Human reviewers will need to become more attuned in critically reviewing those documents in a different way now that AI has been flagging things that were normally flagged by human beings before.”
AI as a powerful training tool in regulatory review
In addition, AI can be an “amazing training platform” for things like regulatory review, Snyder said. It can help train reviewers and personnel by surfacing more potential issues and edge cases to validate against. For example, it can “generate synthetic claims and false claims in different content types that human reviewers then need to analyze and identify,” Snyder added. This allows “much more effective and scalable training of reviewers compared to manually creating training datasets.”
Ultimately, the AI becomes a feedback loop. “Reviewers validate the AI’s outputs, which further improves the model, generates new test cases, and so on,” Snyder said. “Training reviewers to spot false positives or errors from the AI system will be important,” allowing them to become more attuned to places where the AI could go awry and ensure they are critically evaluating the content.
The conversational nature of asking AI questions makes the training process more “interactive and engaging.” In the end, AI training platforms “optimize human expert time while generating robust, personalized training data.”
Filed Under: Data science, Drug Discovery and Development, Industry 4.0, machine learning and AI, Regulatory affairs
