The Medtronic GI Genius’ ColonPro app is one of the best examples of how
artificial intelligence
in medtech can save lives — and Cosmo Intelligent Medical Devices deserves much of the credit.
The Cosmo Pharmaceuticals division developed the neural networks that allow ColonPro to spot signs of colon cancer that physicians might miss during a colonoscopy. It was the first and only AI system for colonoscopy when it won FDA de novo clearance in 2021.
This year, Medtronic and the Cosmo AI team unveiled the latest version, which improved polyp detection, included new features like automated procedure highlights, and opened the GI Genius operating system up to third-party software developers.
Medical Design & Outsourcing interviewed Cosmo Intelligent Medical Devices President Nhan Ngo Dinh and Science, AI and Data SVP Andrea Cherubini to learn more about the system’s development and what tips and advice they had for other device developers.
(Editor’s note: While MDO prefers live interviews to offer readers medtech expertise without the filter of marketing or public relations staff, we conducted this Q&A by email through a Medtronic representative because Cherubini and Ngo Dinh live and work in Italy. The following has been lightly edited for space and clarity.)
MDO: How did you improve the system’s ability to detect polyps?
Cherubini:
“We improved the system’s ability to detect polyps by combining innovative AI training strategies, advanced machine learning algorithms, and expert annotations from gastroenterologists.
“First, we significantly expanded our training dataset for colonoscopy procedures, doubling the number of patients used for training our AI. This was achieved through partnerships with 50% more endoscopy centers worldwide, providing a broader range of images and scenarios to train on.
“Second, we focused on understanding why lesions were missed during colonoscopies. We leveraged generative AI techniques to artificially augment these situations, simulating the types of errors that human clinicians might make when performing colonoscopies.
“Finally, we conducted rigorous tests to validate the system’s performance and ensure its reliability in different clinical settings. As a result of these steps, GI Genius ColonPro next-generation software for polyp detection became more responsive and achieved a 9% reduction in false positives.”
MDO: What were the biggest technical challenges of this project and how did you overcome them?
Ngo Dinh:
“One of the significant technical challenges of GI Genius was developing an AI system that could consistently identify polyps of varying sizes, shapes, and appearances. To overcome this, we implemented a state-of-the-art deep learning architecture, enhancing the system’s ability to detect subtle differences in tissue structure.
“Another challenge was ensuring the system’s robustness against variations in endoscopic equipment and procedure techniques across different countries. We addressed this by extensively testing the system with different types of endoscopes and training it on data collected from multiple sources and a diverse population of patients.
“Additionally, the need for real-time decision support required optimizing our algorithms for low-latency performance, which we achieved by designing a high-performance computing platform for AI.”
MDO: What are some metrics that illustrate the size, scope and impact of this project?
Ngo Dinh: “The AI model was trained on data from over 1,000 patients from all over the world, ensuring optimal performance. GI Genius has been featured in over 30 peer-reviewed scientific publications and eight randomized controlled trials, establishing a solid base of scientific evidence. With a 14.4% increase in adenoma detection rate and a 50% reduction in adenoma miss rates, the clinical impact is clear. It has the potential to impact 3 million patients globally every single year, providing significant benefits in gastroenterology. The technology is available in over 20 countries, demonstrating its extensive global distribution. We have released four software iterations, and the most recent version uses 11 parallel AI algorithms, reflecting continuous improvements. Indeed, GI Genius has rightfully earned its reputation as a synonym for innovation in gastroenterology.”
MDO: What are the major technological advances that made the latest version of this system possible?
Cherubini: “Implementing AI for real-time video summarization and understanding involves several technological hurdles. Indeed, processing the immense amount of data generated by video streams requires efficient algorithms to manage the data load and high-performance computing resources. Another challenge is developing robust AI models that combine image understanding across the temporal dimension effectively. This combination is crucial for capturing the context and sequence of events, which is essential for accurate video summarization. Addressing these challenges requires a comprehensive approach, combining sophisticated AI models and rigorous validation processes.”
MDO: How many neural networks does ColonPro software use, what are they and what are the major similarities and differences of them all? Which were the easiest to build, which were the most challenging, and why?
Cherubini: “ColonPro software is the fourth-generation software update that now integrates 11 specialized neural networks, each tailored to perform distinct tasks that collectively enhance the efficacy of colonoscopy procedures. Among these are networks dedicated to detecting polyps in individual image frames, clustering detections across multiple frames to consolidate views of the same polyp and classifying image frames by quality. Other networks focus on interpreting the endoscopist’s actions at any moment, assessing bowel cleanliness, and pinpointing the scope’s location. Additionally, AI networks are employed to identify the start of the withdrawal phase, detect anatomical landmarks, and synthesize this data to generate key performance indicators. Networks handling straightforward visual tasks, like polyp detection and image quality classification, were easier to develop due to established image processing methodologies. Conversely, networks that required understanding the procedural context, such as tracking the scope’s location and estimating cleanliness, were more challenging due to the need for sophisticated temporal and spatial analysis.”
MDO: How did you identify the key quality metrics and develop the ability to measure them?
Ngo Dinh: “In designing the ColonPro software, we identified key quality metrics by closely following the recommendations from the American Society for Gastrointestinal Endoscopy (ASGE), American College of Gastroenterology (ACG), and other international gastrointestinal societies dedicated to enhancing colonoscopy quality. These organizations highlight the need for specific intraprocedural indicators to be monitored and analyzed over time. Metrics such as bowel preparation quality, cecal intubation rate, and withdrawal time were selected based on their proven impact on procedural success and patient outcomes. The Procedure Highlights function in ColonPro was engineered to capture these indicators seamlessly during procedures, providing actionable data for healthcare organizations. This feature not only aids in meeting regulatory standards but also promotes continuous improvement and excellence in colonoscopy practices through detailed statistical analysis. To accurately measure these metrics, we worked in close collaboration with gastroenterologists globally, who provided invaluable feedback and clinical expertise.”
MDO: What are some potential applications for GI Genius beyond Colon Pro?
Ngo Dinh: “The GI Genius module, coupled with the AI Access program, can revolutionize the real-time analysis of medical videos in gastroenterology and beyond. Potential applications could target detection or therapeutic guidance for a variety of diseases within endoscopic care. The essential requirement is a video feed interpreted by a physician, showcasing the broad applicability of the GI Genius system across various medical disciplines.”
Look for more from Cherubini and Ngo Dinh at MDO in the weeks ahead, including how device developers can work with Cosmo on GI Genius apps, tips for using AI in medtech, and how Cosmo and Medtronic got their project across the finish line as fast as they did.