Real-time coronary stenosis detection using deep learning for intraoperative angiographic image analysis with 0.94 mAP.

TAVI PinPoint
Real-time landmark tracking for safer valve implantation during TAVI procedures
Highlights
The Kemerovo Cardiology Center needed better intraoperative guidance for TAVI procedures to reduce valve misplacement risks.
Develop an AI-driven visual assistance system for real-time surgical support during transcatheter aortic valve implantation.
Applied multi-task deep learning to detect and localize 11 anatomical keypoints from aortography data using ResNet and MobileNet V2.
Delivered 97% accuracy and ~90 FPS speed, improving precision and enabling safer, future-ready valve implantation.
Core Team
Overview
This project addresses one of the most critical challenges in modern cardiovascular intervention: accurately positioning the valve during Transcatheter Aortic Valve Implantation (TAVI). Traditional imaging methods offer limited intraoperative guidance, risking misplacement and complications.
To overcome this, we developed a deep learning-based visual assistance system using multi-task learning (MTL) to simultaneously detect and localize 11 anatomical keypoints on both the aortic valve and the delivery system. This system was trained on real intraoperative aortography data and evaluated across several neural network architectures. The most efficient models (ResNet and MobileNet V2) achieved impressive accuracy and responsiveness, demonstrating real-time prediction capabilities that could enhance clinical outcomes and serve as the basis for future robotic surgical systems.
Data
The solution was developed and validated using real-world surgical imaging data, making its outputs directly relevant for clinical applications. The dataset included:
- •Source: Original aortography imaging series from 14 patients undergoing TAVI procedures (2015 – 2018) at the Kemerovo Cardiology Center
- •Images: 3,730 grayscale images (1,000×1,000 pixels), labeled with up to 11 anatomical and device-related keypoints
- •Annotation: Image annotation was conducted by expert cardiologists using the Supervisely platform
- •Split: Training (80%, 2,984 images) and validation (20%, 746 images) sets
- •Coverage: Three critical surgical stages: delivery system positioning, prosthesis deployment initiation, and partial valve release
These images captured three critical surgical stages providing a diverse and representative training ground for keypoint detection (Figure 1).
Methods
The project implemented a multi-task learning architecture to simultaneously classify the presence and regress the coordinates of keypoints in angiographic images (Figure 2). This approach offered more efficient representation learning compared to single-task methods.
- Architecture: A shared feature extractor that reduced image dimensionality while preserving semantic detail, combined with a classifier for multi-label keypoint detection and a regressor for (x, y) coordinate prediction.
- Backbones Tested: Five architectures were evaluated: MobileNet V2, ResNet V2, Inception V3, Inception ResNet V2, and EfficientNet B5.
- Training Strategy: Fine-tuning and early stopping were employed to boost generalization and prevent overfitting.
- Loss Functions: Log-Cosh loss for regression and binary cross-entropy for classification, balanced via weighted sum.
Results
The system achieved real-time performance with high accuracy in both keypoint detection and localization. Key outcomes included:
- •ResNet V2 (fine-tuned): Reached 97% classification accuracy and a mean absolute percentage error (MAPE) of 4.7%
- •MobileNet V2 (fine-tuned): Achieved 96% accuracy and a 5.6% MAPE, while maintaining the highest prediction speed (~90 FPS)
- •Model Convergence: Rapid convergence with early stopping applied around 22–76 epochs depending on architecture
- •Performance Gain: Fine-tuned models outperformed their non-tuned counterparts across all metrics (F1-score, MAE, RMSE)
The analysis also highlighted challenges from class imbalance, particularly with underrepresented keypoints (e.g., aortic annulus landmarks), which the model handled well despite data limitations. Training convergence for selected architectures is shown in Figure 3, and prediction quality comparison in Figure 4.
MobileNet V2
ResNet V2
Inception V3
Conclusion
This project successfully demonstrates a real-time AI-based system for intraoperative landmark tracking during TAVI, offering a significant step toward safer, more precise cardiovascular surgery. The system's main contributions are:
- •Accurate detection and localization of 11 keypoints critical to valve deployment
- •Support for intraoperative guidance without reliance on preoperative CT models
- •Scalability toward robotic-assisted interventions by supplying continuous, structured input for automated systems
Future directions include expanding the dataset to cover more valve models, optimizing for class imbalance, and integrating with surgical robotics for semi-automated procedures.
Keep Exploring
AI-powered microscopy pipeline for detecting and counting cancer cell biomarkers using EfficientDet, achieving 85% mAP for nuclei detection.






