Skip to content
Back to Portfolio
TAVI PinPoint - Real-time landmark tracking during transcatheter aortic valve implantation

TAVI PinPoint

Real-time landmark tracking for safer valve implantation during TAVI procedures

Client: Kemerovo Cardiology Center · Kemerovo · Russia 🇷🇺
TensorFlowPythonscikit-learnSuperviselyOpenCV

Highlights

Situation

The Kemerovo Cardiology Center needed better intraoperative guidance for TAVI procedures to reduce valve misplacement risks.

Task

Develop an AI-driven visual assistance system for real-time surgical support during transcatheter aortic valve implantation.

Action

Applied multi-task deep learning to detect and localize 11 anatomical keypoints from aortography data using ResNet and MobileNet V2.

Result

Delivered 97% accuracy and ~90 FPS speed, improving precision and enabling safer, future-ready valve implantation.

Core Team

Viacheslav Danilov

Viacheslav Danilov

Senior Data Scientist

Tomsk Polytechnic University

Tomsk · Russia 🇷🇺

Kirill Klyshnikov

Kirill Klyshnikov

Biomedical Scientist

Kemerovo Cardiology Center

Kemerovo · Russia 🇷🇺

Anton Kutikhin

Anton Kutikhin

Research Scientist

Kemerovo Cardiology Center

Kemerovo · Russia 🇷🇺

Vladimir Ganyukov

Vladimir Ganyukov

Cardiothoracic Surgeon

Kemerovo Cardiology Center

Kemerovo · Russia 🇷🇺

Evgeny Ovcharenko

Evgeny Ovcharenko

Biomedical Engineer

Kemerovo Cardiology Center

Kemerovo · Russia 🇷🇺

Overview

This project addresses one of the most critical challenges in modern cardiovascular intervention: accurately positioning the valve during Transcatheter Aortic Valve Implantation (TAVI). Traditional imaging methods offer limited intraoperative guidance, risking misplacement and complications.

To overcome this, we developed a deep learning-based visual assistance system using multi-task learning (MTL) to simultaneously detect and localize 11 anatomical keypoints on both the aortic valve and the delivery system. This system was trained on real intraoperative aortography data and evaluated across several neural network architectures. The most efficient models (ResNet and MobileNet V2) achieved impressive accuracy and responsiveness, demonstrating real-time prediction capabilities that could enhance clinical outcomes and serve as the basis for future robotic surgical systems.

Data

The solution was developed and validated using real-world surgical imaging data, making its outputs directly relevant for clinical applications. The dataset included:

  • •Source: Original aortography imaging series from 14 patients undergoing TAVI procedures (2015 – 2018) at the Kemerovo Cardiology Center
  • •Images: 3,730 grayscale images (1,000×1,000 pixels), labeled with up to 11 anatomical and device-related keypoints
  • •Annotation: Image annotation was conducted by expert cardiologists using the Supervisely platform
  • •Split: Training (80%, 2,984 images) and validation (20%, 746 images) sets
  • •Coverage: Three critical surgical stages: delivery system positioning, prosthesis deployment initiation, and partial valve release

These images captured three critical surgical stages providing a diverse and representative training ground for keypoint detection (Figure 1).

Figure 1. Visual workflow for intraoperative landmark labeling showing the annotation process and keypoint identification across surgical stages.

Methods

The project implemented a multi-task learning architecture to simultaneously classify the presence and regress the coordinates of keypoints in angiographic images (Figure 2). This approach offered more efficient representation learning compared to single-task methods.

  • Architecture: A shared feature extractor that reduced image dimensionality while preserving semantic detail, combined with a classifier for multi-label keypoint detection and a regressor for (x, y) coordinate prediction.
  • Backbones Tested: Five architectures were evaluated: MobileNet V2, ResNet V2, Inception V3, Inception ResNet V2, and EfficientNet B5.
  • Training Strategy: Fine-tuning and early stopping were employed to boost generalization and prevent overfitting.
  • Loss Functions: Log-Cosh loss for regression and binary cross-entropy for classification, balanced via weighted sum.
Figure 2. Multi-task learning workflow showing the shared feature extractor with parallel classification and regression heads.

Results

The system achieved real-time performance with high accuracy in both keypoint detection and localization. Key outcomes included:

  • •ResNet V2 (fine-tuned): Reached 97% classification accuracy and a mean absolute percentage error (MAPE) of 4.7%
  • •MobileNet V2 (fine-tuned): Achieved 96% accuracy and a 5.6% MAPE, while maintaining the highest prediction speed (~90 FPS)
  • •Model Convergence: Rapid convergence with early stopping applied around 22–76 epochs depending on architecture
  • •Performance Gain: Fine-tuned models outperformed their non-tuned counterparts across all metrics (F1-score, MAE, RMSE)

The analysis also highlighted challenges from class imbalance, particularly with underrepresented keypoints (e.g., aortic annulus landmarks), which the model handled well despite data limitations. Training convergence for selected architectures is shown in Figure 3, and prediction quality comparison in Figure 4.

MobileNet V2

ResNet V2

Inception V3

Figure 3. Training convergence of selected neural network architectures showing prediction quality evolution during training.
Figure 4. Comparison of prediction quality at the beginning versus end of training, demonstrating significant improvement in keypoint localization.

Conclusion

This project successfully demonstrates a real-time AI-based system for intraoperative landmark tracking during TAVI, offering a significant step toward safer, more precise cardiovascular surgery. The system's main contributions are:

  • •Accurate detection and localization of 11 keypoints critical to valve deployment
  • •Support for intraoperative guidance without reliance on preoperative CT models
  • •Scalability toward robotic-assisted interventions by supplying continuous, structured input for automated systems

Future directions include expanding the dataset to cover more valve models, optimizing for class imbalance, and integrating with surgical robotics for semi-automated procedures.

Keep Exploring

Image-Guided Surgery

Real-time coronary stenosis detection using deep learning for intraoperative angiographic image analysis with 0.94 mAP.

April 2021
Biomedical Research

AI-powered microscopy pipeline for detecting and counting cancer cell biomarkers using EfficientDet, achieving 85% mAP for nuclei detection.

January 2021