Coronary Insight
OCT plaque segmentation with deep learning for cardiovascular risk assessment
Highlights
Cardiologists at the Kemerovo Cardiology Center needed a faster, more accurate way to analyze OCT scans for plaque vulnerability, as manual annotation was labor-intensive and prone to variability.
Develop a deep learning pipeline to automate segmentation of key plaque features and improve cardiovascular risk assessment.
Designed a hybrid ensemble of 9 neural networks with task-specific models, Bayesian hyperparameter tuning, and explainable AI.
Achieved a weighted Dice score of 88.2% across all classes, enabling fast, accurate, and interpretable plaque quantification at scale.
Core Team
Overview
Cardiovascular disease, often driven by atherosclerosis, remains the leading cause of death globally. While Optical Coherence Tomography (OCT) enables detailed imaging of plaque features, manual segmentation is time-consuming and prone to human error. This project aimed to automate plaque segmentation using a robust machine learning framework trained on real-world OCT data from 103 patients.
We evaluated nine deep learning architectures and designed a hybrid strategy combining single-class and multi-class models to account for class imbalance and feature complexity. The resulting system used ensemble learning to combine the strengths of task-specific models. It achieved a high overall Dice Similarity Coefficient (DSC) of 0.882, surpassing prior approaches. The solution not only accelerates analysis but supports more consistent diagnosis and stratification of cardiovascular risk in clinical practice.
Data
This project utilized a diverse and clinically representative multi-center, multi-scanner OCT dataset:
- •Patients: 103 individuals with stable coronary artery disease
- •Images: 25,698 RGB slices capturing arterial cross-sections
- •Plaque Features: Lumen, fibrous cap, lipid core, and vasa vasorum
- •Institutions: Data sourced from two premier Russian cardiovascular centers (Kemerovo and Tyumen)
- •Scanners: Two vendors (St. Jude Medical and LightLab Imaging) ensured imaging heterogeneity
- •Image Properties: Sizes ranged from 704×704 to 1024×1024 pixels; 215–270 slices per image
- •Annotation Workflow: Two cardiologists annotated all slices using binary segmentation masks via the Supervisely platform, with a third reviewer confirming accuracy
These annotations captured key morphological features essential for cardiovascular diagnosis and formed the foundation for model training and evaluation (Figure 1).
Methods
The project's methodology addressed both architectural optimization and class-specific learning strategies. The following techniques were applied to ensure both high performance and clinical relevance:
- Model Architectures: Nine state-of-the-art segmentation networks were tested, including U-Net, U-Net++, DeepLabV3, DeepLabV3+, FPN, LinkNet, PSPNet, PAN, and MA-Net. These were chosen for their strengths in biomedical image segmentation and complementary design philosophies.
- Hybrid Strategy: Lumen and vasa vasorum were trained using single-class models due to their dominance (lumen) or rarity (vasa vasorum). Fibrous cap and lipid core were trained with a two-class model due to overlapping morphology.
- Hyperparameter Tuning: Over 1,000 configurations were tested using Bayesian Optimization and HyperBand early stopping, focusing on encoder type, input size, optimizer, and learning rate. This tuning was performed on a representative subset of 40 patients to save compute time.
- Data Augmentation: Applied using Albumentations, including random flipping, cropping, scaling, rotation, brightness/contrast adjustment, and Gaussian noise to improve generalization.
- Validation Strategy: Employed 5-fold cross-validation without patient overlap to prevent data leakage. Training and testing progress was monitored through loss and DSC evolution (Figure 2).
- Explainability Tools: Class activation maps (CAM) like GradCAM, LayerCAM, and HiResCAM were used to visualize model attention, especially for the fibrous cap and vasa vasorum features.
Results
The hybrid deep learning framework showed consistent, high performance in accurately segmenting plaque components. Notable outcomes included:
- Lumen (magenta): DSC of 0.987 – indicating nearly perfect agreement with expert annotations.
- Fibrous Cap (blue): DSC of 0.736 – strong performance despite thin, complex structure.
- Lipid Core (green): DSC of 0.751 – reliable detection despite challenging textures.
- Vasa Vasorum (red): DSC of 0.610 – moderate performance for a rare, fine-grained feature.
- Ensemble Weighted DSC: 0.882 – demonstrating the synergy of combined models.
Visual evaluation of model predictions shows a high overlap with ground truth (Figure 3), particularly for the lumen and lipid core. Challenges remained for the fibrous cap due to its thin and diffuse boundaries. Further analysis with class activation maps confirmed that the best-performing models focused on anatomically relevant areas (Figure 4). These results establish the reliability of the segmentation models and affirm the utility of ensemble and explainable AI techniques in high-stakes biomedical imaging tasks.
Conclusion
This project delivers a powerful ML framework for automating atherosclerotic plaque segmentation in OCT scans. The hybrid segmentation design, coupled with rigorous tuning and an ensemble model, achieved high accuracy across both common and rare plaque features. The use of explainability techniques reinforces clinical trust in predictions.
Future enhancements will explore multimodal data fusion (e.g., OCT + IVUS), real-time deployment with lightweight models, and application across diverse populations through expanded datasets.