Ray-casting segmentation algorithm for convex anatomical structures in MRI, achieving up to 91.8% Dice score with millisecond-level runtime.

Deep Anatomy
High-precision 3D organ segmentation via V-net architecture with dense skip connections
Highlights
Cardiac surgeons at Boston Children's Hospital needed accurate 3D segmentation of anatomical structures from MRI and CT scans to guide minimally invasive procedures and improve preoperative planning.
Lead the development of a deep learning pipeline tailored for multi-organ anatomical segmentation across five clinical datasets.
Designed a V-net model with dense skip connections, integrated Test-Time Augmentation, and optimized hyperparameters using a statistical selection algorithm.
Achieved state-of-the-art Dice scores (up to 0.96), outperforming 11 benchmark models in 3 out of 5 tasks, and enabling precise, automated anatomical analysis to support surgical decision-making.
Core Team
Overview
The goal of this project was to develop an advanced machine learning system capable of segmenting complex anatomical structures in 3D medical imagery to assist in diagnostic and surgical applications. Accurate segmentation of structures like the left atrium, pancreas, spleen, hippocampus, and liver is critical for planning minimally invasive procedures and enabling quantitative biomarker extraction.
A novel V-net architecture was implemented, incorporating dense skip connections to address challenges in training deep models, especially the vanishing gradient problem. Trained and validated on datasets from globally recognized medical institutions, the model surpassed 11 leading segmentation solutions in the Medical Segmentation Decathlon. The system demonstrated consistently high Dice similarity scores, notably achieving best-in-class performance for three organs and near-top accuracy for the remaining two. These results underline the model's robustness, generalization power, and real-world clinical applicability.
Data
The training data consisted of de-identified, reformatted 3D medical imaging datasets from five prominent research centers:
- •Left Atrium: 30 MRI scans from King's College London
- •Pancreas: 420 CT scans from Memorial Sloan Kettering Cancer Center
- •Spleen: 190 CT scans from Memorial Sloan Kettering Cancer Center
- •Hippocampus: 394 MRI scans from Vanderbilt University Medical Center
- •Liver: 201 CT scans from IRCAD
All datasets were converted to NIfTI format. The images were split into training, validation, and testing sets. Preprocessing included spatial normalization and augmentation strategies to enhance generalization, especially for smaller organs or low-contrast images.
Methods
To address the challenges of organ segmentation in noisy 3D data, we developed V-net, a fully volumetric convolutional neural network based on the U-net architecture, enhanced for depth, gradient stability, and feature reuse. The V-net employs a symmetric encoder-decoder structure with dense skip-connections not only between matching levels but also within the encoder and decoder themselves. Each block includes dilated convolutions, instance normalization, ELU activations, and dropout, improving feature preservation and convergence in deep layers (Figure 1).
To further mitigate vanishing gradients and preserve low-level spatial cues, we introduced intra-block feature concatenation, allowing early-layer features to influence deeper representations. This mechanism is shown in Figure 2, which depicts our approach to dense feature transfer during both encoding and decoding.
- Training Strategy: Training used a cyclical learning rate schedule and a variable batch size strategy based on the Fibonacci sequence, accelerating convergence while avoiding overfitting.
- Hyperparameter Tuning: We applied a t-test-based selection method, reducing the search space from millions of combinations to a few hundred, significantly improving efficiency.
- Test-Time Augmentation: At inference, test-time augmentation was used to enhance segmentation stability, averaging predictions over several transformed versions of each input.
Results
The proposed V-net model with Test-Time Augmentation (TTA) demonstrated consistently high segmentation performance across a variety of anatomical structures and imaging modalities, as measured by the Dice Similarity Coefficient (DSC). Notably, the model surpassed most existing benchmarks from the Medical Segmentation Decathlon challenge.
Quantitative performance metrics on the test datasets were as follows:
- •Liver: 95.9 ± 1.0%
- •Hippocampus: 94.9 ± 1.5%
- •Spleen: 93.6 ± 1.6%
- •Left Atrium: 92.2 ± 2.7%
- •Pancreas: 82.9 ± 5.8%
These results are particularly remarkable considering the complexity and variability in organ morphology, as well as the different imaging modalities (MRI for brain and cardiac structures, CT for abdominal structures). Visual comparisons in Figures 3–7 show the model's predicted segmentation masks (in white) alongside ground truth annotations (in red) across axial, sagittal, and coronal views.
Even in anatomically challenging cases, such as the pancreas with its irregular shape and low-contrast boundaries, the model achieved strong performance, particularly benefiting from TTA, which improved both accuracy and stability.
Conclusion
This project showcases the successful deployment of a robust, scalable, and highly accurate deep learning system for 3D anatomical segmentation. By leveraging architectural and algorithmic innovations (dense skip connections, custom loss functions, and augmentation techniques), the model consistently outperformed industry and academic benchmarks. Its adaptability across imaging modalities and organ systems suggests broad utility in clinical diagnostics and surgical planning.
Future work could include real-time inference capabilities, integration into surgical navigation platforms, and expansion to other anatomical structures or multimodal datasets.
Keep Exploring
Real-time wildfire detection system for Siberian forests combining EfficientDet and CNN-RNN, achieving 95.6% accuracy at 9 FPS.





