Skip to content
Back to Portfolio
PulmoLens - COVID-19 and pneumonia detection in chest X-rays

PulmoLens

Attention-guided deep learning for COVID-19 and pneumonia detection in chest X-rays

Client: Beth Israel Deaconess Medical Center · Boston · United States 🇺🇸
TensorFlowKerasPythonscikit-learnSHAPOpenCV

Highlights

Situation

Clinicians at Beth Israel Deaconess Medical Center lacked an interpretable, accurate tool to distinguish COVID-19 and pneumonia in chest X-rays, limiting rapid triage and diagnostic support during the pandemic.

Task

Develop a robust AI classification pipeline with visual interpretability for radiographic COVID-19 and pneumonia detection.

Action

Integrated Grad-CAM-based attention into CNN training, combining multi-source labeled data with indirect supervision to guide model focus on clinically relevant lung regions.

Result

Achieved 84% test accuracy, up from 78% after transfer learning alone, and outperformed larger custom models, while producing transparent, lung-focused visual diagnostics.

Core Team

Viacheslav Danilov

Viacheslav Danilov

Lead Data Scientist

Quantori

Cambridge · United States 🇺🇸

Alex Proutski

Alex Proutski

Research Scientist

Quantori

Hague · Netherlands 🇳🇱

Alexander Kirpich

Alexander Kirpich

Research Scientist

Georgia State University

Atlanta · United States 🇺🇸

Diana Litmanovich

Diana Litmanovich

Chief of Cardiothoracic Imaging

Beth Israel Deaconess Medical Center

Boston · United States 🇺🇸

Yuriy Gankin

Yuriy Gankin

Chief Science Officer

Quantori

Cambridge · United States 🇺🇸

Overview

This project aimed to improve the diagnostic performance and explainability of AI models for detecting COVID-19 and pneumonia from chest X-ray images. The key innovation was the introduction of an indirect supervision strategy, where Grad-CAM heatmaps guided convolutional neural networks during training to focus on relevant lung regions.

To overcome data scarcity and inconsistency, we harmonized and relabeled data from five major public datasets. We trained and evaluated multiple standard network architectures, progressively refining them in three stages: transfer learning, full fine-tuning, and indirect supervision. Our experiments showed that standard architectures like VGG-16 not only matched but outperformed larger, custom COVID-Net models while maintaining transparency in predictions. This approach has significant implications for rapid, scalable AI-assisted diagnosis in clinical settings.

Data

The dataset consisted of 2,631 chest X-ray images, sourced from five publicly available datasets:

Images were manually labeled into three consistent classes: Normal, Pneumonia, and COVID-19. To ensure class balance, duplicates were removed and class-specific quotas were enforced. Data was split into training (80%), validation (10%), and testing (10%) sets. Only anterior-posterior (AP) and posteroanterior (PA) images were retained to maintain diagnostic relevance.

Methods

The project's core innovation lies in a three-stage training pipeline that combines conventional transfer learning with indirect supervision, enabling more focused and interpretable AI predictions for medical imaging:

  • Stage I (Transfer Learning): We initialized ten convolutional neural networks (CNNs) such as VGG-16, MobileNet V2, and EfficientNet variants using ImageNet weights. We trained only the classifier heads while keeping the feature extractor layers frozen. Bayesian optimization was used for hyperparameter tuning, and regularization techniques (dropout, L2 regularization, early stopping) were employed to reduce overfitting.
  • Stage II (Full Fine-Tuning): The top-performing models from the first stage (VGG-16, MobileNet V2, EfficientNet B1, B3) were fully unfrozen and retrained, allowing both feature extractors and classification heads to be optimized together at a reduced learning rate.
  • Stage III (Indirect Supervision): This stage introduced the key novelty: an indirect supervision mechanism guided by Grad-CAM (Gradient-weighted Class Activation Mapping). During training, attention heatmaps were generated and used to create soft masks that emphasize the lung areas most relevant for diagnosis. These masks were applied to input images, and the network was trained using a dual-loss function that combined traditional classification loss with an attention loss. This encouraged the network to learn not just what the correct label is, but also where to look when making predictions.
Figure 1. Indirect supervision workflow showing Grad-CAM attention map integration, shared weights, and attention-based masking during backpropagation.

Results

The evaluation of model performance proceeded in three distinct training stages, with each phase demonstrating progressive improvements in classification accuracy, robustness, and clinical relevance:

  • Stage I (Transfer Learning): MobileNet V2, VGG-16, and the EfficientNet B1/B3 models emerged as the top performers, achieving validation accuracies of 79–80% and testing accuracies of up to 78%.
  • Stage II (Full Fine-Tuning): VGG-16 and MobileNet V2 showed the most significant improvements, with validation accuracy increasing by +9% and +6%, respectively. VGG-16 reached 87% validation and 82% test accuracy.
  • Stage III (Indirect Supervision): Networks trained with attention masks demonstrated more anatomically accurate focus on lung regions. VGG-16 achieved the highest accuracy overall: 88% validation and 84% test accuracy, outperforming the best COVID-Net variant (CXR-4A) which peaked at 81% accuracy despite being over 11× more complex.

Performance Summary:

  • •VGG-16 (Stage III): 88% validation, 84% test accuracy
  • •MobileNet V2 (Stage III): 86% validation, 79% test accuracy
  • •CXR-4A (COVID-Net): 81% accuracy on both subsets
  • •Improvement: VGG-16 validation accuracy rose by 9 points across the three stages, from 79% to 88%

The integration of Grad-CAM not only improved classification but also made the predictions interpretable. Attention maps confirmed that the networks were focusing on clinically relevant lung zones, validating the effectiveness of indirect supervision as a lightweight yet powerful enhancement to conventional CNN training in medical imaging contexts.

Figure 2. Grad-CAM attention maps for COVID-19 detection showing model focus on clinically relevant lung regions affected by the disease.
Figure 3. Grad-CAM attention maps for pneumonia detection demonstrating anatomically accurate focus on affected lung areas.

Conclusion

This project demonstrates that indirect supervision with attention guidance can substantially improve both performance and transparency in medical image classification. VGG-16, a standard CNN, surpassed more complex models in identifying COVID-19 from chest X-rays, proving that smart supervision can outperform brute-force architecture scaling.

The method offers a scalable, interpretable AI solution for radiology, with future extensions possible for other diseases and imaging modalities.

Keep Exploring

Medical Imaging

GPU-accelerated 3D anonymization pipeline for CT and MRI scans, detecting and blurring facial/ear regions while preserving 100% of brain anatomy.

September 2021
Image-Guided Surgery

Real-time landmark tracking for safer valve implantation during TAVI procedures using multi-task deep learning with 97% accuracy.

July 2021