Gleam Image Learner - Validating Skin Lesion Classification on HAM10000

Contributors

Khai Van Dang Paulo Cilas Morais Lyra Junior Junhao Qiu Jeremy Goecks

Questions

• How do we validate GLEAM’s Image Learner against a published benchmark on HAM10000?

• How do we set up a balanced train/validation/test split for multi-class image classification?

• How do we interpret accuracy, weighted precision/recall, and weighted F1 for imbalanced medical imaging datasets?

Objectives

• Prepare a balanced HAM10000 subset and perform a stratified 70/10/20 train/validation/test split.

• Train an Image Learner model using a pretrained CaFormer S18 384 backbone.

• Evaluate performance using accuracy and weighted precision/recall/F1, and inspect confusion patterns.

Introduction to GLEAM Image Learner and Galaxy

• Galaxy: A web-based platform for data-intensive biomedical research, enabling users to run tools and workflows without coding.
• GLEAM Image Learner: A no-code deep learning tool for image classification in Galaxy that automates model training and evaluation.
• Goal: Validate Image Learner on the HAM10000 benchmark with a balanced split and clear, reproducible metrics.

Speaker Notes

GLEAM Image Learner simplifies deep learning by automating tasks like data preprocessing, model training with transfer learning, and comprehensive evaluation. In this tutorial, we will explore how Image Learner can be used within Galaxy to build reliable image classifiers, using the HAM10000 skin lesion dataset as a case study.

Use Case: Skin Lesion Classification with HAM10000

• Dataset Source: HAM10000 (Human Against Machine with 10,000 training images).
• Tutorial Dataset: Preprocessed, balanced subset following Shetty et al. (2022).
• Objective: Train and evaluate a multi-class classifier across seven lesion types.

Dataset Preprocessing (Balanced Subset)

• Step 1: Selection: 100 images per class from the original HAM10000 dataset.
• Step 2: Resizing: All images resized to 96×96 pixels and stored as PNG.
• Step 3: Augmentation: Horizontal flips to double each class to 200 images.
• Result: 1,400 total images (200 × 7 classes), balanced across classes.

Balanced Dataset Composition

Data Augmentation Strategy

• Horizontal Flip Augmentation:
  • Creates additional training samples by flipping images
  • Helps model learn orientation-invariant features
  • Effectively doubles training set size
  • Applied during preprocessing to build a balanced dataset

Adapted from Shetty et al., 2022 (Scientific Reports 12, 18134)

Transfer Learning with Image Learner

• Pre-trained Models: Leverage models trained on ImageNet
• Fine-tuning: Adapt pre-trained features to skin lesion classification
• Benefits:
  • Requires fewer training samples
  • Trains faster than from scratch
  • Achieves better performance
  • Especially effective for medical imaging

Image Learner in Galaxy

• GLEAM Image Learner Tool:
  • Available on Galaxy: Galaxy Main and https://cancer.usegalaxy.org
  • Automates training and evaluation of deep learning models
  • Outputs trained model and comprehensive performance report
• Key Features:
  • Automatic image preprocessing (resizing, normalization)
  • Transfer learning with multiple model architectures
  • Data augmentation options
  • Detailed evaluation metrics and visualizations

Model Configuration

Running Image Learner

1. Upload Data:
   • images_96.zip (1,400 images - 200 per class) from Zenodo
   • image_metadata_new.csv (class labels and metadata)
   • Zenodo link
2. Run Image Learner:
   • Input images: images_96.zip
   • Input metadata: image_metadata_new.csv
   • Task: Classification
   • Model: CAFormer S18 384
   • Configure parameters as shown
3. Evaluate Model:
   • Use Image Learner’s report to assess performance
   • Analyze ROC-AUC curves and confusion matrix

Image Learner Model Report

• Interactive HTML Report:
  • Config & Overview: Dataset composition, metrics, and configuration
  • Training & Validation: Learning curves and validation diagnostics
  • Test Results: Final metrics, confusion matrix, ROC/PR curves

Training Performance

• Monitor training and validation metrics
• Identify potential overfitting
• Assess convergence

Test Results and Diagnostics

• Test Performance Summary: Accuracy and weighted precision/recall/F1 on the balanced split.
• Classification Diagnostics: Confusion matrix, ROC/PR curves, and confidence distributions.
• Per-class Metrics: Heatmap view of precision, recall, F1, and related scores by lesion type.

Confusion Matrix

• Diagonal: Correct predictions
• Off-diagonal: Misclassifications
• Identifies which classes are confused with each other
• Useful for understanding model limitations

Comparison with Shetty et al. (2022)

Reference: Shetty et al., 2022 (Scientific Reports 12, 18134)

• Image Learner shows slightly lower accuracy but higher weighted precision/recall/F1.
• Differences can reflect model architecture and training/evaluation details.

Conclusion

• Key Takeaways:
  • Prepared a balanced HAM10000 subset with horizontal-flip augmentation.
  • Trained a CaFormer-based Image Learner model with a 70/10/20 split.
  • Evaluated performance using accuracy and weighted precision/recall/F1 plus diagnostics.
• Why Image Learner?
  • No coding required - accessible to all researchers
  • Automates preprocessing, training, and evaluation
  • Produces publication-ready results and visualizations
  • Enables rapid experimentation with different models

Thank you!