251: Pathology Foundation Models: PathBench, Scanner Shift, and the HER2 Slide Swing

Send us Fan Mail

What if changing the scanner or choosing another slide from the same tumor changes your pathology AI results?

In DigiPath Digest #52, I review three studies on pathology foundation models, scanner shift, and slide selection to ask how much of “model performance” comes from the model itself.

We look at a benchmark comparing 32 foundation models across 41 tasks, a study testing 10 models across scanners and centers, and a breast cancer study examining how predictions change when different slides represent the same patient.

The findings challenge familiar assumptions about bigger models, benchmark rankings, and the idea that one slide can represent a heterogeneous tumor. Combining information from multiple slides improved performance in the breast cancer study, without changing the model architecture.

My takeaway: evaluate the model, the data, and the workflow together. I share three practical questions to ask before accepting an AI benchmark score.

Highlights with Timestamps

• 00:42 — PathVisions updates and plans for conference activities.
• 05:09 — Trailblazer podcast interviews: sharing real implementation experience.
• 08:02 — How much of foundation model performance comes from the model?
• 13:16 — The 32-model benchmark and closely grouped leading scores.
• 16:06 — Why bigger models do not consistently perform better on unfamiliar data.
• 18:58 — Scanner shift and testing models on scanners held out during training.
• 22:18 — Why AUROC alone can miss important performance losses.
• 24:55 — Use-case-specific validation and the “pixel pathway” from scanning to diagnosis.
• 28:37 — Slide selection, tumor heterogeneity, and HER2 prediction.
• 33:46 — Combining slides from the same patient: performance gains and computational cost.
• 35:31 — Three questions to ask before trusting a benchmark.
• 37:27 — Audience Q&A on available datasets and checking AI-generated answers.
• 40:39 — Plans for the Digital Pathology Club and more accessible expert guidance.

Resource from this episode

Research papers:

• A benchmark study of vision and pathology foundation models for computational pathology
• Domain Generalization of Histopathology Foundation Models in Multicenter, Multi-Scanner Cohorts: A Comparative Benchmark
• Slide Selection Introduces Sampling-Induced Prediction Uncertainty in Digital Pathology AI: Evidence from Multi-Slide Breast Cancer Cohorts

Data resource mentioned:

• The Cancer Genome Atlas (TCGA) through the Genomic Data Commons (GDC) portal

Support the show

Get the "Digital Pathology 101" FREE E-book and join us!