NLM DIR Seminar Schedule

UPCOMING SEMINARS

RECENT SEMINARS

Scheduled Seminars on April 22, 2025

Speaker
Stanley Liang, PhD
PI/Lab
Dr. Sameer Antani
Time
11 a.m.
Presentation Title
Large Vision Model for medical knowledge adaptation
Location
Hybrid
In-person: Building 38A/B2N14 NCBI Library or Meeting Link

Contact NLMDIRSeminarScheduling@mail.nih.gov with questions about this seminar.

Abstract:

Large Vision Models (LVMs), built on transformer architectures, have emerged as powerful tools for general-purpose visual representation learning. This study explored the application of LVMs for medical knowledge adaptation, with a focus on automated pneumonia severity prediction from chest X-rays. Leveraging models such as ViT, DINOv2, and BiT, we investigated the roles of LVM as feature encoders under both direct extraction and self-supervised learning settings. Using a dataset of 2,599 chest X-rays labeled with mRALE scores, we evaluate multiple configurations, including Simple Siamese Network based contrastive learning, to assess how LVMs adapt to the medical domain with limited labeled data. The results show that LVMs, particularly when combined with self-supervised learning, can achieve state-of-the-art (SOTA) performance across multiple regression metrics (MSE: 23.83, 95%CI 22.67-25.00; MAE: 3.64, 95%CI 3.54-3.73; explained variance: 0.66, 95%CI 0.63-0.68; R²: 0.65, 95%CI 0.63-0.67; Spearman’s ρ: 0.80, 95%CI 0.77-0.81), when trained on a modest dataset. Notably, while self-supervised learning can enhance general feature quality, it has limited impact on improving alignment with human expert judgment. This work highlights the effectiveness and limitations of LVM-based medical AI pipelines and sets the stage for future efforts involving multi-modality integration and cross-domain adaptation. Ongoing and future work aims to extend this framework through LLM integration and multi-agent learning systems for automated, scalable medical image analysis.