NLM DIR Seminar Schedule

Abstract:

Large language models (LLMs) pretrained on massive data have shown their power as foundation models for pervasive tasks in natural language understanding and beyond. This inspired us to develop large cellular models (LCMs) to decipher the transcriptomic language of cells. We have developed LCMs for single-cell transcriptomics toward this goal using two approaches, which produced the two large models scFoundation and scMulan. With pretraining on tens of millions of human scRNA-seq data covering almost all known cell types and states, the models have shown ability of capturing complex context relations among gene expressions and meta attributes of cells. Experiments showed that the pretrained model can achieve state-of-the-art performances in zero-shot manner or with light fine-tuning on a diverse array of single-cell analysis tasks such as data enhancement, drug-response prediction at tissue and single-cell levels, single-cell perturbation prediction, cell type annotation, gene module inference and conditional cell generation.