Article Title: Review on scGPT and Geneformer in single-cell biology
Introduction to AI in Single-Cell Biology
The use of foundational models, such as scGPT, GeneCompass, and Geneformer, is a hot topic in single-cell biology. However, it’s important to assess their effectiveness, especially in zero-shot settings, where there are exploratory experiments and a lack of clear labels for fine-tuning. In this paper, researchers focus on rigorously evaluating the zero-shot performance of these models.
Assessment of scGPT and Geneformer
Previous studies have shown that fine-tuning these models on specific tasks has limits, especially in the field of single-cell biology due to the nature of the field and high computational requirements. Microsoft researchers took on this challenge and evaluated the zero-shot performance of Geneformer and scGPT models across a range of tasks. They used human tissue datasets to assess cell type clustering, batch effect correction, and the effectiveness of the models’ input reconstruction based on the pretraining objectives.
Evaluation Metrics
For each task, researchers evaluated cell embeddings, batch integration, and the performance of these models in their pretraining objective using specific metrics.
Results and Conclusion
The evaluation results showed that both scGPT and Geneformer performed worse than expected, falling behind in various tasks and metrics. In conclusion, these models displayed sub-par performance when applied to single-cell biology. scGPT outperformed Geneformer in these evaluations. The researchers also provided insights for future work and areas for improvement.
For more information, access the full paper. And don’t forget to join our ML SubReddit, Facebook Community, Discord Channel, and Email Newsletter for the latest AI research news and projects. If you like our work, you will love our newsletter!