I develop reliable language models and NLP methods for medicine, spanning biomedical language model pre-training, adaptation to clinical settings, and model evaluation. The following three themes are topics I have worked on, am working on, and am interested in.
Reliable clinical prediction
I develop (large) language model methods for clinical prediction, including post-discharge mortality and psychiatric readmission. My current focus is on advancing models for better and more reliable clinical prediction, with a research agenda that includes confidence estimation, explanation generation, and effective and efficient processing of long clinical documents.
Language models for biomedicine
I co-led BioBERT, one of the first language models pre-trained on biomedical literature, which has been cited more than 11,000 times. My research interests in this area include model and system design for biomedical NLP. In collaboration with AstraZeneca, I developed TinyBERN2, a distilled transformer model for high-throughput biomedical NER.
Biomedical and clinical information extraction
I have been working on named-entity recognition and relation extraction in biomedical and clinical text. I am currently interested in phenotype extraction and building graphs from clinical text. I participated in BioCreative VII, where our team placed first in biomedical NER and third in relation extraction. I later co-organized the 2024 and 2025 ChemoTimelines shared tasks.