News & Events

Detecting Uncertainty to Enhance AI Reliability

Litian Liu, Research Scientist at Qualcomm AI Research

CSB 601

Abstract: Detecting hallucinations in large language models is a critical open problem with significant implications for safety and reliability. While existing hallucination detection methods achieve strong performance in question-answering tasks, they remain less effective on tasks requiring reasoning. In this talk, we revisit hallucination detection through the lens of out-of-distribution (OOD) detection, a well-studied problem in areas like computer vision. Treating next-token prediction in language models as a classification task allows us to apply OOD techniques, provided appropriate modifications are made to account for the structural differences in large language models. We show that OOD-based approaches yield training-free, single-sample-based detectors, achieving strong accuracy in hallucination detection for reasoning tasks. Overall, our work suggests that reframing hallucination detection as OOD detection provides a promising and scalable pathway toward language model safety. 

Bio: Litian Liu is a Research Scientist at Qualcomm AI Research, where she works on uncertainty, robustness, and efficiency in machine learning. Her research centers on efficient and reliable uncertainty estimation for out-of-distribution detection in classifiers and hallucination detection in large language models, with applications to safety-critical systems such as autonomous driving. She received her Ph.D. in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology.