Title: Understanding “Understanding” In Large Language Models
 
Abstract:
Do Large Language Models (LLMs) “understand” the language that they process? In this talk, I’ll describe three studies that adapt experimental approaches from human psycho- and neuro-linguistics to test whether LLMs exhibit signatures of human-like comprehension. First, I will ask whether semantic information can “penetrate” and influence syntactic processing in LLMs—like it does in humans—or whether some syntactic processing stages in LLMs are “encapsulated” from meaning. Second, I will ask whether LLMs represent a fundamental aspect of linguistic meaning: distinguishing between agents and patients in sentences. Third, I will ask whether Large Vision-Language Models use visual context to interpret language in a manner that exhibits pragmatic-like sensitivity to whether expressions that refer to objects are felicitous, under-informative, or over-informative. These studies reveal both similarities and differences between LLMs and humans, breaking comprehension into theoretically-informed constructs and promoting a nuanced view of how, and in what sense, LLMs understand language.
 
Bio:
Idan A. Blank is an Assistant Professor of Psychology and Linguistics at the University of California, Los Angeles. He leads the BlankLangLab, which studies language comprehension in biological and artificial minds, examining how meaning is represented and processed in the human brain and in AI systems. Using functional neuroimaging, behavioral experiments, and computational approaches, his work investigates how various information sources are integrated during language processing, and how the “mental labor” of comprehension is divided between different cognitive systems. He received his PhD in Cognitive Science from MIT, followed by a post-doc at the McGovern Institute for Brain Research.