Title: Large Language Models for Teacher Feedback

Presenter: Dr. Dora Demszky, Assistant Professor, Education Data Science, Graduate School of Education and Computer Science, Stanford University

Abstract: Large Language Models show unprecedented potential in scaling many aspects of education — can they also facilitate high-quality instruction by serving as a professional learning tool for teachers? This talk synthesizes findings from three papers that evaluate the ability of LLMs to provide feedback to educators on their discourse. Firstly, we evaluate the ability of GhatGPT to perform instructional coaching tasks, such as using an observational instrument to score instruction and provide teachers with suggestions for improvement. We show that while the model generates feedback that is relevant, it struggled with insightfulness. Secondly, we explore the use of LLMs in assisting math tutors with addressing students math mistakes. Despite consistent improvements over the original tutor responses, the models' remediation efforts often fell short when compared to experienced math teachers, indicating a need for keeping an expert in the loop. Lastly, we examine LLMs ability to use growth mindset supportive language (GMSL) in math teaching moments (responding to student mistakes, introducing/debriefing tasks). Encouragingly, after prompt-engineering, LLMs showcased the ability to reframe unsupportive utterances effectively, even surpassing GMSL-trained teachers in certain evaluations. These collective insights demonstrate the promise of LLMs in complementing existing teacher professional learning processes. However, the current limitations also emphasize the continued need for human expertise in improving and overseeing the model's implementation.

Bio: Dora Demszky is an Assistant Professor in Education Data Science at Stanford University, and in Computer Science (by courtesy). Her research focuses developing and deploying tools that combine natural language processing, linguistics and input from practitioners to facilitate equitable, student-centered instruction. Her tools analyze educational discourse such as student-teacher interactions, student group work and textbooks, to identify features of of high-quality instruction and suggest areas of improvement for educators. Dr Demszky has received her PhD in Linguistics at Stanford.

https://www.dorademszky.com/