Join us for our next Quantitative Psychology Brownbag with Dr. Kylie Anglin (Department of Educational Psychology, University of Connecticut)!
This event is online only. Please join us using this meeting link.
Title: Large Language Models and the Measurement of Psychological Constructs: Zero-Shot, Few-Shot, and Fine-Tuning Approaches
Abstract: Given psychology is largely the study of internal processes—such as thoughts, feelings, and mental states– many psychological constructs are latent and cannot be observed directly. However, such constructs can often be inferred from client writing or speech. Thus, in studying psychological constructs, researchers often require high-quality, labeled, text data. This talk demonstrates a rigorous process for the use of pre-trained large language models (LLMs) to obtain such data, with zero-shot classification, few-shot classification, and fine-tuning. The approach is demonstrated through the application of LLMs to the identification of negative core beliefs and meaning making in expressive writing samples. Validity considerations will be discussed throughout the talk and Python code will be provided.
About Dr. Kylie Anglin: Kylie Anglin is an Assistant Professor in Research Methods, Measurement, and Evaluation at the University of Connecticut. Her research aims to increase the quality and depth of evaluations of interventions and policies, most often through the rigorous use of artificial intelligence (AI). Her research provides tools for leveraging insights from data that are either difficult to locate or difficult to analyze using traditional means. In doing so, she addresses critical resource constraints in education research while simultaneously addressing risks in the use and interpretation of AI output. Her research has been funded by the Institute of Education Sciences (IES) and has been published in journals just as the Journal of Educational and Behavioral Statistics, Educational Evaluation and Policy Analysis, AERA Open, the Journal of Research on Educational Effectiveness, and Prevention Science.
Joint Quantitative Psychology Brownbags are organized by psychology programs at universities around the nation, including: The Ohio State University, University of Notre Dame, University of Maryland, College Park, University of North Carolina at Chapel Hill, University of Virginia, Vanderbilt University and University of South Carolina. Participating programs also include those from York University, McGill University, University of Missouri and University of British Columbia.
