Join us for a Quantitative Area Brownbag with Dr. Cheng Liu (University of Notre Dame)!
Title: AI-Assisted Cognitive Diagnosis: Q-Matrix Prediction and Validation
Abstract: Cognitive diagnosis models (CDMs) provide detailed diagnostic information about individuals’ underlying attributes, such as competencies, enabling their applications in psychological and educational assessments. A fundamental component of CDMs is the Q-matrix, which specifies the relationships between assessment items and latent attributes. However, developing an accurate Q-matrix often requires substantial effort from subject matter experts. To address these challenges, we explore the use of artificial intelligence (AI) techniques, including machine learning with text embeddings, Bidirectional Encoder Representations from Transformers (BERT), and large language models (LLMs), to predict Q-matrix entries based on the deep contextualized semantic information extracted from items’ text content. These predictions are subsequently incorporated as prior information into an exploratory CDM framework, allowing empirical response data to further validate and refine the Q-matrix. Together, these developments highlight the potential of integrating AI and psychometric methods to improve measurement practice in psychological and educational assessments.
About Dr. Liu: Cheng Liu is an Assistant Professor of Quantitative Psychology at the University of Notre Dame. Trained in both physics and statistics, he holds a Ph.D. in Physics from the University of Illinois Urbana-Champaign and a Ph.D. in Statistics from the University of Notre Dame. Prior to joining the faculty, he served as Associate Research Professor at Notre Dame’s Lucy Family Institute for Data and Society, where he led data science initiatives for the Center for Social Research.
Dr. Liu’s research focuses on statistical modeling, machine learning, educational and psychological measurement, item response theory, cognitive diagnostic modeling, and artificial intelligence. His work integrates methodological innovation with computational infrastructure development to advance research in education, psychology, and the social sciences. He has published widely in leading journals including Psychometrika, Applied Psychological Measurement, and Journal of Educational Measurement.
