An Efficient, Effective Assessment Strategy for the GenAI Era with a Computer-Based Testing Facility

The rapid rise in genAI capabilities has been a shock to the educational system. GenAI can trivially do most of the well-specified assessment activities that help our students build basic skills in a domain. Educational theory (e.g., constructivism) suggests that building these basic skills remains important to develop higher-level skills. Many instructors are struggling, however, to ensure that their students are learning these basic skills, finding themselves pushed back into in-person paper-based exams and shifting a larger fraction of the student’s grade into a few large assessment, increasing student anxiety.  In this talk, I’ll outline a more modern, holistic strategy for assessment, that focuses on a "two-lane" approach, where the student’s fluency of building block skills is assessed without access to AI (lane 1) and students proficiency with using genAI to solve incompletely specified problems is assessed with open-ended projects (lane 2). In the proposed implementation, lane 1 is highly automated through the use of a Computer-Based Testing Facility (CBTF), allowing frequent, small exams with re-take opportunities to ensure mastery of these concepts with low student anxiety and almost no effort from the course staff. Automating lane 1 frees up course staff to focus on assisting student learning and lane 2 assessment.  I’ll discuss the campus’s CBTF and show data representative of the benefit of frequent and second-chance testing on student learning and test anxiety.

Craig Zilles is a Professor and Severns Faculty Scholar in the Siebel School of Computer and Data Science at the University of Illinois at Urbana-Champaign. His current research focuses on applying computing and data analytics to education, including the development of the Computer-Based Testing Facility (CBTF) at Illinois. Previously, his research focused on the interaction between compilers and computer architecture, and he developed the first algorithm that allowed rendering arbitrary three-dimensional polygonal shapes for haptic interfaces (force-feedback human-computer interfaces). He received the IEEE Education Society's 2010 Mac Van Valkenburg Early Career Teaching Award and an NSF CAREER award. Craig has received a wide range of teaching awards, holds 5 patents, and has been recognized by best paper awards from ASPLOS in 2010 and 2013.