Natural Language Processing in Assessment
Abstract:
As we design new assessments, they are likely to increasingly use NLP and AI for test item creation, feedback and scoring. Everyday digital interactions are also expanding. New types of digital interactions influence how we engage in speaking, writing, reading and listening, as well as interpersonal situations, such as collaboration and transactions. Assessment researchers and designers need to consider how to assess traditional and 21st century skills situated in modern, digitally-mediated contexts. In language assessment, for instance, this might be the assessment of reading on tablet devices or collaborative writing using Google Docs. Since the debut of automated essay scoring systems over 20 years ago for large-scale, high-stakes writing assessment, ideas for innovation in automated writing evaluation have grown. Advanced natural language processing (NLP) methods expanded the ability to evaluate writing quality and content, provide qualitative feedback for instruction and formative assessment, and conduct learning analytics research. Developments in speech recognition led to assessment of spoken language on high-stakes language assessments. As AI improves, AI-enabled assessments continue to leverage these advances. In this talk, I will discuss the evolution of NLP in assessment, engaging with education domain problems to build impactful NLP solutions, and modern AI-enabled assessment. To illustrate, I will present an assessment ecosystem developed for the Duolingo English Test -- a ground-breaking, digital-first, high-stakes English language assessment.
