Machine Learning Seminar Series: Yian Wang, “Unlearning Isn’t Forgetting: Revealing Hidden Leakage in Class Unlearning Evaluations.”

Title: Unlearning Isn’t Forgetting: Revealing Hidden Leakage in Class Unlearning Evaluations.

Abstract: Machine unlearning has become a practical requirement for deployed models, driven in part by regulations such as GDPR and the right to be forgotten. Class unlearning, where the goal is to remove an entire semantic class rather than scattered training points, is now evaluated with a fairly standard toolkit of forget accuracy, retain accuracy, and membership inference. Despite the apparent success of recent methods under these metrics, what an unlearned model actually does with the samples it is supposed to have forgotten remains poorly understood. In this talk, Yian will argue that a model retrained from scratch does not simply stop predicting the forgotten class. It redistributes that class's probability mass toward semantically neighboring classes in a structured and highly reproducible way, and existing unlearning methods fail to reproduce this structure. That failure is itself an attack surface. Yian will present CMIA, a class membership inference attack that reads only the probabilities assigned to the nearest neighbor class and recovers whether a model was ever trained on the forgotten class, including in a black-box setting with a single shadow model. Ten state-of-the-art methods leak under it. The talk will then introduce Tilted REWeighting (TREW), a lightweight fine-tuning objective that tilts the reweighted output distribution using inter-class similarity so that it approximates what retraining would have produced, and will close with what these observations suggest about how unlearning should be evaluated more broadly.

Bio: Yian is a fourth-year CS PhD candidate at UIUC, co-advised by Prof. Hari Sundaram and Prof. Varun Chandrasekaran, working on machine unlearning, LLM safety, and multi-agent systems. Yian's earlier research has examined what it means for a model to genuinely forget rather than to merely stop reporting. In particular, recent work has shown that (i) class unlearning evaluations overlook the underlying class geometry and leak membership through neighboring class probabilities, and that a similarity tilted fine tuning objective closes most of that gap (ICML 2026), and (ii) in multi-agent LLM systems, handoff summaries preserve operational facts while stripping the boundary metadata that licenses their use, a failure mode termed summary collapse (EMNLP 2026). More recently, Yian has been working on concept erasure in text-to-image diffusion models and on topology-aware containment of information in multi-agent systems. This research is guided by the view that forgetting should be judged by a model's whole behavior, not by a single suppressed output.