Machine Learning Seminar: Dr. Haonan Chen, "Multimodal Compositional Robot Learning."

Nov 20, 2026   2:00 pm  
Sponsor
Siebel School of Computing and Data Science
Speaker
Dr. Haonan Chen
Contact
Weixin Chen
E-Mail
weixinc2@illinois.edu
Originating Calendar
Siebel School Speakers Calendar

Abstract: Robots must combine diverse sources of knowledge to interact effectively with the physical world: vision guides motion, touch reveals contact, and physical constraints determine which actions are feasible. How can robots learn specialized capabilities and compose them to solve new tasks? In this talk, I will present a compositional approach to robot learning that brings independently learned policies, reusable motion representations, and physical feedback together during action generation. I will first introduce Policy Consensus, which combines action distributions from separately trained sensory policies, enabling multimodal manipulation without joint retraining. I will then present CoStream, which composes motion from video predictions, task constraints, and contact feedback for precise manipulation, including computer-component assembly. Finally, I will discuss how making force and touch observable can support this approach and motivate future architectures that compose learned skills with models of their physical effects.

Bio: Haonan Chen is a postdoctoral researcher at Harvard University, working with Prof. Yilun Du. He received his Ph.D. from the University of Illinois Urbana-Champaign, advised by Prof. Katherine Driggs-Campbell. His research focuses on multimodal robot learning and dexterous manipulation, with an emphasis on compositional learning, reusable action representations, and force and tactile sensing. His work has been recognized as a Best Paper Finalist at CoRL 2023 and an Outstanding Paper Finalist at RSS 2026.

link for robots only