Abstract: Achieving intelligent robotic manipulation requires bridging the gap between multimodal perception and interpretable, safe control. This talk presents a unified research journey toward neural-symbolic control, where robots reason and act through the integration of multimodal sensing, differentiable optimization, and data-driven learning. I will begin with ManiFeel, a comprehensive benchmark for visuotactile manipulation, which establishes a scalable framework to study how tactile sensing enhances policy learning under visually degraded or contact-rich conditions. ManiFeel systematically evaluates sensing modalities, tactile representations, and policy architectures, revealing when and how multimodal feedback improves manipulation robustness. Building on these insights, I will introduce LeTac-MPC, a learning-based model predictive control (MPC) framework that couples high-resolution tactile sensing with differentiable MPC. LeTac-MPC enables tactile-reactive grasping across diverse object properties, achieving robust, adaptive control in dynamic and force-interactive environments. Finally, I will present LeTO, a learning constrained visuomotor policy that embeds differentiable trajectory optimization into neural networks, combining the interpretability and safety of model-based control with the adaptability of deep learning. LeTO enables robots to generate smoother, constraint-compliant, and dynamically feasible trajectories, ensuring safer and more reliable interactions in real-world manipulation tasks. Together, these works chart a pathway toward intelligent, physics-grounded, and neural-symbolic manipulation driven by multimodal sensing and optimization-based learning.
Bio: Dr. Yu She is an assistant professor at Purdue University Edwardson School of Industrial Engineering. Prior to that, he was a postdoctoral researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT from 2018 to 2021. He earned his PhD degree in the Department of Mechanical Engineering at the Ohio State University in 2018. His research is at the intersection of mechanism design, tactile sensing, intelligent control, and robot learning. He is a recipient of the ASME Feudenstein Young Investigator Award (2025), Showalter Early Investigator Award (2024), and the Google Research Scholar Award (2022) and multiple best paper recognitions.
Location: Please meet in CSL Studio 1232 — please enter through the center doors in the middle of the parking garage's south face, if you do not have card swipe access to CSL Studio.