Title: Learning to Perceive Videos for Embodiment
Abstract: Video understanding has achieved tremendous success in computer vision tasks, such as action recognition, visual tracking and visual representation learning. Recently, this success has gradually been converted into facilitating robots and embodied agents to actively interact with the physical environment. In this talk, I am going to introduce our recent efforts on extracting self-supervisory signals and 3D structure from videos, and using this knowledge to help robots learn. I will talk about our work on self-supervised learning for 3D structure and motion from videos, and how self-supervision in time can help Reinforcement Learning agents to continuously adapt to new environments (e.g., Sim2Real transfers). Going beyond RL, human videos also provide an efficient way for robots to learn by imitation. I will introduce our work on 3D hand-object pose and shape estimations. By embedding these techniques, we propose a new platform and pipeline on imitation learning for Dexterous Manipulation from human Videos (DexMV). Our aim is to bridge 3D vision and robot learning in an end-to-end system. We hope this will provide new opportunities for research in both fields.
Bio: Xiaolong Wang is an Assistant Professor of the ECE department at the University of California, San Diego. He is affiliated with the CSE department, Center for Visual Computing, Contextual Robotics Institute, Artificial Intelligence Group, and the TILOS NSF AI Institute. He received his Ph.D. in Robotics at Carnegie Mellon University. His postdoctoral training was at the University of California, Berkeley. His research focuses on the intersection between computer vision and robotics. He is particularly interested in learning visual representation from videos in a self-supervised manner and uses this representation to guide robots to learn. Xiaolong is the Area Chair of CVPR, AAAI, ICCV. More details are available on his homepage: https://xiaolonw.github.io/
Zoom: https://illinois.zoom.us/j/86238233298?pwd=Y29EWXRPOWtiZ09DczRYMXJZK3JRUT09
Meeting ID: 862 3823 3298
Password: 684009
