Zoom: https://illinois.zoom.us/j/86238233298?pwd=Y29EWXRPOWtiZ09DczRYMXJZK3JRUT09
Title: Reconstructing and Tracking 3D Humans from Video
Abstract: The majority of the visual content we capture and have access to focuses on humans. In order to build AI systems that can understand and learn from this content, it is crucial to enable perception of humans from these visual observations. Despite the advances of 2D perception, we ultimately want our systems to also allow for the 3D perception of humans. In this talk, I will present our recent works on automatic perception of 3D humans from images and videos. First, I will explore the use of large scale movie sequences to train systems for 3D human reconstruction. Despite the rich nature of movie data (i.e., large variety of behaviors and interactions, long range video), the fragmentation of movies in "shots" can complicate their analysis. Instead of treating this fragmentation as noise, we leverage the multi-shot continuity as supervisory signal for 3D reconstruction. Then, I will discuss how we can use the resulting 3D human reconstruction models to lift humans to 3D and then track them over video sequences. In this case, the rich 3D perception of humans is crucial to improve upon traditional tracking on the 2D image plane. Finally, I will expand the scope and present our recent work that considers the perception of humans together with their 3D environment and highlight the benefits that this provides for various tasks.
Bio: Georgios Pavlakos is a Postdoctoral Researcher at UC Berkeley, advised by Angjoo Kanazawa and Jitendra Malik. His research interests include computer vision, machine learning, and robotics. He completed his PhD in Computer Science at the University of Pennsylvania with his advisor, Kostas Daniilidis. He has spent time at Max Planck Institute with Michael Black and at Facebook Reality Labs. His PhD dissertation received the Morris and Dorothy Rubinoff Award for the Best Computer Science Dissertation at UPenn.
