National Center for Supercomputing Applications master calendar

Back to Listing

NCSA staff who would like to submit an item for the calendar can email

Vision Seminar: Soumyadip Sengupta, "Inverse Graphics for Next-Gen Video Communication"

Event Type
Svetlana Lazebnik
wifi event
Apr 26, 2022   9:00 am  
Originating Calendar
Computer Science Speakers Calendar


Meeting ID: 862 3823 3298

Password: 684009


Title: Inverse Graphics for Next-Gen Video Communication


Abstract: Video communication connects our world. The ongoing global pandemic has highlighted the necessity of video as a medium for virtual education, tele-health, business, political discourse, and creative contents. However, the visual quality of these videos significantly lack that of the professionally created videos captured with expensive equipment and edited with human expertise. In this talk, I will discuss my ongoing research on creating next-gen video communication and content creation framework by democratizing high-quality video production and editing. To improve various components of a video computationally (e.g. change lighting, replace the background etc.), we first need to infer those intrinsic components of the video related to the 3D world, and then edit them. This problem, which is often known as Inverse Graphics, is a holy-grail problem in Computer Vision. It is computationally challenging and severely under-constrained. To solve Inverse Graphics for democratizing video production, my research adopts a unique user-centric and personalized AI based approach. I will first discuss how simple user interaction strategies, such as capturing an extra image, can help us perform high-quality background replacement in real-time. Then, I will discuss how we can personalize AI models by training on the data of a specific user captured by their webcam to improve lighting.


Bio: Soumyadip Sengupta is a Postdoctoral Research Associate in Paul G. Allen School of Computer Science and Engineering at the University of Washington, Seattle. He is affiliated with GRAIL (Graphics and Imaging Laboratory), working with Steve Seitz, Brian Curless, and Ira Kemelmacher-Shlizerman. He did his Ph.D. under the supervision of David Jacobs at University of Maryland, College Park and graduated in 2019. His research interest lies at the intersection of Computer Vision and Computer Graphics, with a focus on solving Inverse Graphics for democratizing high-quality video production. His work on Background Matting received Best Student Paper Honorable Mentions at CVPR 2021 (Top 7 papers out of 1600 accepted) and has been adopted by various companies, e.g. Microsoft, Inter State Studio, etc. He will be joining the University of North Carolina at Chapel Hill as an Assistant Professor of Computer Science starting Fall 2022. More information can be found on his webpage .

link for robots only