CANCELLED
TITLE: Complete Dictionary Learning via L4-Norm Maximization over the Orthogonal Group
ABSTRACT: This talk is about the fundamental problem of learning a complete (orthogonal) dictionary from samples of sparsely generated signals. Most existing methods solve the dictionary (and sparse representations) based on heuristic algorithms, usually without theoretical guarantees for either optimality or complexity. The recent L1-minimization based methods do provide such guarantees but the associated algorithms recover the dictionary one column at a time. We propose a new formulation that maximizes the L4-norm over the orthogonal group, to learn the entire dictionary. We prove that under a random Bernoulli Gaussian data model, with nearly minimum sample complexity, the global optima of the L4-norm are very close to signed permutations of the ground truth. Inspired by this observation, we give a conceptually simple and yet effective algorithm based on ``matching, stretching, and projection'' (MSP). The algorithm provably converges locally at a superlinear (cubic) rate and cost per iteration is merely an SVD. In addition to strong theoretical guarantees, experiments show that the new algorithm is significantly more efficient and effective than existing methods, including KSVD and L1-based methods. Preliminary experimental results on real images clearly demonstrate advantages of so learned dictionary over classic PCA bases. We will also show how insights and principles from the dictionary learning may lead to new deep networks, arguably for the first by design, which are extremely simpler, equally or more effective on large-scale real computer vision tasks than networks obtained through years of try and error. Yet the new networks are interpretable, amenable to analysis, and with better guarantee of stability and robustness.
BIOGRAPHY: Yi Ma is currently a professor in residence at the EECS Department of UC Berkeley. He has been a professor and the executive dean of the School of Information and Science and Technology, ShanghaiTech University, China from 2014 to 2017. From 2009 to early 2014, he was a Principal Researcher and the Research Manager of the Visual Computing group at Microsoft Research in Beijing. From 2000 to 2011, he was an assistant and associate professor at the Electrical & Computer Engineering Department of the University of Illinois at Urbana-Champaign. His main research interest is in computer vision, data science, and systems theory. Yi Ma received his Bachelors’ degree in Automation and Applied Mathematics from Tsinghua University (Beijing, China) in 1995, a Master of Science degree in EECS in 1997, a Master of Arts degree in Mathematics in 2000, and a PhD degree in EECS in 2000, all from the University of California at Berkeley. Yi Ma received the David Marr Best Paper Prize at the International Conference on Computer Vision 1999, the Longuet-Higgins Best Paper Prize (honorable mention) at the European Conference on Computer Vision 2004, and the Sang Uk Lee Best Student Paper Award with his students at the Asian Conference on Computer Vision in 2009. He also received the CAREER Award from the National Science Foundation in 2004 and the Young Investigator Award from the Office of Naval Research in 2005. He has written two textbooks: “An Invitation to 3-D Vision” published in 2004, and “Generalized Principal Component Analysis” published in 2016, all by the Springer. He was an associate editor of IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), the International Journal of Computer Vision (IJCV), SIAM journal on Imaging Sciences, IEEE Signal Processing Magazine, and IEEE transactions on Information Theory (TIT). He is currently founding associate editors of the IMA journal on Information and Inference and SIAM journal on Mathematics of Data Science. He has served as Area Chairs for ICCV, CVPR, and NIPS, the Program Chair for ICCV 2013, and the General Chair for ICCV 2015. He is a Fellow of both IEEE and ACM. He is ranked the World's Highly Cited Researchers of 2016 by Clarivate Analytics of Thomson Reuters and is among Top 50 of the Most Influential Authors in Computer Science of the World, ranked by Semantic Scholar, reported by Science Magazine, April 2016.
