Abstract:
Compilers are the workhorse that bridge the gap between human readable and machine executable code. The diversity of modern programs, along with the advent of new and complex hardware architectures, has strained the capabilities of current compilers, making development and maintenance of automatic program optimizations in compilers exceedingly challenging. In spite of this, modern compiler optimizations are still hand-crafted using technology that existed decades ago and usually make optimization decisions considering an abstract machine model. It is high time that we modernize our compiler toolchains using more automated decision procedures to make better optimization decisions while reducing the expertise required to build and maintain compiler optimizations.
In this talk, I will show how we can leverage the changes in the computing environment to modernize compiler optimizations, using auto-vectorization (automatic conversion of scalar code into vector code) as an example.
First, I will demonstrate how we can take advantage of modern solvers and computing platforms to perform vectorization. Modern compilers perform vectorization using hand-crafted algorithms, which typically only find local solutions under linear performance models. I present goSLP, which uses integer linear programming to find a globally optimal instruction packing strategy to achieve superior vectorization performance.
Next, I will discuss how to modernize the construction of compiler optimizations by automatically learning the optimization algorithm. I present Vemal, the first end-to-end learned vectorizer which eliminates the need for hand-writing an algorithm. The key is to formulate the optimization problem as a sequential decision making process in which all steps guarantee correctness of the resultant generated code. Not only does Vemal reduce the need for expert design and heuristics, but also it outperforms hand-crafted algorithms, reducing developer effort while increasing performance.
Finally, I will show how we can use data to learn better non-linear performance models, rather than the complex and incorrect hand-crafted models designed by experts, to enhance the decision procedure used in Vemal. I present Ithemal, the first learned cost model for predicting throughput of x86 code. Ithemal more than halves the error-rate of complex analytical models such as Intel’s IACA.
Both Vemal and Ithemal achieve state-of-the-art results and pave the way towards developing more automated and modern compiler optimizations with minimal human burden.
Bio:
Charith Mendis is a final year PhD student in Computer Science and Artificial Intelligence Laboratory at Massachusetts Institute of Technology. His research interests include Compilers, Machine Learning and Program Analysis. He completed his Master’s degree at MIT for which he received the William A. Martin Thesis Prize and his bachelor’s degree at University of Moratuwa, Sri Lanka for which he received the institute Gold Medal. Charith was the recipient of the best student paper award at IEEE Big Data conference and the best paper award at ML for Systems workshop at ISCA. He has published work at both top programming language venues such as PLDI and OOPSLA as well as at top machine learning venues such as ICML and NeurIPS. Charith’s recent work on performance prediction is used at Google as part of their CPU modeling effort.
Faculty Host: Lawrence Rauchwerger
