Abstract:
Machine Learning is rapidly revolutionizing the development of many modern-day systems. However, testing Machine Learning-based systems is challenging due to 1) the presence of non-determinism in internal components (e.g., stochastic algorithms) and external factors (e.g., execution environment) and 2) the lack of accuracy specifications. Most traditional software testing techniques, while widely used to improve software reliability, cannot tackle these challenges since they predominantly rely on an assumption of determinism and lack domain knowledge. The goal of my research is to develop novel testing techniques and tools to make Machine Learning-based systems more reliable.
In this talk, I will present my work on automatically detecting bugs in Machine Learning-based systems and improving the quality of developer-written tests in such systems. My research exploits the fundamental principle that we can systematically reason about non-determinism and accuracy using rigorous statistical and probabilistic reasoning. I develop novel static and dynamic analyses for testing ML-based systems that build on this principle. My research exposed more than 50 bugs and improved the quality of hundreds of tests in more than 60 popular Machine Learning libraries, some of which are used in large-scale software ecosystems at companies like Microsoft, Google, Meta, Uber, and DeepMind.
Bio:
Saikat Dutta is a PhD Candidate in the Computer Science Department at UIUC, advised by Prof. Sasa Misailovic. Saikat’s research interests are at the intersection of Software Engineering and Machine Learning. Saikat’s research focuses on improving the reliability of Machine-learning based systems by developing novel testing techniques and tools. Saikat is the recipient of the Facebook PhD Fellowship, 3M Foundation Fellowship, and the Mavis Future Faculty Fellowship.
https://wiki.illinois.edu/wiki/display/grad/PILOT+SEMINAR
