Safe learning-enabled control with limited data
How to transfer the successes of reinforcement learning on games (e.g., AlphaZero) and language models (e.g., ChatGPT) to real-world physical systems (e.g., autonomous driving, smart grids, robotics, etc.) is one of the most important open questions for the next few decades. Many challenges need to be tackled to solve this open question. For example, a prominent challenge is ensuring safety during learning-enabled control. Besides, real-world physical systems may have less data than traditional training tasks (e.g., ChatGPT uses 300 billion words) due to expensive/time-consuming data collection and potentially time-varying systems.
Today, we will focus on the two challenges above on safety and limited data. We will discuss two directions: a model-based one and a model-free one, because we believe both directions are vital for addressing the open question above. The Part 1 of the talk is on the model-based direction: system identification (ID.), then robust control for safety (then system ID. again if considering adaptive control). Our major novel contribution is to system ID. Motivated by the good empirical performance of set membership (SM) on estimating the uncertainty set of the system parameters, we provide the first non-asymptotic diameter bound for SM’s uncertainty set on linear systems, which solves a long-lasting open question. We show that SM’s uncertainty sets achieve a faster (transient) convergence rate than the state-of-the-art LSE's confidence bounds when the process noises are bounded and i.i.d. Further, this diameter bound opens the door to non-asymptotic performance guarantees for robust (adaptive) control using SM for system ID, which potentially needs less training data. In Part 2, we consider a model-free online switching control framework since it shows promises of bridging the sim-2-real gap, which is an important issue since simulators are widely adopted to complement the limited real-world data. We design an online switching control algorithm that guarantees stability during learning despite not knowing a specific stabilizing controller but only knowing a finite set where a stabilizing controller belongs to. Further, our algorithm is shown to achieve near-optimal regrets compared to the optimal stabilizing controller in this finite set. Lastly, we mention that our model-free approach in Part 2 can be naturally combined with the model-based approach in Part 1.
For more information.
