Enhancing Inspection and Monitoring Strategies with Graphics-Based Digital Twins
Advisor: Professor Billie F. Spencer Jr.
Abstract
Structural health assessment relies on information obtained from inspection and/or monitoring and is crucial for enabling societal resilience. In the aftermath of natural disasters such as earthquakes or hurricanes, rapid structural inspection and health assessment are essential for ensuring the safety of individuals returning home and facilitating the prompt restoration of normalcy in life, work, and productivity. Similarly, continuous structural health monitoring (SHM) facilitates timely detection of deterioration and anomalous behavior, which not only can reduce unexpected failures, but also substantially reduces maintenance costs. Recently, numerous researchers have proposed computer vision (CV)-based methods to aid inspection and SHM of civil infrastructure. However, conducting field validation of these approaches is challenging, hindering significantly their advancement and practical implementation. In the case of post-earthquake inspection, for example, damaging earthquakes occur infrequently; even when they do occur, access to the affected regions is often limited due to the locations being remote or due to safety concerns. Post-earthquake imagery of damaged infrastructure is usually made available after-the-fact and used for automated damage detection and localization; nonetheless, understanding the meaning of this imagery in terms of the condition of the individual structures remains largely unexplored. On the other hand, while continuous SHM of infrastructure faces fewer difficulties to access, field validation is still challenging due to the lack of ground truth regarding the condition of the structure. Moreover, evaluation of the influence of field conditions (e.g., lighting, wind-induced camera ego-motion, etc.) on vision-based measurements is limited. Thus, CV-based methods for inspection and SHM have largely remained in an academic setting.
To address the above challenges, a Graphics-based Digital Twin (GBDT)-aided structural health assessment framework is proposed. The GBDT is a virtual replica of an as-built structure, incorporating both a high-fidelity numerical model and a photorealistic portrayal of the structure. This comprehensive representation of both the physical and visual aspects of an as-built structure in the GBDT renders it an effective tool for assessing and advancing proposed CV-based inspection and monitoring strategies. Development of the GBDT-aided structural health monitoring and inspection framework requires completion of the following tasks: 1) Creation of a GBDT integrating a high-fidelity finite element (FE) model and a computer graphics (CG) model. The FE model enables the simulation of structural responses and the prediction of damage under specific loads, while the CG model offers a photorealistic representation of the structure and predicted damage, facilitating simulation and assessment of vision-based inspection or monitoring procedures in the field. 2) Development of GBDT-aided post-event structural inspection and safety assessment, involving UAV inspection planning, as well as localization and visualization of detected damage on the GBDT. 3) GBDT-aided development and validation of vision-based
structural response monitoring strategies addressing practical concerns such as lighting variations and camera ego-motion during measurements, as well as interpreting measurement outcomes to infer structural condition.
The proposed GBDT framework is demonstrated for two important use cases: rapid post-earthquake inspection and inland navigational infrastructure monitoring. In the post-earthquake inspection scenario, a five-story benchmark building is employed to showcase the construction of the GBDT and the modeling of seismic damage. Subsequently, the GBDT of a high-rise building on the campus of Guangzhou University in China is utilized to demonstrate UAV inspection planning and safety condition assessment from collected images. For the monitoring case, miter gates crucial for lock and dam operations on inland waterways are selected for consideration. Their structural responses induced by hydrostatic loads are monitored to deduce changes in structural condition. Vision-based response measurement methods are developed and evaluated within the GBDT framework, accounting for practical field measurement challenges such as lighting changes and camera ego-motion. The obtained responses are compared to the predicted response from the GBDT to deduce structural condition. The framework is then applied to a miter gate at The Dalles Lock and Dam on the Columbia River in Oregon. These two applications demonstrate the efficacy of the GBDT-aided structural health assessment framework for computer-vision-based inspection and monitoring.