AI-Driven Project Controls: Integrated Computer Vision Production Tracking & AI-driven Forecasting in Building Construction
Advisor: Professor Mani Golparvar-Fard
Abstract
This dissertation introduces a systemic approach and computational frameworks for integrating Computer Vision and AI-driven forecasting to detect, monitor, and predict the progress of under-construction systems in Building Construction and their associated labor productivity. The development of a series of deep-learning and autoregressive methods helps overcome the challenges of modern management practices in collecting and processing daily production in construction sites while providing explainable labor productivity forecasts for better decision-making and labor management on daily jobsite operations.
The proposed approach presents a new classification system as an extension to the ASTM Uniformat II Classification for Building Elements to determine partial states of progress of most construction elements. This system, denominated Visual States of Work-in-Progress (ViS-WP), is used to develop a Transformer-based deep learning computational framework for detecting production values through semantic segmentation of 360-degree images. The developed framework combines region boundary image segmentation from Large Vision Models (LVM) to provide precise object shape boundary, depth and surface normal maps inference to aggregate pixel class probabilities based on their 3D geometry features, and a custom-trained encoder-decoder model for creating class-based detection using a hierarchy-based feature inference that aggregates fine-grained class features to their coarse class equivalent. Lastly, a Multiview inference algorithm creates a probabilistic consensus of the correct coarse and fine-grained pixel class based on the shared pixel features between overlapping camera-to-3D point correspondences.
Large data requirements of Transformer-based models are addressed by developing a procedural data generation algorithm to automatically collect ground-truth data of indoor construction environments based on high Level of Detail (LOD) BIMs. As a result, the first public large-scale synthetic dataset of construction site indoor environments denominated the Synthy Construction Dataset, was created. Upon the datapoint frequency increase of labor-productivity, a deep learning-based time series forecasting model is developed to detect trend features based on Autoregression, classifying their differentials to an associated root-cause, producing explainable forecasts for proactive progress monitoring.
Several studies have been carried out across commonly observable element classes in building construction to understand the validity, accuracy, and precision of the proposed framework. The obtained results are compared against other traditional detection and forecasting benchmarks, showing significant improvements in scalability across the board.