Domain-Specific Adaptation of Large Language Models and Integration with Knowledge
Graph Analytics for Enhanced Bridge Maintenance Decision Making
Advisor: Professor Nora El-Gohary
Zoom Link:
https://illinois.zoom.us/j/82308014601?pwd=QXcUVFt4yNYkrwr9jSaiYZA8yVsUxV.1
Abstract
Bridges are critical components of transportation infrastructure, ensuring mobility and economic
connectivity. However, a large portion of U.S. bridges are in poor condition, raising significant
safety and maintenance challenges. According to the 2025 ASCE Infrastructure Report Card, 42%
of U.S. bridges are over 50 years old and 6.8% are in poor condition, with an estimated $191
billion in bridge-related system rehabilitation needs. Despite extensive data collection efforts by
transportation agencies, existing data-driven models for bridge condition assessment and
maintenance decision making remain limited. Most models rely primarily on abstract data from
single sources, such as the National Bridge Inventory (NBI), which lacks the rich contextual
information contained in textual inspection reports, images, and other heterogeneous data sources.
Consequently, these models struggle to leverage the full potential of the available multimodal data
to support accurate condition assessment, deterioration prediction, and cost-effective maintenance
strategies. To address these limitations, a novel large language model (LLM)-based analytics
framework for bridge data integration and enhanced maintenance decision support is proposed.
The proposed framework is composed of six primary components: (1) an LLM-based semantic
information extraction method for extracting information entities that describe bridge conditions
and maintenance actions from bridge textual reports; (2) a low-rank adaptation (LoRA)-based
finetuning method to efficiently adapt pre-trained LLMs to the bridge domain; (3) an LLM-based
semantic relation extraction method for extracting semantic relations from bridge reports to link
the extracted, yet isolated, information entities in the form of a bridge knowledge graph; (4) an
LLM-based data augmentation method to mitigate training data scarcity and enable advanced
encoding of high-dimensional datasets; (5) a vector encoding-based multimodal data linking
method to link diverse inspection data types (e.g., text, images) across multiple sources; and (6)
an LLM-based method that integrates knowledge-enriched prompts, retrieval-augmented
generation (RAG), reinforcement learning from human feedback, and the bridge knowledge graph
to support bridge maintenance decision-making, including automated analysis of bridge conditions
and generation of context-aware maintenance plans. The experimental results demonstrated the
promise of the proposed framework.