Ballast Condition and Degradation Evaluation using Computer Vision Techniques: Algorithms and Applications
Advisor: Professor Erol Tutumluer
Abstract
Railroads serve as one of the most efficient means of transportation within the U.S. The track
substructure plays a significant and irreplaceable role in the overall performance of a railway
track system in response to repeated train loading. A majority of the railroad structures are
ballasted tracks, which may contain up to six basic components including rail, crossties, ballast,
subballast, subgrade, and embankment. Among these, railroad ballast stands as a key component
of the railroad track substructure. It is comprised of large, uniformly graded, and fully crushed
coarse aggregate materials placed between and immediately underneath the crossties, providing
drainage and structural support for the track system. Ballast degradation denotes the process
where the material deteriorates as the voids in unbound aggregate layers are filled with relatively
finer materials, or fouling agents, commonly resulting from the breakdown of ballast,
contamination, and subgrade soil intrusion. As the ballast ages, it is progressively subjected to
fouling and degradation caused by particle breakage and abrasion, thus leading to poor drainage,
rapid and excessive settlement, and reduced lateral stability. To measure ballast fouling and
degradation levels, and correlate it with ballast and track performance, several ballast fouling
indices have been proposed. These indices fall into different categories: weight-based, volumebased
and imaging-based. Depending on the specific analysis scenarios, different indices should
be employed to provide accurate estimation of the ballast condition.
Current ballast evaluation methods mainly rely on visual inspections, laboratory analysis of
sampled ballast, and Ground Penetrating Radar (GPR) techniques. While visual inspections are
subjective and sample collection has its limitations, GPR has been effective in assessing ballast
conditions to some extent but falls short in detailed geotechnical analysis. Over the years, field
inspection systems have evolved, with vision-based methods being developed for analyzing track
conditions, but these primarily focus on track superstructure components, lacking in substructure
evaluations like ballast fouling evaluation. Non-vision methods like GPR and Light Detection
and Ranging (LiDAR) have been used for substructure inspections; however, they lack detailed
geotechnical information about the ballast, accompanied by various resolution concerns and
limitations.
This doctoral research study focused on developing a novel ballast evaluation system by
leveraging deep learning-based computer vision techniques to evaluate ballast condition and
degradation level in the field. The proposed system entails the development of both software and
hardware components to facilitate automated field ballast data collection and in-depth ballast
condition evaluation. An imaging-based index, Percent Degraded Segments (PDS), is introduced
to correlate image segmentation results with ballast degradation indicators, primarily Fouling
Index (FI). The deep learning algorithms are developed and implemented, specifically Mask
Region-based Convolutional Neural Network (Mask R-CNN), for precise ballast instance
segmentation and establishing regression models to link PDS and FI. An automated Ballast
Scanning Vehicle (BSV) is designed and constructed to acquire high-quality field data during
operation, which is then analyzed using a companion data analysis software, I-BALLAST, for
streamlining the evaluation procedure and minimizing user-dependency. Model improvements
are pursued through (i) exploring advanced deep learning architectures like Vision Transformer,
and (ii) a semi-supervised learning framework, Ballast Semi-Supervised Framework (BSSF), to
enhance ballast segmentation performance. Several field experiments have been conducted
during the development and validation of the ballast evaluation system. Two field experiments,
in conjunction with Ballast Shoulder Cleaner (SBC), in Indiana and Ohio Norfolk Southern (NS)
railroads, have been conducted to test the functionalities of the developed BSV and collect field
data. The fully developed ballast evaluation system, after integrating the BSV and I-BALLAST,
is successfully validated through the last field experiment at the High Tonnage Loop (HTL) in
the Transportation Technology Center (TTC), demonstrating its potential to provide accurate and
robust ballast condition evaluations and geometric analyses, crucial for optimizing rail
maintenance and rehabilitation planning on-site. In the longer term, the fully developed ballast
evaluation system could serve as the key component of a comprehensive Ballast Management
System (BMS) to study the deterioration mechanisms and improvement of ballasted track
designs and ballast maintenance practices.
