Computer Vision and Machine Learning Method for Detection and Assessment of Wheel Anomalies Using Sensor Fusion of Thermal and Visible Spectrum Cameras

MTU Project Information

NuRail Project IDNURail2013 -MTU-R06
Project TitleComputer Vision and Machine Learning Method for Detection and Assessment of Wheel Anomalies Using Sensor Fusion of Thermal and Visible Spectrum Cameras
UniversityMichigan Technological University
Project ManagerPasi Lautala
Principal InvestigatorTimothy C. Havens
PI Contact Information
Funding Source(s) and Amounts Provided (by each agency or organization)$29,011 NURail Funds; $27,371 Michigan Tech; $7,500 UP (existing images)
Total Project Cost$63,882
Agency ID or Contract NumberDTRT12-G-UTC18 (Grant 1)
Start Date2014-01-01
End Date2016-03-31
Brief Description of Research ProjectAutomating inspections and identification of defects are both high priority research items for the rail industry. The main objective of this research project is to develop computer vision and machine learning methods for automatically detecting the defects of rail car wheels using thermal and visible spectrum (color) camera sensors. The project will concentrate on identifying flat wheels which are a serious concern for both wheel and suspension hardware, and also rail and track structure. Recently, Union Pacific (UP) deployed a thermal imaging sensor that collects images of wheels on a notoriously problematic downhill grade which is known to cause wheel problems due to skidding. The first phase of the project will focus on algorithm development for autonomously detecting and scoring anomalous wheel conditions using thermal images provided by UP. UP has provided Dr. Havens with more than 100,000 images from their thermal sensor, incorporating various types of rail cars. For the second phase of this research, we will deploy thermal and visible spectrum cameras on the LS&I Railroad, headquartered in Marquette, Michigan. We will take measurements at several locations, for different car types, and in different environmental conditions. These data will be used to validate our algorithms and to provide further opportunity to tune our software for use in different scenarios. Lastly, we will collaborate with UP to develop a sensor fused version of our anomalous wheel detection software, which will combine the images from thermal and visible spectrum cameras. Based on Dr. Haven's previous successes in using sensor fusion for multi-modal detection of landmines, we believe that the combination of thermal and visible-spectrum imagery will result in a highly capable automatic wheel anomaly scoring system.
Describe Implementation of Research Outcomes (or why not implemented)
Impacts/Benefits of Implementation (actual, not anticipated)
Web Links
Project Website
Final ReportNURail2013-MTU-R06_-_Computer_Vision_and_Machine_Learning_Method_Final_Report.pdf