Developing a driving assistance system based on object detection through thermal camera analysis / Mohammad Sojon Beg

Faculty of Mechanical and Automotive Engineering Technology

Thesis (Master of Science) -- Universiti Malaysia Pahang – 2024

Includes bibliographical references

Road safety is a major concern in Malaysia, particularly at night. Accidents during this time are more hazardous due to reduced visibility of obstacles, impaired vision, and difficulty in calculating distances. The number of collisions has significantly increased over the years, especially at night when roadside lamps are not always available. This project aims to address this issue by implementing driving assistance systems with IR sensor-based object detection technologies to prevent collisions. The driving assistance system has been designed and tested in real-time scenarios using primary datasets. The real-time on-road scenarios were captured in the UMPSA Pekan to Hospital Pekan for daytime and UMPSA Pekan to UMPSA Gambang for nighttime videos. The dataset includes various road conditions, such as straight roads, junctions, and T-junctions. Over two thousand images were annotated on the machine learning-based Roboflow dataset manager for both normal camera sensors and IR spot-finding cameras. The main objective of this study isto integrate an IR spot-finding camera sensor with the YOLO_v8 deep learning algorithm to create a thermal object-detecting driving assistance system. The second objective is to develop the driving assistance system using deep learning analysis for both normal visual camera sensors and IR spot-finding camera sensors. The third objective is to evaluate the efficiency of the YOLO_v8 deep learning algorithm for both the IR spot-finding camera and the normal visual camera sensor. The system performed better with the IR spot-finding camera sensors compared to the normal camera sensors, achieving 91.9% precision, 92.8% recall, and 94.7% mean average precision for the IR sensor, while the normal camera achieved 69.2% precision, 51.1% recall, and 58.1% mean average precision. The study also analyzed the temperature differences of objects and grayscale levels in the thermal images. The outcome of this study will serve as a benchmark for the automotive industry to integrate IR spot-finding camera sensors into vehicles. It emphasizes the importance of drivers being aware of on-road obstacles that may not be visible to the naked eye. This project represents a significant step towards implementing a driving assistance system that can enhance the efficiency of the traffic system and ensure a safe driving environment on Malaysia's roads. The information collected will be used to provide drivers with driving assistance, improving vehicle detection performance in low-light conditions at night.