Development of an Algorithm for Analysing the Condition of the Road Surface Using Artificial Intelligence

Motorways are a strategically important part of a country’s infrastructure. High requirements for their condition stipulate regular monitoring of quality of the road surface. The large length of motorways in Russia and the impact of weather and climate conditions (seasonal temperature fluctuations, precipitation) emphasise the relevance of searching for non-destructive testing methods for road diagnostics that ensure short terms of diagnostic work and the use of minimal resources.

The considered existing solutions for detecting road surface damage include the use of ground penetrating radar, laser method, method of analysing vibration effects of road surface irregularities, detection of damage based on lidar data and mobile mapping systems.

The objective of the study was to develop an algorithm for analysing the condition of the road surface that allows detecting road surface distresses based on images obtained during the diagnostics of motorways by the KP-514-RDT airfield and road measuring mobile laboratory completed by the IndorRoad and RDT-Line software packages.

The development of an algorithm for detecting road surface defects was carried out using machine learning methods. The detected defects have precise georeferencing according to stationing of the measured road segment. As a result of development, a trained model was obtained that allows automatic marking of defects of different classes on the image. The developed algorithm is integrated into the software for managing the monitoring of the condition of regional and municipal roads.

1. Uchaeva, A. A., Gatiyatullin, M. Kh. Road conditions and traffic safety. Equipment and technology of transport, 2020, Iss. 2 (17), pp. 9. EDN: SNQXFK.

2. Jiaying Chen, Xiaoming Huang, Binshuang Zheng, Runmin Zhao, Xiuyu Liu, Qingqing Cao, Shengze Zhu. Real-time identification system of asphalt pavement texture based on the close-range photogrammetry. Construction and Building Materials, 2019, Vol. 226, pp. 910–919. DOI: 10.1016/j.conbuildmat.2019.07.321.

3. Bakanov, K. S., Lyakhov, P. V., Aisanov, A. S., Isaev, M. M., Nikulin, E. D., Koblov, P. S., Sergunova, A. S., Selednikov, N. V., Naumov, S. B., Knyazev, A. S. Road traffic accidents in the Russian Federation in 2022. Information and analytical review [Dorozhno-transportnaya avariinost v Rossiiskoi Federatsii za 2022 god. Informatsionno-analiticheskiy obzor]. [Electronic resource]: https://media.mvd.ru/files/embed/4761994. Last accessed 22.09.2023.

4. Yuchen Wang, Bin Yu, Xiaoyu Zhang, Jia Liang. Automatic extraction and evaluation of pavement three dimensional surface texture using laser scanning technology. Automation in Construction, 2022, Vol. 141, 104410. DOI: 10.1016/j.autcon.2022.104410.

5. Xu Yang, Jianqi Zhang, Wenbo Liu, Jiayu Jing, Hao Zheng, Wei Xu. Automation in road distress detection, diagnosis and treatment. Journal of Road Engineering, 2024, Vol. 4, Iss.1, pp. 1–26. DOI: 10.1016/j.jreng.2024.01.005.

6. Rada, A. O., Akulov, A. O., Nikitina, O. I. Development of a web application for monitoring highways in the digital management system of a resource-type region. Ugol, 2023, Iss. S12, pp. 117–123. DOI: 10.18796/0041-5790-2023-S12-117-123.

7. Tretyakov, S. A., Popov, I. A.Analysis of methods for determining the evenness of the road surface [Analiz sposobov opredeleniya rovnosti pokrytiya avtomobilnykh dorog]. Interexpo Geo-Siberia, 2017, Vol. 1, Iss. 1, pp. 76–80. EDN: YMXPYV.

8. Dormidontova, T. V., Domnin, O. V. Identification of defects, determination of the thickness of the pavement by the georadolocation method. In: Traditions and innovations in construction and architecture: collection of articles of the 77th All-Russian Scientific and Technical Conference. Ed. by M. V. Shuvalov, A. A. Pishchulev, V. Y. Alpatov. Samara, SSTU publ., 2020, pp. 126–131. EDN: PWSSTT.

9. Ma, L., Li, Y., Li, J., Wang, C., Wang, R., Chapman, M.A. Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review. Remote Sensing, 2018, Vol. 10, Iss. 10, 1531. DOI: 10.3390/rs10101531

10. Davidovich, M., Kuzmich, T., Vasich, D., Hiv, V., Brunn, A., Bulatovich, V. Methodology of Detection and Administration of Road Defects Based on Mobile Mapping Data. Computer modeling in engineering and science, 2021, Vol. 129, Iss. 1, pp. 207–226. DOI: 10.32604/cmes.2021.016071.

11. Milykh, V. A., Sokolova, O. S., Stepkina, E. Yu. A method for diagnosing the evenness of the pavement surface. Patent for the invention RU 2519002 C2, 10.06.2014. Patent Office: Russia. Application No. 2012138844/03 dated 10.09.2012. [Electronic resource]: https://rusneb.ru/catalog/000224_000128_0002519002_20140610_C2_RU/. Last accessed 21.02.2024.

12. Munawar, H. S., Hammad, A. W. A., Haddad, A., Soares, C. A. P., Waller, S. T. Image-Based Crack Detection Methods: A Review. Infrastructures, 2021, Vol. 6, Iss.8, p. 115. DOI: 10.3390/infrastructures6080115.

13. Song, W., Jia, G., Zhu, H., Jia, D., Gao, L. Automated Pavement Crack Damage Detection Using Deep Multiscale Convolutional Features. Journal of Advanced Transportation, 2020, Vol. 2020, Iss.1. DOI: 10.1155/2020/6412562.

14. Pan, Y., Zhang, X., Cervone, G., Yang, L. Detection of Asphalt Pavement Potholes and Cracks Based on the Unmanned Aerial Vehicle Multispectral Imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, Vol. 11, Iss. 10, pp. 3701–3712. DOI: 10.1109/JSTARS.2018.2865528.

15. Iraldi, F., Al Maki, W. F. Damage Classification on Roads Using Machine Learning. 2021 International Conference on Data Science and Its Applications (ICoDSA). Bandung, Indonesia, 2021, pp. 151–156. DOI: 10.1109/ICoDSA53588.2021.9617520.

16. Minh-Tu Cao, Quoc-Viet Tran, Ngoc-Mai Nguyen, Kuan-Tsung Chang. Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources. Advanced Engineering Informatics, 2020, Vol. 46, 101182. DOI: 10.1016/j.aei.2020.101182.

17. Silva Vaz, E., Gasparello, L. F., Gouveia, L. T, Senger, L. J. Detecting damage in roads using convolutional neural networks. Iberoamerican Journal of Applied Computing, 2023, Vol. 11, Iss. 1, pp. 1089–1098. DOI: 10.1109/BigData.2018.8621899.

18. Mokrushin, N. Yu., Speransky, D. V., Chudinov, S. A. Automated detection of defects on the roadbed using convolutional neural networks. In: Effective response to modern challenges taking into account the interaction of man and nature, man and technology: socio-economic and environmental problems of the forest complex: proceedings of the XV International Scientific and Technical Conference. Yekaterinburg, ULSTU, 2024, pp. 489–494. [Electronic resource]: https://elar.usfeu.ru/bitstream/123456789/12844/1/konf_24_087.pdf. Last accessed 21.02.2024.

19. Vasiliev, P. V., Senichev, A. V.Application of neural network technologies in the problem of control of surface defects. Izvestiya vuzov. The North Caucasus region. Series: Technical Sciences, 2020, Iss. 1 (205), pp. 33–40. DOI: http://dx.doi.org/10.17213/1560-3644-2020-1-33-40.

20. Sobol, B. V., Soloviev, A. N., Vasiliev, P. V., Podkolzina, L. A. Deep convolution neural network model in problem of crack segmentation on asphalt images. Vestnik of Don State Technical University, 2019, Vol. 19, Iss.1, pp. 63–73. DOI: 10.23947/1992-5980-2019-19-1-63-73.

21. Kanaeva, I. A., Ivanova, Yu. A., Spitsyn, V. G. Deep convolutional generative adversarial network-based synthesis of datasets for road pavement distress segmentation. Computer Optics, 2021, Vol. 45, Iss.6, pp. 907–916. DOI: https://doi.org/10.18287/2412-6179-CO-844.

22. Kravtsov, S. S., Chantieva, M. E.Automatic road damage detection with convolutional neural networks. Colloquium-journal, 2022, Iss. 3 (126), pp. 42–47. DOI: 10.24412/2520-6990-2022-3126-42-47.

23. Misyurina, I. A., Yakimov, P. Yu. Detection of pavement damage in the video stream of an automobile registrar. In: Information technologies and nanotechnologies (ITNT-2023): collection of works based on the materials of IX International Conference and Youth School (Samara, April 17–23, 2023): in 6 volumes. Ministry of science and higher education of the Russian Federation, Samara National Research University n. a. S. P. Korolev (Samara University), Image Processing Systems Institute of RAS – a subsidiary of Federal Research Centre «Crystallography and Photonics» of RAS. Vol. 4: Artificial intelligence. Ed. by A. V. Nikonorov, Samara, Publishing house of Samara University, 2023, 040382. [Electronic resource]: http://repo.ssau.ru/handle/Informacionnye-tehnologii-i-nanotehnologii/Obnaruzhenie-povrezhdenii-dorozhnogo-pokrytiya-v-videopotoke-avtomobilnogo-videoregistratora-10572. Last accessed: 27.05.2024.