Hyperparameter optimization of convolutional neural networks using Orthogonal Array for concrete crack detection
Downloads
##plugins.themes.academic_pro.article.main##
Abstract
Convolutional Neural Networks (CNNs) currently hold a prominent position among the most effective methods for classifying images across a wide range of fields. In construction, CNNs are extensively applied for detecting concrete cracks, supporting Structural Health Monitoring (SHM). However, selecting appropriate hyperparameters for models with a high expected accuracy poses a significant challenge. Although some recent techniques have demonstrated remarkable performance on common datasets, they have not consistently performed well on concrete crack datasets. To address this, this study uses the established Orthogonal Array Tuning (OAT) framework to efficiently optimize the hyperparameters of CNNs for concrete crack detection. By evaluating four hyperparameters across four levels, this approach significantly reduces the computational burden, requiring only 16 experimental combinations instead of an exhaustive grid search of 256 (44). This reduction facilitates rapid model deployment, a crucial factor for real-time Structural Health Monitoring (SHM). Additionally, the obtained results are compared with baseline methods, including Analysis of Variance (ANOVA) and Tukey. The applied OAT method achieves higher accuracy (0.823) than the ANOVA method (0.811) and effectively reduces the overfitting observed in previous approaches.
Keywords
convolutional neural networks hyperparameter tuning; Orthogonal Array; structural health monitoring; Taguchi’s method
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
- Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(2), 281-305.
- Dorafshan, S., Maguire, M., & Marc, J. (2017). Autonomous detection of concrete cracks on bridge decks and fatigue cracks on steel members. Journal of Performance of Constructed Facilities, 31(6), Article 04017080.
- Fraley, S., Oom, M., Terrien, B., & Date, J. Z. (2006). Design of experiments via Taguchi methods: orthogonal arrays. The Michigan Chemical Process Dynamic and Controls Open Text Book, 2(3), 4.
- He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770-778.
- Huang, G., Liu, S., Van der Maaten, L., & Weinberger, K. Q. (2018). Condensenet: An efficient densenet using learned group convolutions. In (IE) (Eds.), Proceedings of the IEEE conference on computer vision and pattern recognition.
- Kalaiselvi, T., & Padmapriya, S. (2021). Brain tumor diagnostic system-a deep learning application. In Machine vision inspection systems: Volume 2. Machine learning‐based approaches (pp. 69-90). Wiley.
- Karna, S. K., & Sahai, R. (2012). An overview on Taguchi method. International Journal of Engineering and Mathematical Sciences, 1(1), 1-7.
- Klein, A., Falkner, S., Bartels, S., Hennig, P., & Hutter, F. (2017). Fast Bayesian optimization of machine learning hyperparameters on large datasets. In A. Singh & J. Zhu (Eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (Vol. 54, pp. 528-536). PMLR. https://proceedings.mlr.press/v54/klein17a.html.
- Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), Advances in neural information processing systems (Vol. 25, pp. 1097-1105). Curran Associates. https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
- Lazić, L. (2013). Use of orthogonal arrays and design of experiments via Taguchi methods in software testing. Recent Advances in Applied Theoretical Mechanics, 23(2), 256-267.
- Li, X., Guo, H., Xu, L., & Xing, Z. (2023). Bayesian-based hyperparameter optimization of 1D-CNN for structural anomaly detection. Sensors, 23(11), Article 5058.
- Ottoni, A. L. C., Novo, M. S., & Oliveira, M. S. (2023). A statistical approach to hyperparameter tuning of deep learning for construction machine classification. Arabian Journal for Science and Engineering, 49(2), 1-12
- Puruncajas, B., Vidal, Y., & Tutivén, C. (2020). Vibration-response-only structural health monitoring for offshore wind turbine jacket foundations via convolutional neural networks. Sensors, 20(12), Article 3429.
- Seventekidis, P., Giagopoulos, D., Arailopoulos, A., & Markogiannaki, O. (2020). Structural health monitoring using deep learning with optimal finite element model generated data. Mechanical Systems and Signal Processing, 145, Article 106972.
- Taguchi, G. (1988). System of experimental design (Vol. 1). UNIPUB/Kraus International.
- Zhang, X., Chen, X., Yao, L., Ge, C., & Dong, M. (2019). Deep neural network hyperparameter optimization with orthogonal array tuning. arXiv. https://arxiv.org/abs/1907.13359