Computer Vision-Based Progress Monitoring and Deviation Detection in Robotic Construction Operations
Keywords:
robotic construction; computer vision; scan-to-BIM registration; progress monitoring; deviation detection; uncertainty-aware perceptionAbstract
Robotic construction operations require continuous awareness of what has been built and how far the as-built state departs from the intended design. Project-level monitoring systems based on periodic imagery, laser scans, or manual inspection are poorly matched to task-level control because they do not resolve element placement, occlusion, registration uncertainty, or tolerance violations at the pace of a robotic work cycle. This paper develops a methodological framework for computer vision-based progress monitoring and deviation detection in robotic construction operations. The framework couples robot pose, multi-view images, depth or point-cloud observations, and building information model element graphs within a shared coordinate representation. Element-level progress is inferred as a probabilistic state from visual and geometric evidence, while geometric deviation is treated separately as a registered residual against element-specific tolerances. A robust scan-to-model registration objective, a progress-evidence model, a tolerance-exceedance measure, a Bayesian temporal fusion rule, a cost-sensitive decision rule, and an active-view selection criterion are formulated. The central argument is that progress and conformity should not be conflated: absence of evidence for completion differs from evidence of geometric error, and temporal fusion plus viewpoint planning can reduce false alarms under occlusion and partial observability. The framework provides a technically testable basis for supervisory control, corrective robot action, and inspection planning in robotic construction.
References
1. Teizer, J. (2015). Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites. Advanced Engineering Informatics, 29(2), 225–238.
2. Golparvar-Fard, M., Peña-Mora, F., & Savarese, S. (2009). D4AR–a 4-dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication. Journal of Information Technology in Construction, 14, 129–153.
3. Bosché, F. (2010). Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction. Advanced Engineering Informatics, 24(1), 107–118.
4. Han, K. K., & Golparvar-Fard, M. (2015). Appearance-based material classification for monitoring of operation-level construction progress using 4D BIM and site photologs. Automation in Construction, 53, 44–57.
5. Wang, Q., & Kim, M. K. (2019). Applications of 3D point cloud data in the construction industry: A fifteen-year review from 2004 to 2018. Advanced Engineering Informatics, 39, 306–319.
6. Ham, Y., Han, K. K., Lin, J. J., & Golparvar-Fard, M. (2016). Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): a review of related works. Visualization in Engineering, 4(1), 1–8.
7. Liang, C. J., Lundeen, K. M., McGee, W., Menassa, C. C., & Kamat, V. R. (2019). A vision-based marker-less pose estimation system for articulated construction robots. Automation in Construction, 104, 80–94.
8. Asadi, K., Ramshankar, H., Noghabaei, M., & Han, K. (2019). Real-time image localization and registration with BIM using perspective alignment for indoor monitoring of construction. Journal of Computing in Civil Engineering, 33(5), 04019031.
9. Kim, M. K., Sohn, H., & Chang, C. C. (2014). Automated dimensional quality assessment of precast concrete panels using terrestrial laser scanning. Automation in Construction, 45, 163–177.
10. Tang, P., Huber, D., Akinci, B., Lipman, R., & Lytle, A. (2010). Automatic reconstruction of as-built building information models from laser-scanned point clouds: A review of related techniques. Automation in Construction, 19(7), 829–843.
11. Xiong, X., Adan, A., Akinci, B., & Huber, D. (2013). Automatic creation of semantically rich 3D building models from laser scanner data. Automation in Construction, 31, 325–337.
12. Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.
13. Newcombe, R. A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohli, P., Shotton, J., Hodges, S., & Fitzgibbon, A. (2011). KinectFusion: Real-time dense surface mapping and tracking. In 2011 10th IEEE International Symposium on Mixed and Augmented Reality (pp. 127–136). IEEE.
14. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
15. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).
16. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 779–788).
17. Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 652–660).
18. Luo, D., & Yu, L. (2021). From factory to site—designing for industrial robots used in on-site construction. In Automating cities: design, construction, operation and future impact (pp. 87-109). Singapore: Springer Singapore.
19. Kim, M. K., Cheng, J. C., Sohn, H., & Chang, C. C. (2015). A framework for dimensional and surface quality assessment of precast concrete elements using BIM and 3D laser scanning. Automation in Construction, 49, 225–238.
20. Bircher, A., Kamel, M., Alexis, K., Oleynikova, H., & Siegwart, R. (2016). Receding horizon next-best-view planner for 3D exploration. In 2016 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1462–1468). IEEE.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Artificial Intelligence Engineering and Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
