Multimodal Perception and Adaptive Control in Intelligent Robotic Fabrication Systems
Keywords:
multimodal perception; adaptive control; robotic fabrication; uncertainty quantification; sensor fusion; cyber-physical constructionAbstract
Robotic fabrication systems increasingly operate in unstructured or semi-structured environments where material behavior, tool dynamics, and environmental conditions vary during execution. Conventional preprogrammed toolpaths and single-modality sensing are poorly suited to this setting because they cannot resolve the coupled uncertainty that arises when a robot manipulates compliant, partially cured, or geometrically variable materials. This manuscript develops a methodological framework for multimodal perception and adaptive control in intelligent robotic fabrication. The central contribution is an uncertainty-aware architecture that fuses vision, force, pose, and process observations into a fabrication state estimate, propagates that estimate through a Gaussian-process material model, and updates a constrained adaptive control law in real time. The framework treats fabrication as a partially observable dynamic process rather than a static path-following problem. It specifies the fabrication state, the multimodal observation model, a recursive Bayesian estimator, a computed-torque adaptive controller with parameter adaptation, a covariance propagation relation, and a risk-sensitive decision rule for selecting control actions under material and sensing uncertainty. Analytical discussion clarifies stability conditions, calibration requirements, failure modes, and trade-offs between model fidelity and responsiveness. The framework is intended to support robotic systems that must maintain geometric tolerance and process quality when material response cannot be fully characterized before execution. Its significance lies in connecting perception, material inference, and control through explicit uncertainty terms, thereby providing a technically grounded basis for intelligent fabrication systems that adapt during operation rather than after error accumulation.
References
1. Keating, S. J., Spielberg, N. A., Klein, J., & Oxman, N. (2017). Toward site-specific and self-sufficient robotic fabrication on architectural scales. Science Robotics, 2(5), eaam8986.
2. Brell-Çokcan, S., & Braumann, J. (Eds.). (2013). Robotic fabrication in architecture, art and design 2012. Springer.
3. Perez, L., Rodríguez, Í., Rodríguez, N., Usamentiaga, R., & García, D. F. (2016). Robot guidance using machine vision techniques in industrial environments: A comparative review. Sensors, 16(3), 335.
4. Khaleghi, B., Khamis, A., Karray, F. O., & Razavi, S. N. (2013). Multisensor data fusion: A review of the state-of-the-art. Information Fusion, 14(1), 28-44.
5. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press.
6. Luo, D., Chen, D., Wang, J., Zhu, G., & Xu, W. (2020). The smart robot crafting approach to computing materials. Construction Robotics, 4(3), 239-249.
7. Deisenroth, M. P., Fox, D., & Rasmussen, C. E. (2015). Gaussian processes for data-efficient learning in robotics and control. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(2), 408-423.
8. Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016). Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Transactions on Robotics, 32(6), 1309-1332.
9. Thrun, S., Burgard, W., & Fox, D. (2005). Probabilistic robotics. MIT Press.
10. Slotine, J.-J. E., & Li, W. (1991). Applied nonlinear control. Prentice Hall.
11. Siciliano, B., & Khatib, O. (Eds.). (2016). Springer handbook of robotics (2nd ed.). Springer.
12. Khalil, H. K. (2002). Nonlinear systems (3rd ed.). Prentice Hall.
13. Hall, D. L., & Llinas, J. (1997). An introduction to multisensor data fusion. Proceedings of the IEEE, 85(1), 6-23.
14. Ko, J., & Fox, D. (2009). GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models. Autonomous Robots, 27(1), 75-90.
15. Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2010). Robotics: Modelling, planning and control. Springer.
16. Rus, D., & Tolley, M. T. (2015). Design, fabrication and control of soft robots. Nature, 521(7553), 467-475.
17. Gibson, I., Rosen, D. W., & Stucker, B. (2015). Additive manufacturing technologies: 3D printing, rapid prototyping, and direct digital manufacturing (2nd ed.). Springer.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Artificial Intelligence Engineering and Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
