Explainable Artificial Intelligence for Interpreting Human Perception of Urban Streetscapes
Keywords:
streetscape perception; explainable artificial intelligence; concept bottleneck models; urban analytics; ordinal regression; spatial uncertaintyAbstract
Machine learning models now predict perceived safety, beauty, and walkability from street-level imagery, but their explanations often remain pixel-level and poorly aligned with the concepts urban designers and residents use. This paper develops a concept-mediated, spatially aware framework for explainable artificial intelligence in streetscape perception research. The framework treats perception ratings as ordinal, rater-dependent measurements of latent constructs; constrains visual models through interpretable concepts such as greenery, enclosure, maintenance, and human presence; decomposes predictions into concept contributions; and propagates uncertainty and spatial dependence into explanation. It also specifies evaluation logic for fidelity, stability, human interpretability, and spatial residual diagnostics. The contribution is methodological rather than empirical: it links perceptual measurement theory, concept-based vision models, attribution analysis, and spatial statistics into a single inferential workflow. The framework clarifies why objective scene descriptors and subjective ratings may cohere or diverge, how correlated urban features can produce misleading attributions, and which validation claims are defensible at the image, rater, and neighborhood scales. Its significance lies in making streetscape AI explanations auditable for design review, public deliberation, and urban policy while avoiding the false precision of unvalidated saliency maps.
References
1. Ewing, R., & Handy, S. (2009). Measuring the unmeasurable: Urban design qualities related to walkability. Journal of Urban Design, 14(1), 65–84. https://doi.org/10.1080/13574800802451155
2. Naik, N., Philipoom, J., Raskar, R., & Hidalgo, C. (2014). Streetscore: Predicting the perceived safety of one million streetscapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 779–785). IEEE.
3. Dubey, A., Naik, N., Parikh, D., Raskar, R., & Hidalgo, C. A. (2016). Deep learning the city: Quantifying urban perception at a global scale. In Computer Vision – ECCV 2016 (pp. 196–212). Springer.
4. Zhang, F., Zhou, B., Liu, L., Liu, Y., Fung, H. H., Lin, H., & Ratti, C. (2018). Measuring human perceptions of a large-scale urban region using machine learning. Landscape and Urban Planning, 180, 148–160. https://doi.org/10.1016/j.landurbplan.2018.08.020
5. Salesses, P., Schechtner, K., & Hidalgo, C. A. (2013). The collaborative image of the city: Mapping the inequality of urban perception. PLoS ONE, 8(7), e68400. https://doi.org/10.1371/journal.pone.0068400
6. Liu, L., Silva, E. A., Wu, C., & Wang, H. (2017). A machine learning-based method for the large-scale evaluation of the qualities of the urban environment. Computers, Environment and Urban Systems, 65, 113–125. https://doi.org/10.1016/j.compenvurbsys.2017.06.003
7. Song, Q., Li, M., Qiu, W., Li, W., & Luo, D. (2022, November). The coherence and divergence between the objective and subjective measurement of street perceptions for Shanghai. In International Conference on Advanced Data Mining and Applications (pp. 244-256). Cham: Springer Nature Switzerland.
8. Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., & Müller, K.-R. (Eds.). (2019). Explainable AI: Interpreting, explaining and visualizing deep learning. Springer.
9. Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30 (pp. 4765–4774). Curran Associates.
10. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should I trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144). ACM.
11. Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2018). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning (pp. 2668–2677). PMLR.
12. Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., & Liang, P. (2020). Concept bottleneck models. In Proceedings of the 37th International Conference on Machine Learning (pp. 5338–5348). PMLR.
13. Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x
14. Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv:1702.08608.
15. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778). IEEE.
16. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations.
17. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (pp. 8748–8763). PMLR.
18. Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (pp. 1050–1059). PMLR.
19. Kendall, A., & Gal, Y. (2017). What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems 30 (pp. 5574–5584). Curran Associates.
20. Anselin, L. (1995). Local indicators of spatial association—LISA. Geographical Analysis, 27(2), 93–115. https://doi.org/10.1111/j.1538-4632.1995.tb00338.x
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Artificial Intelligence Engineering and Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
