Federated Learning for Privacy-Preserving Sentiment Analysis: An Adaptive Attention Approach for Distributed Text Mining
Keywords:
Federated Learning; Sentiment Analysis; Privacy-Preserving Machine Learning; Adaptive Attention; Distributed Text Mining; Differential Privacy; Edge Computing; Fairness; GovernanceAbstract
Sentiment analysis has become indispensable for extracting public opinion from massive streams of user-generated text, yet centralized collection and processing of such data raises profound privacy and governance concerns. Federated learning offers a paradigm in which machine learning models are trained collaboratively across distributed data silos without raw data leaving client devices, thus preserving data locality and user confidentiality. However, text sentiment is highly context-dependent and exhibits non-independent and identically distributed characteristics across devices, rendering static global models ineffective. This paper presents a federated learning framework for privacy-preserving sentiment analysis that integrates an adaptive attention mechanism into the global model architecture. The adaptive attention module dynamically modulates the importance of different contextual features based on local data distributions while supporting secure aggregation and differential privacy at the server. We provide a systems-oriented analysis, focusing on architectural trade-offs between model expressiveness and communication efficiency, the interplay between dynamic adaptivity and privacy budgets, robustness to heterogeneous and adversarial clients, and fairness across diverse linguistic communities. The discussion extends to infrastructure deployment across edge-cloud continuums, sustainability considerations arising from distributed training, and the broader policy implications for compliance with data protection regulations and algorithmic accountability. By treating federated sentiment analysis as a socio-technical infrastructure, the paper highlights that adaptive attention, when carefully integrated with privacy-preserving protocols, can reconcile the conflicting demands of personalization, performance, and privacy in large-scale distributed text mining.
References
1. McMahan, B., Moore, E., Ramage, D., Hampson, S., & y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (pp. 1273–1282).
2. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998–6008).
3. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1480–1489).
4. Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016). Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (pp. 308–318).
5. Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017). Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (pp. 1175–1191).
6. Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., & others. (2021). Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1–2), 1–210.
7. Sukhbaatar, S., Grave, E., Bojanowski, P., & Joulin, A. (2019). Adaptive attention span in Transformers. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 331–335).
8. Lin, B. Y., He, C., Zeng, Z., Wang, H., Huang, Y., Soltanolkotabi, M., Ren, X., & Avestimehr, S. (2022). FedNLP: Benchmarking federated learning methods for natural language processing tasks. In Findings of the Association for Computational Linguistics: NAACL 2022 (pp. 157–175).
9. Mohri, M., Sivek, G., & Suresh, A. T. (2019). Agnostic federated learning. In Proceedings of the 36th International Conference on Machine Learning (pp. 4615–4625).
10. Dean, J., Corrado, G. S., Monga, R., Chen, K., Devin, M., Le, Q. V., Mao, M. Z., Ranzato, M., Senior, A., Tucker, P., Yang, K., & Ng, A. Y. (2012). Large scale distributed deep networks. In Advances in Neural Information Processing Systems (pp. 1223–1231).
11. Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019). Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proceedings of the IEEE, 107(8), 1738–1762.
12. Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečný, J., Kumar, S., & McMahan, H. B. (2021). Adaptive federated optimization. In Proceedings of the 9th International Conference on Learning Representations.
13. Li, Q. (2026). Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classification. arXiv preprint arXiv:2604.10459.
14. Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019). Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology, 10(2), 1–19.
15. Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017). Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (pp. 3–18).
16. Gupta, V., & Lehal, G. S. (2009). A survey of text mining techniques and applications. Journal of Emerging Technologies in Web Intelligence, 1(1), 60–76.
17. Liu, B. (2012). Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1), 1–167.
18. McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2018). Learning differentially private recurrent language models. In Proceedings of the 6th International Conference on Learning Representations.
19. Blanchard, P., El Mhamdi, E. M., Guerraoui, R., & Stainer, J. (2017). Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems (pp. 119–129).
20. Rieke, N., Hancox, J., Li, W., Milletarì, F., Roth, H. R., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B. A., Maier-Hein, K., & others. (2020). The future of digital health with federated learning. NPJ Digital Medicine, 3(1), 1–7.
21. Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An ethical framework for a good AI society. Minds and Machines, 28(4), 689–707.
22. Veale, M., Binns, R., & Edwards, L. (2018). Algorithms that remember: Model inversion attacks and data protection law. Philosophical Transactions of the Royal Society A, 376(2133), 20180083.
23. Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness and machine learning: Limitations and opportunities. fairmlbook.org.
24. Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30–39.
25. Gao, T., Yao, X., & Chen, D. (2021). SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (pp. 6894–6910).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Artificial Intelligence Engineering and Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
