Comparative Analysis of Machine Learning and Deep Learning Approaches for Sentiment Analysis across Heterogeneous Datasets under Class Imbalance Conditions
DOI:
https://doi.org/10.53840/e-jpi.v13i2.422Keywords:
Sentiment analysis, cross-domain, machine learning, deep learning, BERT, class imbalance, Random OverSampling, CNN, BiLSTMAbstract
Sentiment analysis across multiple domains remains a significant challenge in natural language processing, mainly due to domain shift, where a model trained on one domain tends to experience performance degradation when applied to another domain. To address this issue, this paper presents a comparative analysis of machine learning and deep learning approaches for cross-domain sentiment analysis on three different datasets COVIDSENTI, Yelp reviews and IMDb movie reviews. These datasets were intentionally selected to reflect different text-length categories, namely short, medium, and long texts. Such variation allows a more comprehensive evaluation of model performance under different linguistic characteristics and textual complexities across domains. Three class imbalance handling strategies were studied with no balancing, ROS, and ROS combined with class weighting. The models were evaluated including LR, SVM, NB, CNN, BiLSTM and BERT. The results show that ROS greatly enhanced the F1-score for imbalanced datasets and COVIDSENTI achieved the greatest improvement +8.83% for LR. The results indicated that BERT outperformed machine learning and deep learning baselines with an F1-score of 95.32%, 73.79%, and 75.76% on COVIDSENTI, Yelp, and IMDb accordingly. An extra benefit of class weighting was shown in deep learning models on COVIDSENTI but not for machine learning models as an improvement after balancing with ROS. Future work will construct a hybrid BERT+BiLSTM+Attention framework for improving cross-domain generalization.
Downloads
References
Abdullah, T., & Ahmet, A. (2023). Deep Learning in Sentiment Analysis: Recent Architectures. ACM Computing Surveys, 55(8). https://doi.org/10.1145/3548772
Agrawal, R., Majumder, M., Yadav, I., Taneja, N., Hamdare, S., & Hemnani, P. (2025). Evaluating sentiment analysis models: A comparative analysis of vaccination tweets during the COVID-19 phase leveraging DistilBERT for enhanced insights. MethodsX, 14. https://doi.org/10.1016/j.mex.2025.103407
Alahmadi, K., Alharbi, S., Chen, J., & Wang, X. (2025). Generalizing sentiment analysis: a review of progress, challenges, and emerging directions. In Social Network Analysis and Mining (Vol. 15, Number 1). Springer. https://doi.org/10.1007/s13278-025-01461-8
Bordoloi, M., & Biswas, S. K. (2023). Sentiment analysis: A survey on design framework, applications and future scopes. Artificial Intelligence Review, 56(11), 12505–12560. https://doi.org/10.1007/s10462-023-10442-2
Devlin, J., Chang, M.-W., Lee, K., & Kristina Toutanova. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186. https://doi.org/10.18653/v1/N19-1423
Du, Y., He, M., Wang, L., & Zhang, H. (2020). Wasserstein based transfer network for cross-domain sentiment classification. Knowledge-Based Systems, 204. https://doi.org/10.1016/j.knosys.2020.106162
Fujiwara, K. (2024). Knowledge distillation with resampling for imbalanced data classification: Enhancing predictive performance and explainability stability. Results in Engineering, 24. https://doi.org/10.1016/j.rineng.2024.103406
Ganganwar, V., & Rajalakshmi, R. (2024). Employing synthetic data for addressing the class imbalance in aspect-based sentiment classification. Journal of Information and Telecommunication, 8(2), 167–188. https://doi.org/10.1080/24751839.2023.2270824
He, Y. (2023). BERT-CNN-BiLSTM: A Hybrid Deep Learning Model for Accurate Sentiment Analysis. 2023 IEEE 5th International Conference on Power, Intelligent Computing and Systems, ICPICS 2023, 921–926. https://doi.org/10.1109/ICPICS58376.2023.10235335
Kumar, A., Abhishek, K., & Shafeeq B M, A. (2025). SentXFormer: a transformer-enhanced hybrid deep learning framework for cross-domain sentiment analysis of customer reviews. Scientific Reports. https://doi.org/10.1038/s41598-025-33526-1
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011). Learning Word Vectors for Sentiment Analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 142–150.
Mao, Y., Liu, Q., & Zhang, Y. (2024). Sentiment analysis methods, applications, and challenges: A systematic literature review. In Journal of King Saud University - Computer and Information Sciences (Vol. 36, Number 4). King Saud bin Abdulaziz University. https://doi.org/10.1016/j.jksuci.2024.102048
Mohd Sofi, S., & Selamat, A. (2023). Aspect Based Sentiment Analysis: Feature Extraction using Latent Dirichlet Allocation (LDA) and Term Frequency - Inverse Document Frequency (TF-IDF) in Machine Learning (ML). Malaysian Journal of Information and Communication Technology (MyJICT), 169–179. https://doi.org/10.53840/myjict8-2-102
Mohd Sofi, S., Selamat, A., Alwani Shaffiei, Z., Kuala Lumpur, M., Sultan Yahya Petra, J., & Lumpur, K. (2026). An Adaptive Ensemble Machine Learning Classifier for Sentiment Analysis on Twitter. Journal of Advanced Research Design Journal Homepage: Journal of Advanced Research Design, 136, 340–357. https://doi.org/10.37934/ard.136.1.340357
Naseem, U., Razzak, I., Khushi, M., Eklund, P. W., & Kim, J. (2021). COVIDSenti: A Large-Scale Benchmark Twitter Data Set for COVID-19 Sentiment Analysis. IEEE Transactions on Computational Social Systems, 8(4), 976–988. https://doi.org/10.1109/TCSS.2021.3051189
Permataning Tyas, S. M., Sarno, R., Haryono, A. T., & Rossa Sungkono, K. (2023). A Robustly Optimized BERT using Random Oversampling for Analyzing Imbalanced Stock News Sentiment Data. ICCoSITE 2023 - International Conference on Computer Science, Information Technology and Engineering: Digital Transformation Strategy in Facing the VUCA and TUNA Era, 897–902. https://doi.org/10.1109/ICCoSITE57641.2023.10127725
Ramaziyah, Y. A., & Setiawan, E. B. (2024). Hybrid Deep Learning CNN and BiLSTM with FastText as Feature Expansion for Sentiment Analysis in President Election 2024. COMNETSAT 2024 - IEEE International Conference on Communication, Networks and Satellite, 176–183. https://doi.org/10.1109/COMNETSAT63286.2024.10862946
Siino, M., Tinnirello, I., & La Cascia, M. (2024). Is text preprocessing still worth the time? A comparative survey on the influence of popular preprocessing methods on Transformers and traditional classifiers. Information Systems, 121. https://doi.org/10.1016/j.is.2023.102342
Talukder, M. A., Uddin, M. A., Roy, S., Ghose, P., Sarker, S., Khraisat, A., Kazi, M., Rahman, M. M., & Hakimi, M. (2025). A hybrid deep learning model for sentiment analysis of COVID-19 tweets with class balancing. Scientific Reports, 15(1). https://doi.org/10.1038/s41598-025-97778-7
Tsirmpas, D., Gkionis, I., Papadopoulos, G. T., & Mademlis, I. (2024). Neural natural language processing for long texts: A survey on classification and summarization. In Engineering Applications of Artificial Intelligence (Vol. 133). Elsevier Ltd. https://doi.org/10.1016/j.engappai.2024.108231
Wijayashantha, A. K. S., Samarasinghe, U. S., Perera, H. A. D. U., & Subhashini, L. D. C. S. (2026). An Attention Enhanced CNN BERT Model with TF IDF Feature Fusion for Cross Domain Sentiment Analysis. 2026 6th International Conference on Advanced Research in Computing: Responsible AGI: Balancing Intelligence, Responsibility and Sustainability, ICARC 2026 - Conference Proceedings. https://doi.org/10.1109/ICARC68737.2026.11453951
Yang, S., Xing, J., Dong, Z., & Liu, Z. (2025). Heterogeneous Ensemble Sentiment Classification Model Integrating Multi-View Features and Dynamic Weighting. Electronics (Switzerland), 14(21). https://doi.org/10.3390/electronics14214189
Younesi, R. T., Tanha, J., Namvar, S., & Mostafaei, S. H. (2024). A CNN-BiLSTM based deep learning model to sentiment analysis. 2024 20th CSI International Symposium on Artificial Intelligence and Signal Processing, AISP 2024. https://doi.org/10.1109/AISP61396.2024.10475311
Zhang, X., & LeCun, Y. (2015). Text Understanding from Scratch. Advances in Neural Information Processing Systems 28 (NIPS 2015). http://arxiv.org/abs/1502.01710
Zou, H., & Wang, Y. (2025). Large language model augmented syntax-aware domain adaptation method for aspect-based sentiment analysis. Neurocomputing, 625. https://doi.org/10.1016/j.neucom.2025.129472
Downloads
Published
Issue
Section
License
Copyright (c) 2026 e-Jurnal Penyelidikan dan Inovasi
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
