Comparative Analysis of Machine Learning and Deep Learning Approaches for Sentiment Analysis across Heterogeneous Datasets under Class Imbalance Conditions

Abstract

Sentiment analysis across multiple domains remains a significant challenge in natural language processing, mainly due to domain shift, where a model trained on one domain tends to experience performance degradation when applied to another domain. To address this issue, this paper presents a comparative analysis of machine learning and deep learning approaches for cross-domain sentiment analysis on three different datasets COVIDSENTI, Yelp reviews and IMDb movie reviews. These datasets were intentionally selected to reflect different text-length categories, namely short, medium, and long texts. Such variation allows a more comprehensive evaluation of model performance under different linguistic characteristics and textual complexities across domains. Three class imbalance handling strategies were studied with no balancing, ROS, and ROS combined with class weighting. The models were evaluated including LR, SVM, NB, CNN, BiLSTM and BERT. The results show that ROS greatly enhanced the F1-score for imbalanced datasets and COVIDSENTI achieved the greatest improvement +8.83% for LR. The results indicated that BERT outperformed machine learning and deep learning baselines with an F1-score of 95.32%, 73.79%, and 75.76% on COVIDSENTI, Yelp, and IMDb accordingly. An extra benefit of class weighting was shown in deep learning models on COVIDSENTI but not for machine learning models as an improvement after balancing with ROS. Future work will construct a hybrid BERT+BiLSTM+Attention framework for improving cross-domain generalization.