Sentiment Analysis of the Emirati Dialect text using Ensemble Stacking Deep Learning Models
Loading...
Date
2023-03
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
The British University in Dubai (BUiD)
Abstract
The study of thoughts, feelings, judgments, values, attitudes, and emotions regarding goods, services, organizations, persons, tasks, occasions, titles, and their attributes is known as sentiment analysis and it involves a polarity classification task for recognizing positive, negative, or neutral text to quantify what individuals believe using textual qualitative data. The rise of social media platforms provided an excellent source for sentiment analysis data. People use these platforms for various reasons, ranging from sharing their opinions and thoughts to gaining knowledge. Twitter, Instagram, and Facebook are examples of social media platforms. As more users join social media platforms, the amount of data that is generated online continues to grow at an accelerating pace. Most of the previous research that studied sentiment analysis for the Arabic language focused on Modern Standard Arabic and Egyptian, Saudi, Algerian, Jordanian, Tunisian, and Levantine Dialects. However, to our knowledge, no study involved employing deep learning models to conduct sentiment analysis on the Emirati dialect texts. Dialects are the informal form of a language. Each country of the Arab world has its own Dialect, and each dialect may have several sub-dialects.
The main objective of this study is to develop a deep learning model that outperforms the state-of-the-art for Sentiment Analysis of the Emirati Dialect. Toward this objective, I first conducted a systematic review to identify the research gaps in the existing literature and investigate the available constructed resources for Arabic dialects and the used approaches for sentiment analysis of Arabic dialects. The systematic review focused on empirical research on the subject of Sentiment Analysis of the Arabic Dialect that was released between January 2015 and January 2021. Through the analysis, I found, with the exception of a few articles that investigated Saudi, Levantine, Jordanian, Algerian, Tunisian, and Egyptian dialects, researchers rarely specified the dialect type in their papers; instead, it was mostly mentioned (MSA and Arabic Dialects). Emirati Dialect has not been explored for Sentiment Analysis purposes. The sizes of most datasets of previous research were between 10,000 and 50,000. Moreover, the Twitter platform was the most popular online platform for constructing Arabic datasets. Most of the studies evaluated basic Machine Learning approaches for Sentiment Analysis of Arabic Dialects. My research aims to fill these identified gaps as I detail below. Since Instagram is one of the most popular social media platforms in UAE, I constructed a dataset of the Emirati dialect from the Instagram platform. My dataset consists of 216,000 posts, of which 70,000 posts were manually annotated by three human annotators. Each post is annotated into (Positive/ Negative/Neutral), and it is further annotated into (Emirati Dialect/ Arabic Dialect/ MSA). In order for the dataset to be used as a benchmark, the inter-annotator agreement (IAA) was measured using Fleiss's Kappa coefficient. The findings reveal that the overall Fleiss Kappa coefficient is = 0.93, indicating an almost-perfect agreement amongst the three annotators.
Once the dataset was constructed and validated, I then conducted a performance evaluation and comparison of various basic Machine Learning algorithms, Deep Learning models, and stacking deep learning models on different datasets of Sentiment Analysis of Arabic Dialects. For the basic machine learning algorithms, LR, NB, SVM, RF, DT, MLP, AdaBoost, GBoost, and an ensemble model of machine learning classifiers were used. For deep-learning models, CNN, Bi-LSTM, Bi-GRU, as well as Hybrid deep-learning models were used for Sentiment Analysis. In order to improve performance further, I have proposed three ensemble-stacking deep-learning models with meta-learner layers of classifiers. The first stacking deep learning model combined 2 of the used deep learning models that produced the best results in terms of accuracy, the second stacking deep learning model combined 4 of the used deep learning models that produced the best results in terms of accuracy, and the final stacking deep-learning model combined all the trained deep learning models in this research. The proposed ensemble stacking model was evaluated using three datasets: the ESAAD Emirati Sentiment Analysis Annotated Dataset (which is one of this thesis contributions), and two other benchmark datasets (A Twitter-based Benchmark Arabic Sentiment Analysis Dataset ASAD and Arabic Company Reviews dataset). Experimental results show that my proposed ensemble stacking model outperformed existing deep learning models and achieved an accuracy of 95.54% for the ESAAD dataset, 96.71% for the benchmark ASAD, and 96.65% for the Arabic Company Reviews Dataset.
Description
Keywords
sentiment analysis, Emirati dialect text, deep learning models, social media platforms, Arab world