A Deep Neural Network Chatbot for the Gulf Arabic Dialect: A Hybrid BiLSTM-Transformer Approach

Abstract

Artificial Intelligence (AI) is a technology that enables machines to mimic human intelligence, with core fields including Natural Language Processing (NLP) and Machine Learning (ML). Chatbot is a prominent AI application, that uses NLP techniques to engage in human-like conversations, enhancing human-machine interactions. This thesis explores the development of a chatbot that can automatically answer natural language questions, a key goal in AI. It provides a historical overview of chatbot evolution, generic workflow, and applications across various sectors. However, Chatbots are widely used, but there is a significant gap in systems specifically designed for the Gulf Emirati Arabic dialect, particularly in educational institutions and public sector universities, Existing systems are often trained on general corpora or other languages, highlighting a research gap in this area. Our proposed model addresses this gap by combining two advanced approaches in NLP. First, we employ Bidirectional Long Short-Term Memory (BiLSTM) networks for text generation, leveraging their ability to grasp contextual information and model long-term dependencies. Second, we integrate the Transformer model for both the encoding and decoding processes. This dual architecture enables our model to generate responses in Modern Standard Arabic (MSA) even when questions are posed in Gulf Arabic Dialect (GAD). The Transformer model was developed to handle MSA and GAD inputs in mixed-language environments. Three models were developed: BiLSTM, BiLSTM with Farasa Segmentation, and Hybrid BiLSTM-Transformer. After extensive experimentation, the Hybrid BiLSTM-Transformer model was the best-performing, achieving a BLEU score of 0.8674, 83% accuracy, and an F1 score of 0.86.