Context is All You Need: Why Transformers Outperform Traditional ML in Tweet Sentiment

Abstract

This research paper presents a comprehensive comparative analysis of sentiment analysis techniques applied to Twitter data using Natural Language Processing (NLP) methodologies. The study evaluates the performance of conventional machine learning algorithms alongside state-of-the-art transformer-based models on the Sentiment140 dataset containing 1.6 million tweets. We implement and compare four distinct approaches: Naive Bayes and Support Vector Machine (SVM) as traditional machine learning baselines, RoBERTa as an encoder-only transformer, and GPT-2 as a decoder-only transformer architecture. Our methodology encompasses extensive preprocessing pipelines, feature extraction techniques, and rigorous evaluation metrics including accuracy, precision, recall, and F1-score. Experimental results demonstrate that transformer-based models significantly outperform conventional approaches, with RoBERTa achieving 93.87% accuracy and GPT-2 attaining 89.01% accuracy, compared to SVM's 86.22% and Naive Bayes' 75.45%. The findings highlight the superior capability of transformer architectures in capturing contextual semantics and handling the complexities inherent in social media text, establishing new benchmarks for Twitter sentiment analysis applications.