Movie Review Sentiment Analysis
A two-stage NLP investigation for a film studio researching audience sentiment ahead of a new science-fiction release: a classical machine learning baseline, followed by a systematic comparison of four recurrent neural network architectures on a harder, fine-grained sentiment benchmark.
The Challenge
Bidirectional LSTM classified movie review sentiment with 38.5% accuracy on a hard, five-class benchmark, while a simpler TF-IDF baseline hit 79% on binary sentiment, giving a film studio a validated way to compare audience reception across films with different production budgets.
The studio was planning a new science-fiction release and wanted to research audience feedback on comparable films, specifically looking for a possible relationship between positive sentiment and production budget. That required a reliable way to classify sentiment across a large volume of review text, plus a technical answer to a follow-on question: how much modelling sophistication is actually needed to do this well?
The project was run in two stages: first a classical ML baseline on binary-labelled reviews, then a deeper investigation using recurrent neural networks on a harder, five-class fine-grained sentiment benchmark, to see whether more complex architectures were worth their added cost.
Approach
Results
On the binary-labelled SST-2 data, a simple TF-IDF plus logistic regression baseline reached 79% accuracy, with one counterintuitive finding: retaining stopwords improved accuracy (79% vs 78%), challenging the common NLP default of removing them. For movie reviews specifically, function words appear to carry genuine sentiment signal rather than being pure noise.
On the harder, fine-grained SST-5 benchmark, Bidirectional LSTM was the clear winner at 38.51% accuracy, benefiting from processing sequences in both directions to capture fuller context, though it showed clear overfitting (training accuracy reached 85.82% while validation loss rose from 1.36 to 2.07). More surprisingly, the simplest architecture, Vanilla RNN, outperformed both LSTM and GRU (26.4% vs 23.1% each) on the test set, despite its theoretical vulnerability to vanishing gradients.
Reported in the interest of transparency: all four SST-5 architectures only modestly exceeded the 20% random baseline for this five-class task, underscoring how much harder fine-grained sentiment classification is than binary classification. This is an honest, useful negative result, not a headline metric — it argues for more training epochs, pre-trained embeddings, and regularisation before any production deployment.
Business Impact
For the studio's original question, the 79%-accurate binary model is sufficiently reliable to categorise sentiment across thousands of reviews and compare positive-sentiment share across films with different production budgets, directly answering the brief.
The deeper RNN comparison adds a second layer of value: it identifies which architecture is worth the added training and inference cost for more nuanced, multi-level sentiment work in future, and it surfaces concrete next steps, regularisation for Bidirectional LSTM, longer training for LSTM/GRU, and pre-trained embeddings (GloVe/Word2Vec) throughout, rather than shipping an unvalidated "best" model.