Titanic Survival Prediction: Optimiser Comparison
A controlled, progressively rigorous comparison of the Adam and RMSprop optimisers on a binary classification task — from baseline training through regularisation, early stopping, and 5-fold cross-validation — to establish an evidence-based optimiser selection methodology.
The Challenge
Adam and RMSprop land within a single percentage point of each other after 5-fold cross-validation (82.83% vs 81.93%) — proof that for this dataset, disciplined evaluation methodology matters far more than optimiser choice.
Choice of optimiser is one of the most consequential, and most under-examined, decisions in neural network training. This project used the classic Titanic survival dataset (891 records, 12 raw features) as a controlled test bed to systematically compare Adam against RMSprop, not just on raw accuracy, but through progressively more rigorous evaluation: baseline comparison, regularisation, early stopping, and cross-validation.
Approach
Results
Across every stage of evaluation, the two optimisers landed within a percentage point or two of each other, a result that is itself informative: for a small-to-medium tabular dataset like this, optimiser choice matters far less than getting preprocessing, regularisation, and validation strategy right. Adam consistently converged faster and edged out RMSprop on balanced accuracy and F1 score after cross-validation, while RMSprop consistently showed more stable validation loss and better precision, fewer false positives, at a small cost to recall.
Regularisation and early stopping made a real, measurable difference for both optimisers, reducing overfitting and improving validation stability, and the confidence intervals from 5-fold cross-validation overlap enough that the practical choice between Adam and RMSprop should be driven by the deployment context rather than treated as a settled question.
Business Impact
The recommendation is context-dependent rather than a single winner: Adam with early stopping for scenarios that need maximum survivor identification (higher recall, at the cost of some false positives), and RMSprop with regularisation where minimising false positives matters more than catching every case. More broadly, the project demonstrates a disciplined evaluation methodology, baseline → regularise → early-stop → cross-validate, that generalises well beyond this dataset to any binary classification problem where a confident, defensible model choice is needed.