titanic_nn_analysis

Project Methodology & Process Flow

1

Data Preprocessing

Load Titanic dataset, handle missing values (Age, Embarked), drop irrelevant columns, apply one-hot encoding for categorical variables

2

Data Splitting & Scaling

Split into train/validation/test sets (80/10/10), apply stratification to maintain class balance, standardise features

3

Basic Model Comparison

Create identical neural network architectures with Adam and RMSprop optimisers, train for 10 epochs, evaluate performance

4

Regularisation Implementation

Add L2 regularisation (0.01) and dropout (0.1) to prevent overfitting, compare performance across optimisers

5

Early Stopping Integration

Implement early stopping callback with patience=1, monitor validation loss to optimise training efficiency

6

Cross-Validation Analysis

Perform 5-fold stratified cross-validation to ensure robust performance estimates and reduce overfitting bias

Comprehensive Activity Analysis

Basic Optimiser Comparison

Model Architecture:

Input layer: neurons equal to feature count
Hidden layer: 64 neurons (ReLU activation)
Hidden layer: 32 neurons (ReLU activation)
Output layer: 1 neuron (Sigmoid activation)

Key Observations:

Adam showed faster initial convergence
RMSprop demonstrated more stable validation loss
Both achieved similar final accuracy (~80%)
Adam exhibited slight overfitting after epoch 6

Regularisation Techniques

Regularisation Methods:

L2 regularisation: kernel_regularizer=l2(0.01)
Dropout layers: rate=0.1 (10% neurons dropped)

Impact Assessment:

Reduced overfitting in both optimisers
More stable validation performance
Slightly lower training accuracy (expected)
Better generalisation capabilities

Early Stopping Implementation

Callback Configuration:

Monitor: validation loss
Patience: 1 epoch
Min_delta: 0.001
Restore_best_weights: True

Training Efficiency:

Prevented unnecessary training epochs
Maintained model performance
Reduced computational overhead
Optimised training-validation balance

Cross-Validation Results

5-Fold Stratified Cross-Validation:

Adam: 82.83% ± 4.78% accuracy
RMSprop: 81.93% ± 4.30% accuracy

Statistical Significance:

Overlapping confidence intervals
Marginal performance difference
Both optimisers show consistency
RMSprop slightly more stable

Detailed Performance Metrics Comparison

Model Configuration	Accuracy	Precision	Recall	F1 Score	Loss
Model - Adam	0.8249	0.8385	0.6754	0.7474	0.4641
Model - RMSprop	0.8238	0.8428	0.6667	0.7439	0.6414
Model - Adam	0.8216	0.8235	0.6843	0.7464	Not specified
Model - RMSprop	0.8204	0.8288	0.6726	0.7413	Not specified

Best Overall Performance

Model with Adam optimizer achieved the highest balanced performance with F1 Score: 0.7474 and Accuracy: 82.49%

Key Findings & Conclusions

Optimiser Performance

Adam demonstrates slight superiority in balanced performance metrics, particularly F1 score and overall accuracy. RMSprop excels in precision, making fewer false positive predictions.

Regularisation Impact

L2 regularisation and dropout successfully reduced overfitting whilst maintaining model performance. Both techniques improved validation stability across optimisers.

Early Stopping Effectiveness

Early stopping proved valuable for training efficiency without compromising performance. Optimal for preventing overfitting whilst reducing computational costs.

Cross-Validation Insights

Robust validation confirmed consistent performance across data splits. Confidence intervals overlap, suggesting practical equivalence between optimisers.

Model Selection Criteria

Choice between optimisers depends on specific requirements: Adam for balanced performance, RMSprop for high precision scenarios, considering recall needs.

Statistical Significance

Performance differences are marginal but consistent. Adam shows ~0.9% higher mean accuracy with slightly higher variance in cross-validation results.

Business Implications & Recommendations

Production Model Selection

Recommended: Model with Adam optimiser
Best balanced performance across all metrics
82.49% accuracy with robust F1 score of 0.7474
Suitable for general-purpose survivor prediction
Effective with L2 regularisation and dropout

Risk Management Applications

High-Precision Scenarios: Use RMSprop optimiser
84.28% precision minimises false positive predictions
Ideal for resource-constrained rescue operations
Reduces unnecessary resource allocation
Conservative approach to survivor identification

Comprehensive Survivor Detection

High-Recall Priority: Model with Adam
68.43% recall captures maximum survivors
Critical for emergency response scenarios
Minimises missed survivor identifications
Acceptable trade-off for life-saving applications

Implementation Strategy

Deploy ensemble of both optimisers for robustness
Implement cross-validation in production pipelines
Monitor model performance with early stopping
Regular retraining with new historical data
A/B testing for optimiser performance validation

Technical Infrastructure

Standardised feature preprocessing pipeline
Automated model retraining workflows
Performance monitoring and alerting systems
Version control for model iterations
Rollback capabilities for model deployment

Future Enhancements

Hyperparameter tuning for optimised performance
Feature engineering for improved predictive power
Advanced architectures (deeper networks, attention)
Ensemble methods combining multiple models
Real-time inference capabilities

Strategic Recommendation

Deploy Model with Adam optimiser as the primary production model, with capability to switch to RMSprop configuration for high-precision requirements. Implement comprehensive monitoring and regular performance evaluation to maintain optimal predictive accuracy.

Neural Network Optimiser Comparison