Student Non-Completion Prediction
Predicting which students will not complete their course, in three stages as more data becomes available: application, engagement and academic results.
The Challenge
With engagement data added, XGBoost identified non-completing students with an F1 score of 0.65, early enough in the course to act on.
An education provider wanted to know which students are at risk of not completing, and how early that can be predicted. Data arrives in stages: application details first, then engagement during the course, then assessment results. Each stage improves prediction but leaves less time to intervene.
Approach
Results
| XGBoost, non-completer class | F1 | F1 with SMOTE | ROC AUC |
|---|---|---|---|
| Stage 1 - application data | 0.614 | 0.524 | 0.879 |
| Stage 2 - + engagement | 0.650 | 0.609 | 0.914 |
| Stage 3 - + academic results | 0.879 | 0.880 | 0.998 |
Engagement data gave the best balance of accuracy and lead time. SMOTE did not help: it lowered F1 in stages 1 and 2.
Stage 3's near-perfect AUC suggests the academic-results data may already encode the outcome, so it needs checking for leakage before being used as an early-warning signal.
Correction: an earlier version of this page quoted recall and F1 figures produced by the inverted evaluation; they have been withdrawn. Neural-network and tuned-threshold results need a re-run and are not quoted.
How it was tested
- Every model evaluated on the non-completer class, at each of the three data stages.
- SMOTE oversampling tested against the unbalanced baseline; decision thresholds explored.
- A 2026 review found the threshold-based evaluation had inverted the predicted labels for the non-completer class. The code was corrected and the affected figures were withdrawn.
- SMOTE lowered F1 in stages 1 and 2.
- Stage 3's near-perfect AUC (0.998) suggests the academic-results data may already encode the outcome; it needs a leakage check before use as an early warning.
- Neural-network and tuned-threshold results need a re-run and are not quoted.