Back to Projects
Supervised Learning / Education

Student Non-Completion Prediction

Predicting which students will not complete their course, in three stages as more data becomes available: application, engagement and academic results.

Type
Supervised Learning
Domain
Higher Education
Methods
XGBoost, Neural Network, SMOTE
Origin
Cambridge DS Career Accelerator coursework

The Challenge

With engagement data added, XGBoost identified non-completing students with an F1 score of 0.65, early enough in the course to act on.

An education provider wanted to know which students are at risk of not completing, and how early that can be predicted. Data arrives in stages: application details first, then engagement during the course, then assessment results. Each stage improves prediction but leaves less time to intervene.

Approach

01
Staged data
25,059 students with a non-completion rate of about 14.5%. Stage 1 used applicant and course data (16 columns), stage 2 added engagement (18), stage 3 added academic performance (21).
02
Models
For each stage, XGBoost tuned with randomised search and a Keras neural network, evaluated on the non-completer class.
03
Class imbalance
SMOTE oversampling tested against the unbalanced baseline; decision thresholds explored.
04
Evaluation review (2026)
A later review found the threshold-based evaluation had inverted the predicted labels for the non-completer class. The code was corrected; only results unaffected by the error are reported here.

Results

XGBoost, non-completer classF1F1 with SMOTEROC AUC
Stage 1 - application data0.6140.5240.879
Stage 2 - + engagement0.6500.6090.914
Stage 3 - + academic results0.8790.8800.998

Engagement data gave the best balance of accuracy and lead time. SMOTE did not help: it lowered F1 in stages 1 and 2.

Stage 3's near-perfect AUC suggests the academic-results data may already encode the outcome, so it needs checking for leakage before being used as an early-warning signal.

Correction: an earlier version of this page quoted recall and F1 figures produced by the inverted evaluation; they have been withdrawn. Neural-network and tuned-threshold results need a re-run and are not quoted.

How it was tested

TEST REPORTTEST REPORTSTUDENT DROPOUTSTUDENT DROPOUT
Checked
  • Every model evaluated on the non-completer class, at each of the three data stages.
  • SMOTE oversampling tested against the unbalanced baseline; decision thresholds explored.
Found and fixed
  • A 2026 review found the threshold-based evaluation had inverted the predicted labels for the non-completer class. The code was corrected and the affected figures were withdrawn.
Did not work
  • SMOTE lowered F1 in stages 1 and 2.
Limits
  • Stage 3's near-perfect AUC (0.998) suggests the academic-results data may already encode the outcome; it needs a leakage check before use as an early warning.
  • Neural-network and tuned-threshold results need a re-run and are not quoted.

Technology Stack

Python XGBoost TensorFlow / Keras scikit-learn imbalanced-learn (SMOTE) pandas Matplotlib
Interested in this work or something similar?