Automobile Price Prediction
Feature engineering, feature selection and dimensionality reduction on a 205-vehicle dataset to predict price, comparing linear regression with Random Forest on a held-out test set.
The Challenge
On a held-out 20% test set, Random Forest explained 89% of price variance (R² = 0.894) against 72% for linear regression (R² = 0.717), using the same 25 selected features.
Predicting vehicle price from specification is a classic feature-engineering problem: 205 vehicles and 26 raw features spanning categorical, numerical and free-text fields, with brand-name typos, correlated measurements and a high-cardinality car name field. The goal was an interpretable pricing model, plus a look at whether dimensionality reduction reveals structure in the data.
Approach
Results
The gap between the two models points to non-linear relationships between specification and price that a linear model cannot capture. With all features rather than the selected 25, Random Forest reached R² = 0.903 on the same split, so feature selection cost very little accuracy while simplifying the model.
Methodology notes. The PCA and t-SNE views are exploratory only: PCA was fitted on all 190 vehicles with price among the inputs, so it is not used as a modelling input. Its first three components explain 37.8% of variance across the engineered features, not of price. With 38 vehicles in the test set, the R² figures will move with a different split; cross-validation is the next step before treating them as stable.
What It Means
Specification data predicts price well, and a non-linear model captures it better than a linear one. The features carrying most information, curb weight, engine size, fuel economy and horsepower, are the natural basis for a pricing model or for segmenting vehicles into price bands.