Deep Learning Fundamentals

Neural Network Manual Backpropagation

A recommendation-engine neural network built and trained entirely by hand in NumPy — no TensorFlow, no PyTorch — to demonstrate genuine first-principles understanding of forward propagation, gradient computation, and gradient descent.

Type

From-Scratch Implementation

Domain

Retail / E-commerce

Methods

Manual Forward & Backward Pass

Status

Completed

MANUAL IMPLEMENTATION

2-2-1 ARCHITECTURE

FORWARD PASS → PREDICTION

LOSS GRADIENT COMPUTED

BACKPROP → WEIGHT UPDATE

The Challenge

A complete neural network, forward pass, backpropagation, and gradient descent, built and trained using nothing but NumPy, establishing first-principles understanding before scaling to a production retail recommendation engine.

Before reaching for a deep learning framework, this project set out to build and train a neural network entirely by hand, to establish genuine first-principles understanding of forward propagation, backpropagation, and gradient descent, rather than treating them as a framework's black box.

Approach

Network Definition

Built a 2-2-1 architecture by hand: two input features (customer signals), a two-neuron ReLU hidden layer, and a single Sigmoid output neuron for binary recommendation prediction, with all weights and biases explicitly initialised.

Forward Propagation

Manually implemented the forward pass, computing hidden-layer activations via ReLU and the final prediction via Sigmoid, entirely with NumPy matrix operations.

Manual Backpropagation

Derived and coded the loss gradient with respect to the output, then propagated gradients backward through the output and hidden layers by hand, computing every derivative explicitly rather than relying on autodiff.

Gradient Descent

Updated all weights and biases using the computed gradients at a learning rate of 0.02, closing the loop from prediction to error to parameter update with no framework involved.

Results

2-2-1

Network architecture: input, hidden, and output layers implemented entirely from scratch

Deep learning framework dependencies — pure NumPy forward and backward pass

ReLU + Sigmoid

Activation functions and their derivatives implemented and verified by hand

The exercise successfully traced a full training step end-to-end: a customer feature vector flows forward through ReLU and Sigmoid activations to a prediction, the loss gradient is computed against a target label, and that error is propagated backward through both layers to produce weight updates via gradient descent.

Working through the maths manually surfaces exactly why frameworks like TensorFlow and PyTorch exist, and exactly what they're automating underneath their abstractions. That distinction matters directly for architecture decisions on real projects: knowing what backpropagation is actually computing makes it much easier to reason about vanishing gradients, activation choice, and learning rate sensitivity when scaling a model up.

Business Impact

For a retail recommendation engine specifically, this foundation supports better-informed architecture decisions when scaling from a toy 2-2-1 network to a production system serving millions of customers and products, more personalised recommendations translate fairly directly into higher engagement, conversion, and customer lifetime value, and being able to reason about the mechanics rather than just calling model.fit() is a genuine differentiator when diagnosing why a production model isn't learning as expected.

Technology Stack

Python NumPy Matplotlib Manual Backpropagation Gradient Descent

Interested in this work or something similar?

Get in Touch View All Projects