Neural Networks as Linear Algebra

Dense layers, forward passes, and tensors beyond 2D

A dense neural network layer is just y = Wx + b -- a matrix multiplication plus a bias vector, followed by a nonlinear activation. This module strips away the deep learning mystique and shows you the linear algebra at the core. You'll manually implement a forward pass using only NumPy, then extend to tensors (3D+ arrays) that represent batches of images and sequences. Finally, you'll see that PyTorch's torch.Tensor is just NumPy with autograd. Mini-lab: Manually code a 2-layer neural network forward pass using only NumPy matrix operations (no frameworks), then verify against PyTorch.

Estimated time: 60 minutes

Stuck on something? The AI tutor sees this lecture—just ask.