Linear Algebra for AI
Master the language of AI: vectors, matrices, transformations, and eigenvalues -- with Python code to ground every concept. From NumPy basics to LoRA fine-tuning and mechanistic interpretability.
About This Course
Linear algebra is the mathematical backbone of modern AI and machine learning. This course teaches you to think in vectors and matrices, with every concept grounded in worked Python.
You won't just memorize formulas -- you'll build intuition for what linear transformations actually do, visualize high-dimensional spaces, and understand why eigenvalues matter for everything from Google's PageRank to neural networks.
The course bridges theory and production: you'll see how SVD powers LoRA fine-tuning, how embeddings live in vector spaces, how attention is pure linear algebra, and how quantization trades precision for speed. Every module is a narrated lesson deck with the maths typeset, and you can ask questions against the slide you are on.
An AI tutor answers questions against the slide you are on, so help stays tied to the lesson rather than to the internet at large.
Inspired by Gilbert Strang's MIT 18.06, 3Blue1Brown's Essence of Linear Algebra, and modern AI research (LoRA, mechanistic interpretability, quantization).
Prerequisites
- Basic algebra (solving equations, working with variables)
- Familiarity with Python basics helpful but not required
- No prior linear algebra experience needed
What You Will Learn
- Understand vectors, matrices, and their geometric interpretations
- Visualize transformations with matplotlib and build geometric intuition
- Solve systems of linear equations using Gaussian elimination
- Grasp linear independence, span, and basis -- the core of vector spaces
- Compute and interpret determinants, inverses, eigenvalues, and eigenvectors
- Apply PCA for dimensionality reduction on real datasets
- Use SVD for image compression and low-rank approximation
- Understand how word embeddings and LLM embeddings work geometrically
- Know how ANN algorithms (HNSW, IVF) power vector search at scale
- Read neural network architectures as chains of matrix operations
- Understand the attention mechanism (QKV) as pure linear algebra
- Explain how LoRA compresses fine-tuning via low-rank factorization
- Grasp how mechanistic interpretability uses linear directions to decode model behavior
- Understand quantization as an affine transformation trading precision for speed
已發布產物溯源
這門課程是如何製作的
本記錄來自已發布的課程清單與課件清單，既說明產物中有據可查的資訊，也明確列出未被記錄的部分。
- 來源說明
- Inspired by Gilbert Strang's MIT 18.06, 3Blue1Brown's Essence of Linear Algebra, and modern AI research (LoRA, mechanistic interpretability, quantization).
- 已發布課件
- 16 節課中已有 16 節發布課件
- 投影片
- 16 份課件共 226 頁（每份 13–15 頁）
- 已測量的旁白時長
- 已發布課件清單未記錄旁白時長。
- 預估學習時間
- 約 16 小時學習時間，並非影片時長
- 課件產生時間
- 2026年3月5日 至 2026年3月5日
- 課程清單產生時間
- 2026年8月17日
已記錄的講述者設定
- Lena Brookshost · curious
語音 ID：English_CaptivatingStoryteller - Dr. Lena Hartmannteacher · workshop
語音 ID：Weidong2025 - Kaistudent · hands_on
語音 ID：English_CalmWoman - Avery Chenassistant · engaged
語音 ID：English_CalmWoman
逐課建置記錄
| 課程 | 已發布 | 投影片 | 產生時間 | 已記錄版本 | 已測量的旁白時長 |
|---|---|---|---|---|---|
| Vectors: The Language of Data01-vectors-the-language-of-data | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Matrices as Transformations02-matrices-as-transformations | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Systems of Linear Equations03-systems-of-equations | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Vector Spaces and Subspaces04-vector-spaces-and-subspaces | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Determinants and Inverses05-determinants-and-inverses | 是 | 13 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Linear Transformations06-linear-transformations | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Eigenvalues and Eigenvectors07-eigenvalues-and-eigenvectors | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| PCA: Dimensionality Reduction08-pca-dimensionality-reduction | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Singular Value Decomposition09-svd-and-low-rank-approximation | 是 | 15 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Embeddings: From Words to Vectors10-embeddings-from-words-to-vectors | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Vector Search at Scale11-vector-search-at-scale | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Neural Networks as Linear Algebra12-neural-networks-as-linear-algebra | 是 | 15 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| The Attention Mechanism13-the-attention-mechanism | 是 | 15 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| LoRA: Low-Rank Adaptation14-lora-and-low-rank-adaptation | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Mechanistic Interpretability15-mechanistic-interpretability | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
| Quantization: Precision vs. Speed16-quantization-precision-vs-speed | 是 | 14 | 2026年3月5日 | 課程 2026-03-05 · 課件架構 v3 | 未記錄 |
Your Learning Path
Each module builds on the last. Take your time—the AI tutor is with you at every step.
Vectors: The Language of Data — From arrows to arrays -- how AI represents everything as vectors
Vectors are the atoms of linear algebra and the native data format of AI. This module builds your intuition from geometric arrows to NumPy arrays. You'll learn vector addition, scalar multiplication, and the dot product -- then discover that dot products measure similarity, the idea behind cosine similarity in recommendation systems and LLM embeddings. Visualize everything with matplotlib so the geometry clicks before the formulas.
Matrices as Transformations — Every matrix is a function that reshapes space
Matrices aren't just grids of numbers -- they are linear transformations. This module teaches matrix addition, multiplication, and transposition, but always through the lens of geometry: rotation, scaling, shearing, and projection. You'll visualize each operation with matplotlib, watching the unit square deform in real time. This geometric framing is essential because every layer of a neural network is a matrix transformation.
Systems of Linear Equations — Gaussian elimination and the art of solving Ax = b
Most of applied math reduces to solving Ax = b. This module teaches you Gaussian elimination and row reduction -- the algorithmic backbone of linear algebra. You'll see how systems of equations correspond to intersecting hyperplanes, understand when solutions exist (and when they don't), and implement row reduction in Python.
Vector Spaces and Subspaces — Linear independence, span, basis, and dimension
This module introduces the abstract structure that unifies all of linear algebra: vector spaces. You'll learn what it means for vectors to be linearly independent, what 'span' really means, and why choosing the right basis simplifies everything. These concepts are the vocabulary you need to understand PCA, SVD, and embedding spaces.
Determinants and Inverses — The scaling factor of transformations and when you can undo them
The determinant tells you how a matrix transformation scales area (or volume). If it's zero, the transformation crushes space into a lower dimension -- and the matrix has no inverse. This module builds geometric intuition for determinants, then connects to matrix invertibility. You'll compute determinants and inverses by hand and with NumPy.
Linear Transformations — Kernel, image, rank-nullity, and change of basis
Now that you know matrices are transformations, this module digs deeper: what gets sent to zero (the kernel), what the transformation can produce (the image), and the fundamental rank-nullity theorem that connects them. You'll also learn change of basis -- the technique behind diagonalization, PCA, and every 'feature extraction' pipeline in ML.
Eigenvalues and Eigenvectors — The directions that survive a transformation unchanged
Eigenvectors are the directions a matrix only stretches (never rotates). Eigenvalues tell you how much. This module builds geometric intuition first -- watching vectors get transformed and identifying the special ones that stay on their line -- then covers the characteristic polynomial and diagonalization. The payoff: eigenvalues are the key to PCA, Google's PageRank, and stability analysis.
PCA: Dimensionality Reduction — Finding the directions of maximum variance in your data
Principal Component Analysis is eigenvalues applied to data. You compute the covariance matrix of your dataset, find its eigenvectors (the principal components), and project onto the top-k directions of maximum variance. This module is the bridge from abstract eigenvalue theory to practical ML: you'll reduce a real dataset from high dimensions to 2D and visualize the clusters that emerge.
Singular Value Decomposition — The Swiss army knife of matrix decompositions
SVD decomposes any matrix (not just square ones) into three factors: U * Sigma * V^T. The singular values in Sigma tell you how much 'information' each component carries. By keeping only the top-k singular values, you get the best rank-k approximation -- this is the mathematical foundation of image compression, latent semantic analysis, and (crucially) LoRA fine-tuning. This module builds from the geometry of SVD to hands-on applications.
Embeddings: From Words to Vectors — How AI maps discrete objects into continuous vector spaces
An embedding is a learned linear map from a discrete set (words, users, products) into a continuous vector space. This module covers the geometry of embeddings: why king - man + woman = queen works, how sentence embeddings capture semantic meaning, and what it means for LLM token embeddings to live in a 12,288-dimensional space. You'll also confront the curse of dimensionality -- why intuition breaks in high dimensions.
Vector Search at Scale — ANN algorithms, HNSW, and the rise of vector databases
Once you have millions of embeddings, brute-force cosine similarity is too slow. This module covers approximate nearest neighbor (ANN) algorithms that trade a tiny bit of accuracy for massive speedups. You'll learn locality-sensitive hashing, IVF (inverted file index), HNSW (hierarchical navigable small world graphs), and product quantization. Then see how vector databases (pgvector, Qdrant, Pinecone) wrap these algorithms into production infrastructure for RAG and recommendation systems.
Neural Networks as Linear Algebra — Dense layers, forward passes, and tensors beyond 2D
A dense neural network layer is just y = Wx + b -- a matrix multiplication plus a bias vector, followed by a nonlinear activation. This module strips away the deep learning mystique and shows you the linear algebra at the core. You'll manually implement a forward pass using only NumPy, then extend to tensors (3D+ arrays) that represent batches of images and sequences. Finally, you'll see that PyTorch's torch.Tensor is just NumPy with autograd.
The Attention Mechanism — How transformers use matrix operations to focus on what matters
Attention is the innovation behind transformers, and it's pure linear algebra. This module derives scaled dot-product attention step by step: project inputs into Query, Key, and Value matrices, compute attention scores via QK^T/sqrt(d), apply softmax, then multiply by V. You'll see that multi-head attention is just running several smaller attention operations in parallel -- a block-diagonal matrix structure. No black boxes.
LoRA: Low-Rank Adaptation — Fine-tuning billion-parameter models with tiny matrices
LoRA is the most practical application of matrix decomposition in modern AI. Instead of updating a giant weight matrix W (d x k parameters), you freeze W and train two small matrices B (d x r) and A (r x k) where r << d, so the update is Delta-W = B*A. This module connects SVD theory to practice: you'll understand why weight updates tend to be low-rank, implement a LoRA layer from scratch, and see how this enables fine-tuning LLMs on consumer hardware.
Mechanistic Interpretability — Using linear algebra to reverse-engineer what neural networks learn
Mechanistic interpretability is the 'forensics' of AI: researchers use linear algebra to decode what individual neurons and layers actually represent. The key insight is the Linear Representation Hypothesis -- high-level concepts like 'truthfulness,' 'sentiment,' or 'programming language' are encoded as linear directions (vectors) in activation space. You can find these directions, measure them, and even add or subtract them to steer model behavior. This module covers probing classifiers, activation steering, and concept vectors.
Quantization: Precision vs. Speed — How shrinking numbers from 32 bits to 4 bits keeps AI fast and cheap
As models grow to hundreds of billions of parameters, storing every weight in 32-bit float becomes impractical. Quantization maps continuous values to a smaller discrete set using affine transformations: Q = round(W/S + Z). This module covers the linear algebra of quantization: why it's an affine transformation, how it distorts the vector space, why 'outlier features' break naive approaches, and how techniques like GPTQ, AWQ, and SmoothQuant handle this. You'll implement basic quantization and measure the accuracy/speed tradeoff.