Determinants and Inverses

From subspaces to "can we undo this transform?"

Dr. Lena Hartmann: Every time an AI system tries to recover a hidden vector, like solving a linear system inside least squares or whitening features, it is betting that a matrix transform can be undone.

Dr. Lena Hartmann: Today we are going to discover determinants and inverses: one number that tells you how a matrix changes space, and a matrix that reverses the change when possible.

Dr. Lena Hartmann: We will start from the subspace idea from last time, then look at a geometric picture, then verify everything in NumPy.

Kai: So determinants tell me if we lost information, and inverses are how we get it back?

Dr. Lena Hartmann: Exactly. First, remember the L04 picture: a matrix maps all vectors into its column space, which can be smaller than the full space.

Dr. Lena Hartmann: If the output space collapses into a lower-dimensional subspace, then different inputs can land on the same output, and undoing becomes impossible.

Dr. Lena Hartmann: Our driving question is: can we detect that collapse quickly? The determinant is a fast signal for that in square matrices.

Why $\det(A)$ and $A^{-1}$ matter in AI

Dr. Lena Hartmann: In production machine learning, you rarely invert a matrix just for fun. A quick definition to anchor us: if the determinant is zero, the matrix is singular, meaning it is not invertible.

Kai: Is this why people warn against using an inverse directly in code?

Dr. Lena Hartmann: Yes. When the determinant is close to zero, the matrix is almost collapsing space, and small input changes can cause big output changes after you try to undo it. But be careful: det close to zero is only a rough warning, while the condition number of A is what really predicts numerical sensitivity, and scaling a matrix can change det a lot without changing conditioning nearly as much.

Dr. Lena Hartmann: We will come back to where these determinant terms show up in probability models later, once we have a solid definition and a few properties.

Dr. Lena Hartmann: And the everyday case is solving A x equals b. We will use determinants to predict when the solution is unique, and then build the inverse when it exists.

$\det(A)$ as area scaling in $2$D

Dr. Lena Hartmann: Before we talk about inverses, let me show you what the determinant feels like geometrically: it is about how a matrix scales area in two dimensions.

Dr. Lena Hartmann: This equation gives the determinant of a two by two matrix. It is the number we will interpret as signed area scaling.

Kai: So if I apply the matrix to the unit square, the area change is the absolute value of that number?

Dr. Lena Hartmann: Exactly. The magnitude tells you how much area is stretched or squashed. If it is zero, the square gets flattened into a line and information is lost.

Dr. Lena Hartmann: The sign tells you whether the transform preserves orientation or flips it, like a mirror reflection.

Let me show you in code: area scaling from a matrix

Dr. Lena Hartmann: A two by two matrix can stretch, shear, and possibly flip the plane, and the determinant tells you the area scaling and the orientation change.

Dr. Lena Hartmann: Now we will run a short numpy demo: compute the determinant of A, and send the unit square corners through A. Before you look at the printed value, predict it: will the determinant be positive or negative, and bigger or smaller than 1?

Kai: My prediction is that it will be positive and greater than 1, because the first column looks like it stretches x by about 2, and the second column also has a decent y component, so the shape should get bigger overall.

Dr. Lena Hartmann: Good prediction. When we compute it, we get det(A) equals 2.5. That matches positive and greater than 1: the transform expands area to about two and a half times the original, and because the determinant is positive, it does not flip orientation.

Determinant $\neq 0$ means an inverse exists

Dr. Lena Hartmann: Now we formalize the payoff: a square matrix has an inverse exactly when it does not squash space down to a lower dimension.

Dr. Lena Hartmann: This equation shows the explicit inverse for a two by two matrix: you swap and negate entries in a pattern, then divide by the determinant.

Kai: So the determinant is literally in the denominator. If it is zero, the inverse would need division by zero, so it cannot exist.

Dr. Lena Hartmann: Exactly, and geometrically that corresponds to a collapse into a smaller subspace, which we learned to spot in L04.

What an inverse does: "undo" the linear transform

Dr. Lena Hartmann: An inverse is not just a formula: it is the promise that you can go forward with A and then come back exactly.

Dr. Lena Hartmann: This equation is the undo statement: A inverse after A returns the original vector, and multiplying gives the identity matrix.

Kai: And if A is not invertible, then different x values can map to the same b, so there is no single answer.

Dr. Lena Hartmann: Right. When A is a square matrix and is invertible, then for every b in R to the n there is exactly one x with A x equals b. If A is not invertible, depending on b you can get no solutions or infinitely many solutions.

Let me show you in code: computing $A^{-1}$ (and what breaks)

Dr. Lena Hartmann: Let me show you the practical workflow: compute an inverse, then immediately do a quick sanity check by multiplying back together.

Dr. Lena Hartmann: In this code, numpy linalg inv gives A inverse, and A times A inverse should come out extremely close to the do nothing matrix, up to tiny floating point noise.

Kai: Before we run the second part, I want to predict: since one row is a multiple of the other, the determinant should be zero, and trying to invert it should fail.

Dr. Lena Hartmann: Exactly—pause there and make your own prediction: will det of B print as zero or nonzero, and will numpy invert it or raise an error? Now we reveal it: the printed determinant is 0.0, and the try except catches a LinAlgError when we call inv on B. If your guess differed, that mismatch is useful: it means you just found a spot to connect the geometry, rank, and what the library can and cannot do.

Worked example: solve $A\mathbf{x}=\mathbf{b}$ without explicitly forming $A^{-1}$

Dr. Lena Hartmann: Here is the core computational task: given A and b, find x. This is what shows up inside regression and many optimization routines.

Dr. Lena Hartmann: This code uses numpy linalg solve, which is typically more stable than forming the inverse and multiplying.

Kai: So even if the inverse exists, solve is the safer tool because it avoids extra numerical error?

Dr. Lena Hartmann: Exactly. And the residual norm is your quick verification: A times x should land back on b.

AI connection: determinants show up as "density correction"

Dr. Lena Hartmann: Now the modern AI cameo: in normalizing flows, a network is built from invertible layers, and determinants tell you how volumes change.

Dr. Lena Hartmann: This equation is the change of variables idea in log space: with x equal to f inverse of y, you take the base log density at x and subtract the log absolute determinant of the Jacobian of f at x. This requires f to be invertible and differentiable, with a Jacobian determinant that is not zero everywhere.

Kai: So that is literally the same geometry as the unit square example, but in high dimensions and inside a neural network.

Dr. Lena Hartmann: Exactly. In a valid flow the determinant cannot be zero, but if it gets very close to zero, the layer is almost collapsing volume, and that makes the inverse and the likelihood numerically fragile.

Checkpoint: what you should now be able to predict

Dr. Lena Hartmann: Let us lock in the mental model: determinant is geometry, and inverse is the undo operation that only exists when geometry is not collapsed.

Dr. Lena Hartmann: First: determinant is the signed scaling factor for volume, which is area in two dimensions and volume in three dimensions.

Kai: Second: if determinant is zero, the transform is not one to one, so there is no way to recover the input uniquely.

Dr. Lena Hartmann: Third: in code, when you want x from A x equals b, go straight to a solver. That is how ML libraries stay stable and fast.

Exit ticket: decide invertible, then solve

Dr. Lena Hartmann: Time to practice with a concrete two by two matrix. You will use the determinant to decide if a unique solution exists.

Dr. Lena Hartmann: In this exit ticket, start by computing the determinant, then use it to decide whether the matrix is invertible and whether the system has a unique solution. Let the unknown vector be x equals the column vector with entries x one and x two.

Dr. Lena Hartmann: Determinant is 2 times 2 minus 1 times 3, which equals 1, so the matrix is invertible and the solution is unique. Solving gives x one equals 2 and x two equals minus 3.

Kai: Geometrically, determinant 1 means the transform preserves area, even though it still shears and rotates directions.

Thank you for watching!

Thanks for watching. Subscribe and share if you found this useful—see you next time!