Lecture 04: Elimination

The idea in one sentence

Elimination turns a system of equations into a staircase you can read from the bottom up, and every step of it is a matrix you can write down.

Two lines, one crossing point

Every numerical library has a function called solve. Underneath it, on matrices with millions of entries, the same procedure runs that you would use by hand on two equations. So start with two equations.

x - 2y = 1, \qquad 3x + 2y = 11.

Each equation is a line in the plane. A solution is a point sitting on both lines, so solving means finding where they cross. You could plot them and squint. Elimination does something better. It moves one of the lines without moving the crossing point.

Subtract 3 times the first equation from the second:

(3x + 2y) - 3(x - 2y) = 11 - 3 \cdot 1 \qquad \Longrightarrow \qquad 8y = 8.

The $x$ is gone. The new second equation says $y = 1$ on its own, and the first equation then hands you $x = 1 + 2 \cdot 1 = 3$ .

Why is the crossing point safe? A point that satisfies both original equations satisfies any combination of them, so it survives the subtraction. And nothing new sneaks in, because adding 3 times the first equation back recovers the original pair. The two systems have exactly the same solutions. What changed is the picture.

Fig. 1

Orange is the first equation and green is the second, in this figure and the next. Elimination flattens the green line and leaves the blue crossing point exactly where it was. A flat line is a solved variable: you can read $y = 1$ off it without doing any more work.

The number 3 has a name. It is the multiplier, the count of copies you subtract. It came from dividing the entry you wanted to kill by the entry above it, $3 / 1$ . That entry above, the leading 1 of the first equation, is the pivot. Every step of elimination divides by a pivot to get a multiplier, then subtracts. Nothing else ever happens. The one thing that can go wrong is a pivot that comes out zero, and that is where the whole story lives.

Three endings

Two lines in a plane can do three things, and a square system of any size can do the same three. They cross once. They never meet. They lie on top of each other.

Fig. 2

The three endings of a square system, with the same orange and green equations throughout and the solutions in blue. Elimination reaches each ending without any drawing: the last row tells you which one you are in.

Elimination finds out which case you are in, in any number of dimensions, with no picture at all. Take the parallel pair $x - 2y = 1$ and $3x - 6y = 11$ . Subtract 3 times the first from the second and you get $0 = 8$ . Nothing satisfies that, so there is no solution. Now take $x - 2y = 1$ and $3x - 6y = 3$ . The same subtraction gives $0 = 0$ , a true statement carrying no information. One equation was doing the work of two, and every point on the single line solves the system.

The warning sign is the same in both failures: a pivot position that comes out zero and cannot be repaired. That is elimination telling you the matrix is singular, the word Lecture 2 attached to the ring road machine that could not be undone. The right-hand side decides which of the two failures you get.

The staircase

Real systems have more than two unknowns, and the picture stops helping. Here is the example this chapter and Lecture 6 both run on:

\begin{aligned} 2x + 4y - 2z &= 2 \\ 4x + 9y - 3z &= 8 \\ -2x - 3y + 7z &= 10 \end{aligned} \qquad A = \begin{bmatrix} 2 & 4 & -2 \\ 4 & 9 & -3 \\ -2 & -3 & 7 \end{bmatrix}, \quad b = \begin{bmatrix} 2 \\ 8 \\ 10 \end{bmatrix}.

Work one column at a time, top down. The first pivot is the 2 in the corner. Below it sit 4 and $-2$ , so the multipliers are $4/2 = 2$ and $-2/2 = -1$ :

\begin{bmatrix} 2 & 4 & -2 \\ 4 & 9 & -3 \\ -2 & -3 & 7 \end{bmatrix} \;\longrightarrow\; \begin{bmatrix} 2 & 4 & -2 \\ 0 & 1 & 1 \\ 0 & 1 & 5 \end{bmatrix}.

Column 1 is clear. Now ignore row 1 forever and repeat on what is left. The second pivot is the 1 in the middle, the entry below it is also 1, so the multiplier is 1 and row 3 loses a copy of row 2:

U = \begin{bmatrix} 2 & 4 & -2 \\ 0 & 1 & 1 \\ 0 & 0 & 4 \end{bmatrix}, \qquad c = \begin{bmatrix} 2 \\ 4 \\ 8 \end{bmatrix}.

The right-hand side rode along: $8 - 2 \cdot 2 = 4$ , then $10 + 2 = 12$ , then $12 - 4 = 8$ . The letter $U$ is for upper triangular. Its diagonal holds the pivots 2, 1 and 4. None of them is zero, so this system has exactly one solution and elimination already knows it.

Reading the answer from the bottom

$Ux = c$ takes almost no work, because the last equation has only one unknown left in it.

4z = 8 \;\Rightarrow\; z = 2, \qquad y + z = 4 \;\Rightarrow\; y = 2, \qquad 2x + 4 \cdot 2 - 2 \cdot 2 = 2 \;\Rightarrow\; x = -1.

Fig. 3

The staircase, with the pivots in yellow. Each row has one more zero than the row above it, so the bottom row solves itself and every row above it inherits an answer. Read upward.

So $x = (-1, 2, 2)$ . Check it against the original equations: $2(-1) + 4(2) - 2(2) = 2$ , and $4(-1) + 9(2) - 3(2) = 8$ , and $-2(-1) - 3(2) + 7(2) = 10$ .

The two halves of the work cost very different amounts. Elimination on an $n$ by $n$ matrix takes about $n^3/3$ multiply-and-subtract operations; back substitution takes about $n^2/2$ . At $n = 1000$ that is roughly 300 million against 500 thousand. Nearly all the money goes into making the staircase. That asymmetry is the reason Lecture 6 saves the staircase as a factorization instead of rebuilding it for every new right-hand side.

Fig. 4

Where the cost comes from. Yellow is the work still to do. Once a column is cleared, elimination never looks at it again, so the yellow square shrinks by one on each side every step. Adding up those squares is what makes the total about $n^3/3$ .

One object instead of two

Notice that $b$ never influenced a decision. The multipliers came from $A$ alone; $b$ just followed along getting the same treatment. So stop writing them apart. Glue $b$ on as a fourth column and call the result the augmented matrix:

[\,A \;|\; b\,] = \left[\begin{array}{ccc|c} 2 & 4 & -2 & 2 \\ 4 & 9 & -3 & 8 \\ -2 & -3 & 7 & 10 \end{array}\right] \;\longrightarrow\; \left[\begin{array}{ccc|c} 2 & 4 & -2 & 2 \\ 0 & 1 & 1 & 4 \\ 0 & 0 & 4 & 8 \end{array}\right] = [\,U \;|\; c\,].

One matrix, one pass, and the vertical bar is only there to remind you which column is not a coefficient. Every hand computation from here on runs on the augmented matrix.

Fig. 5

One object, one pass. The yellow column is $b$ , riding along and taking the same treatment as the coefficients. Nothing about the step changed; only the bookkeeping did.

Every step is a matrix

Here is the move that turns a hand procedure into algebra.

Start from the identity matrix $I$ , the machine that leaves every vector alone. Its rows are the purest possible recipes: row 2 of $I$ is $(0, 1, 0)$ , which says “take one copy of row 2 and nothing else”. Change one entry of one row and the recipe changes with it. Put $-2$ in position $(2,1)$ :

E_{21} = \begin{bmatrix} 1 & 0 & 0 \\ -2 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}.

Row 2 of $E_{21}$ now reads $(-2, 1, 0)$ : minus two copies of row 1, plus one copy of row 2. That is the first elimination step, written as a matrix.

Fig. 6

An elimination matrix is the identity with one entry changed. That entry, in yellow, is the negative of the multiplier, and it sits in the position it is about to zero out.

Multiply and check. Row $i$ of a product $EA$ is row $i$ of $E$ applied to the rows of $A$ , so $E_{21}A$ keeps rows 1 and 3 of $A$ and replaces row 2 by row 2 minus twice row 1. That is exactly what your pencil did. The same $E_{21}$ acting on $b$ does the same bookkeeping to the right-hand side, which is why the augmented matrix works at all.

Two more entries finish the job. $E_{31}$ carries $+1$ in position $(3,1)$ , because the multiplier there was $-1$ . $E_{32}$ carries $-1$ in position $(3,2)$ :

E_{32} E_{31} E_{21} A = U.

Fig. 7

Three matrices, three zeros, one staircase. Each blue arrow is a single left multiplication, and the yellow cell marks the entry it has just wiped out.

Because matrix multiplication can be regrouped, that long product collapses into a single matrix $E = E_{32}E_{31}E_{21}$ with $EA = U$ . Multiply it out and something odd shows up:

E = \begin{bmatrix} 1 & 0 & 0 \\ -2 & 1 & 0 \\ 3 & -1 & 1 \end{bmatrix}, \qquad E^{-1} = \begin{bmatrix} 1 & 0 & 0 \\ 2 & 1 & 0 \\ -1 & 1 & 1 \end{bmatrix}.

The multipliers were 2, then $-1$ , then 1. In $E$ the bottom-left entry is 3, which is not any multiplier: the steps interfered with each other. In $E^{-1}$ the three multipliers sit in their own slots, unmixed. That is not luck, and cashing it in is the whole of $A = LU$ in Lecture 6.

When the pivot is zero

Elimination as described breaks the moment a pivot position holds a zero, because the multiplier would ask you to divide by it. Sometimes that is fatal. Often it is not. Take

B = \begin{bmatrix} 1 & 2 & 1 \\ 2 & 4 & 3 \\ 1 & 1 & 2 \end{bmatrix}, \qquad b = \begin{bmatrix} 4 \\ 9 \\ 4 \end{bmatrix}.

Clear column 1 the usual way. Row 2 minus twice row 1, row 3 minus row 1:

\left[\begin{array}{ccc|c} 1 & 2 & 1 & 4 \\ 0 & 0 & 1 & 1 \\ 0 & -1 & 1 & 0 \end{array}\right].

The second pivot position holds a 0, and no multiple of row 2 will ever change that, since row 2 starts with two zeros of its own. But look one row down. There is a perfectly good $-1$ waiting in that column. The rows of a system are in no particular order, so exchange them.

Fig. 8

A zero in the pivot position is not always fatal. Pink marks the dead pivot, yellow the working one that replaces it. If a nonzero sits below in the same column, trade the two rows and carry on.

The exchange is a matrix too. Take the identity and swap its rows 2 and 3:

P_{23} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{bmatrix}.

Row 1 of $P_{23}$ still says “take row 1”. Row 2 now says “take row 3”, and row 3 says “take row 2”. So $P_{23}M$ is $M$ with those two rows traded, for any $M$ with three rows. A permutation matrix in general is the identity with its rows written in some other order. There are $n!$ of them at size $n$ , and they cost nothing to apply. Undoing a single swap is doing it again, so $P_{23}P_{23} = I$ .

After the exchange the staircase finishes on its own, with pivots 1, $-1$ and 1:

\left[\begin{array}{ccc|c} 1 & 2 & 1 & 4 \\ 0 & -1 & 1 & 0 \\ 0 & 0 & 1 & 1 \end{array}\right] \qquad \Longrightarrow \qquad z = 1, \quad y = 1, \quad x = 1.

Try it

Press the button and watch a zero appear. The picture on the left is the matrix responsible for that step. The state on the right is computed live, by actually multiplying the two. The second preset stalls on a dead pivot and needs the exchange.

Try it. One step at a time, one zero at a time

step 0 of 3 pivots so far: none

Yellow on the left is the one entry doing the work. Yellow on the right is what it changed. Watch the second preset stall on a zero pivot.

Elimination in code

The first block builds the elimination matrices out of the identity and checks the chain. The frameworks disagree about one thing worth knowing: NumPy and PyTorch let you assign into an array, while JAX and TensorFlow refuse and hand you a modified copy instead.

python

import numpy as np

A = np.array([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = np.array([2., 8., 10.])

E21 = np.eye(3); E21[1, 0] = -2.0      # row 2 minus 2 times row 1
E31 = np.eye(3); E31[2, 0] = 1.0       # row 3 plus row 1
E32 = np.eye(3); E32[2, 1] = -1.0      # row 3 minus row 2

print(E21 @ A)              # [[2 4 -2] [0 1 1] [-2 -3 7]]  one zero
print(E31 @ E21 @ A)        # [[2 4 -2] [0 1 1] [ 0  1 5]]  column 1 clear
print(E32 @ E31 @ E21 @ A)  # [[2 4 -2] [0 1 1] [ 0  0 4]]  the staircase U
print(E32 @ E31 @ E21 @ b)  # [2. 4. 8.]                    and c

E = E32 @ E31 @ E21
print(E)                    # [[1 0 0] [-2 1 0] [ 3 -1 1]]  the 3 is a mix
print(np.linalg.inv(E))     # [[1 0 0] [ 2 1 0] [-1  1 1]]  multipliers, unmixed

python

import torch

A = torch.tensor([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = torch.tensor([2., 8., 10.])

E21 = torch.eye(3); E21[1, 0] = -2.0   # row 2 minus 2 times row 1
E31 = torch.eye(3); E31[2, 0] = 1.0    # row 3 plus row 1
E32 = torch.eye(3); E32[2, 1] = -1.0   # row 3 minus row 2

print(E21 @ A)              # [[2 4 -2] [0 1 1] [-2 -3 7]]  one zero
print(E31 @ E21 @ A)        # [[2 4 -2] [0 1 1] [ 0  1 5]]  column 1 clear
print(E32 @ E31 @ E21 @ A)  # [[2 4 -2] [0 1 1] [ 0  0 4]]  the staircase U
print(E32 @ E31 @ E21 @ b)  # tensor([2., 4., 8.])          and c

E = E32 @ E31 @ E21
print(E)                    # [[1 0 0] [-2 1 0] [ 3 -1 1]]  the 3 is a mix
print(torch.linalg.inv(E))  # [[1 0 0] [ 2 1 0] [-1  1 1]]  multipliers, unmixed

python

import jax
jax.config.update("jax_enable_x64", True)   # jax is single precision unless you ask
import jax.numpy as jnp

A = jnp.array([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = jnp.array([2., 8., 10.])

E21 = jnp.eye(3).at[1, 0].set(-2.0)    # jax arrays never change in place:
E31 = jnp.eye(3).at[2, 0].set(1.0)     # .at[...].set(...) returns a new one
E32 = jnp.eye(3).at[2, 1].set(-1.0)

print(E21 @ A)              # [[2 4 -2] [0 1 1] [-2 -3 7]]  one zero
print(E31 @ E21 @ A)        # [[2 4 -2] [0 1 1] [ 0  1 5]]  column 1 clear
print(E32 @ E31 @ E21 @ A)  # [[2 4 -2] [0 1 1] [ 0  0 4]]  the staircase U
print(E32 @ E31 @ E21 @ b)  # [2. 4. 8.]                    and c

E = E32 @ E31 @ E21
print(E)                    # [[1 0 0] [-2 1 0] [ 3 -1 1]]  the 3 is a mix
print(jnp.linalg.inv(E))    # [[1 0 0] [ 2 1 0] [-1  1 1]]  multipliers, unmixed

python

import tensorflow as tf

A = tf.constant([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = tf.constant([[2.], [8.], [10.]])

def edit(i, j, v):                     # tensorflow tensors are immutable too
    return tf.tensor_scatter_nd_update(tf.eye(3), [[i, j]], [v])

E21, E31, E32 = edit(1, 0, -2.0), edit(2, 0, 1.0), edit(2, 1, -1.0)

print(E21 @ A)              # [[2 4 -2] [0 1 1] [-2 -3 7]]  one zero
print(E31 @ E21 @ A)        # [[2 4 -2] [0 1 1] [ 0  1 5]]  column 1 clear
print(E32 @ E31 @ E21 @ A)  # [[2 4 -2] [0 1 1] [ 0  0 4]]  the staircase U
print(E32 @ E31 @ E21 @ b)  # [[2.] [4.] [8.]]              and c

E = E32 @ E31 @ E21
print(E)                    # [[1 0 0] [-2 1 0] [ 3 -1 1]]  the 3 is a mix
print(tf.linalg.inv(E))     # [[1 0 0] [ 2 1 0] [-1  1 1]]  multipliers, unmixed

The second block writes the algorithm out in full, refuses to divide by a zero pivot, and then compares itself against the library. Same numbers in every framework, since the arithmetic is identical.

python

import numpy as np

def eliminate(M, rhs):
    """Forward elimination, no row exchanges. Returns U, c and the pivots."""
    M, rhs, n = M.copy(), rhs.copy(), len(rhs)
    for k in range(n):
        if M[k, k] == 0.0:
            raise ZeroDivisionError(f"zero pivot in column {k + 1}")
        for i in range(k + 1, n):
            m = M[i, k] / M[k, k]
            M[i, k:] -= m * M[k, k:]
            rhs[i] -= m * rhs[k]
    return M, rhs, np.diag(M).copy()

def back_substitute(U, c):
    x = np.zeros(len(c))
    for i in range(len(c) - 1, -1, -1):
        x[i] = (c[i] - U[i, i + 1:] @ x[i + 1:]) / U[i, i]
    return x

A = np.array([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = np.array([2., 8., 10.])
U, c, pivots = eliminate(A, b)
print(pivots)                    # [2. 1. 4.]
print(back_substitute(U, c))     # [-1.  2.  2.]
print(np.linalg.solve(A, b))     # [-1.  2.  2.]   the library agrees
print(np.prod(pivots))           # 8.0   the pivots multiply to det A

B = np.array([[1., 2., 1.], [2., 4., 3.], [1., 1., 2.]])
d = np.array([4., 9., 4.])
try:
    eliminate(B, d)
except ZeroDivisionError as e:
    print(e)                     # zero pivot in column 2
print(np.linalg.solve(B, d))     # [1. 1. 1.]   solve exchanged rows and finished

python

import torch

def eliminate(M, rhs):
    """Forward elimination, no row exchanges. Returns U, c and the pivots."""
    M, rhs, n = M.clone(), rhs.clone(), len(rhs)
    for k in range(n):
        if M[k, k] == 0.0:
            raise ZeroDivisionError(f"zero pivot in column {k + 1}")
        for i in range(k + 1, n):
            m = M[i, k] / M[k, k]
            M[i, k:] -= m * M[k, k:]
            rhs[i] -= m * rhs[k]
    return M, rhs, torch.diag(M).clone()

def back_substitute(U, c):
    x = torch.zeros(len(c))
    for i in range(len(c) - 1, -1, -1):
        x[i] = (c[i] - U[i, i + 1:] @ x[i + 1:]) / U[i, i]
    return x

A = torch.tensor([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = torch.tensor([2., 8., 10.])
U, c, pivots = eliminate(A, b)
print(pivots)                    # tensor([2., 1., 4.])
print(back_substitute(U, c))     # tensor([-1.,  2.,  2.])
print(torch.linalg.solve(A, b))  # tensor([-1.,  2.,  2.])   the library agrees
print(pivots.prod())             # tensor(8.)   the pivots multiply to det A

B = torch.tensor([[1., 2., 1.], [2., 4., 3.], [1., 1., 2.]])
d = torch.tensor([4., 9., 4.])
try:
    eliminate(B, d)
except ZeroDivisionError as e:
    print(e)                     # zero pivot in column 2
print(torch.linalg.solve(B, d))  # tensor([1., 1., 1.])   rows were exchanged

python

import jax
jax.config.update("jax_enable_x64", True)   # jax is single precision unless you ask
import jax.numpy as jnp

def eliminate(M, rhs):
    """Forward elimination, no row exchanges. Returns U, c and the pivots."""
    n = len(rhs)
    for k in range(n):
        if M[k, k] == 0.0:
            raise ZeroDivisionError(f"zero pivot in column {k + 1}")
        for i in range(k + 1, n):
            m = M[i, k] / M[k, k]
            M = M.at[i, k:].add(-m * M[k, k:])
            rhs = rhs.at[i].add(-m * rhs[k])
    return M, rhs, jnp.diag(M)

def back_substitute(U, c):
    x = jnp.zeros(len(c))
    for i in range(len(c) - 1, -1, -1):
        x = x.at[i].set((c[i] - U[i, i + 1:] @ x[i + 1:]) / U[i, i])
    return x

A = jnp.array([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = jnp.array([2., 8., 10.])
U, c, pivots = eliminate(A, b)
print(pivots)                    # [2. 1. 4.]
print(back_substitute(U, c))     # [-1.  2.  2.]
print(jnp.linalg.solve(A, b))    # [-1.  2.  2.]   the library agrees
print(jnp.prod(pivots))          # 8.0   the pivots multiply to det A

B = jnp.array([[1., 2., 1.], [2., 4., 3.], [1., 1., 2.]])
d = jnp.array([4., 9., 4.])
try:
    eliminate(B, d)
except ZeroDivisionError as e:
    print(e)                     # zero pivot in column 2
print(jnp.linalg.solve(B, d))    # [1. 1. 1.]   rows were exchanged

python

import numpy as np
import tensorflow as tf

def eliminate(M, rhs):
    """Forward elimination, no row exchanges. Returns U, c and the pivots."""
    M, rhs, n = np.array(M), np.array(rhs), len(rhs)   # loop in numpy, then hand back
    for k in range(n):
        if M[k, k] == 0.0:
            raise ZeroDivisionError(f"zero pivot in column {k + 1}")
        for i in range(k + 1, n):
            m = M[i, k] / M[k, k]
            M[i, k:] -= m * M[k, k:]
            rhs[i] -= m * rhs[k]
    return tf.constant(M), tf.constant(rhs), tf.constant(np.diag(M).copy())

A = tf.constant([[2., 4., -2.], [4., 9., -3.], [-2., -3., 7.]])
b = tf.constant([[2.], [8.], [10.]])
U, c, pivots = eliminate(A, tf.reshape(b, [-1]))
print(pivots)                       # [2. 1. 4.]
print(tf.linalg.triangular_solve(U, tf.reshape(c, [-1, 1]), lower=False))
                                    # [[-1.] [2.] [2.]]   back substitution, built in
print(tf.linalg.solve(A, b))        # [[-1.] [2.] [2.]]   the library agrees
print(tf.reduce_prod(pivots))       # 8.0   the pivots multiply to det A

B = tf.constant([[1., 2., 1.], [2., 4., 3.], [1., 1., 2.]])
d = tf.constant([[4.], [9.], [4.]])
try:
    eliminate(B, tf.reshape(d, [-1]))
except ZeroDivisionError as e:
    print(e)                        # zero pivot in column 2
print(tf.linalg.solve(B, d))        # [[1.] [1.] [1.]]   rows were exchanged

One more thing the library does that the hand version does not. A tiny pivot is almost as dangerous as a zero one, because dividing by it inflates every rounding error in the row. So real solvers exchange rows at every column, moving the largest available entry into the pivot position. It is called partial pivoting, and here is why it exists:

python

eps = 1e-16
T = np.array([[eps, 1.], [1., 1.]])
t = np.array([1., 2.])          # the honest answer is x = 1, y = 1
U, c, _ = eliminate(T, t)
print(back_substitute(U, c))  # [2.22044605 1.  ]  the first coordinate should be 1
print(np.linalg.solve(T, t))  # [1. 1.]            solve traded the rows first

Nothing about that matrix is close to singular. Its determinant is

10^{-16} - 1

, about

-1

: for a matrix whose entries are all around size one, a determinant of size one is as healthy as they come. The only mistake was the order of the rows.

Where this is going

You now have the algorithm that sits under every call to solve, and one surprising fact about it: the whole procedure is a product of matrices standing to the left of $A$ . That fact was used here on trust. Row $i$ of $EA$ mixes the rows of $A$ , and a long product regroups without changing anything. Both are claims about what $AB$ means, and there are four different useful answers to that question. The next chapter computes all four on the same small pair of matrices, and then uses them to say when a matrix can be undone at all.

Check yourself

Solve $x + y = 3$ and $2x + 4y = 8$ by elimination. What is the multiplier, what is the second pivot, and where does the crossing point sit?
Run elimination on $[[1, 3], [2, 6]]$ with right-hand side $(5, 10)$ , then with $(5, 11)$ . Same matrix, two different endings. Explain both from the last row.
Write the $3 \times 3$ matrix that adds 4 times row 1 to row 3 and changes nothing else. Now write its inverse without computing anything.
In the chapter, $E_{32}E_{31}E_{21}$ came out with a 3 in the bottom-left corner even though no multiplier was 3. Multiply the three matrices by hand and find where the 3 is born.
The matrix $[[0, 1], [1, 0]]$ needs a row exchange at the very first step. Which permutation matrix fixes it, and what are the pivots afterwards?
Redraw Figure 3 from memory, pivots and labels included. The drawing is the test.