Skip to main content

MATH 345: Linear Algebra and Optimization

Sebastien Roch with Jacob Denson and Kaitlyn Phillipson

Section 2.6 Applications and computation recap

Subsection Review activities

Activity 2.6.1. A sensor that drops height.

A sensor records only horizontal position. It keeps the first two coordinates and drops height.
🔗
import numpy as np

P = np.array([
    [1, 0, 0],
    [0, 1, 0],
])

x = np.array([2, 1, 5])
z = np.array([0, 0, 1])

P @ x, P @ (x + 6*z), P @ z
Output:
🔗
(array([2, 1]), array([2, 1]), array([0, 0]))
  1. What does \(P\) do to an input vector?
    🔗
    🔗
  2. Why do \(P\mathbf{x}\) and \(P(\mathbf{x}+6\mathbf{z})\) agree?
    🔗
    🔗
  3. What does \(P\mathbf{z}=\mathbf{0}\) say about the direction \(\mathbf{z}\text{?}\)
    🔗
    🔗
  4. Why can the original input not be recovered uniquely from the output?
    🔗
    🔗
Solution.
The matrix \(P\) keeps the first two coordinates and forgets the third.
🔗
The vectors
🔗
\begin{equation*} \mathbf{x} = \begin{bmatrix} 2\\ 1\\ 5 \end{bmatrix} \qquad\text{and}\qquad \mathbf{x}+6\mathbf{z} = \begin{bmatrix} 2\\ 1\\ 11 \end{bmatrix} \end{equation*}
have different heights, but the same first two coordinates. Therefore their outputs agree.
🔗
\begin{equation*} P\mathbf{z}=\mathbf{0}, \end{equation*}
the direction
🔗
\begin{equation*} \mathbf{z} = \begin{bmatrix} 0\\ 0\\ 1 \end{bmatrix} \end{equation*}
is a forgotten direction. Inputs that differ by a multiple of \(\mathbf{z}\) have the same output.
🔗
🔗
🔗

Activity 2.6.2. Reading a null-space basis in code.

The following code asks SymPy for a basis of a null space.
🔗
import sympy as sp

A = sp.Matrix([
    [1, 2, 3],
    [2, 4, 6],
    [0, 1, 1],
])

A.nullspace()
Output:
🔗
[Matrix([
[-1],
[-1],
[ 1]])]
  1. What kind of object does nullspace() return?
    🔗
    🔗
  2. Is the displayed vector an input direction or an output direction?
    🔗
    🔗
  3. Verify from the code output that \(\mathbf{z} = \begin{bmatrix} -1\\ -1\\ 1 \end{bmatrix}\) is in \(\operatorname{null}(A)\text{.}\)
    🔗
    🔗
  4. If \(A\mathbf{x}=\mathbf{y}\text{,}\) what is \(A(\mathbf{x}+t\mathbf{z})\text{?}\)
    🔗
    🔗
Solution.
The command returns a list of basis vectors for \(\operatorname{null}(A)\text{.}\)
🔗
The vector
🔗
\begin{equation*} \mathbf{z} = \begin{bmatrix} -1\\ -1\\ 1 \end{bmatrix} \end{equation*}
is an input direction. It satisfies
🔗
\begin{equation*} A\mathbf{z} = \mathbf{0}. \end{equation*}
If \(A\mathbf{x}=\mathbf{y}\text{,}\) then
🔗
\begin{equation*} A(\mathbf{x}+t\mathbf{z}) = A\mathbf{x}+tA\mathbf{z} = \mathbf{y}+t\mathbf{0} = \mathbf{y}. \end{equation*}
Thus every input of the form \(\mathbf{x}+t\mathbf{z}\) gives the same output.
🔗
🔗
🔗

Activity 2.6.3. Reading an augmented rref output.

The code row-reduces the augmented matrix for trying to solve
🔗
\begin{equation*} A\mathbf{x}=\mathbf{b}, \qquad A= \begin{bmatrix} 1\amp 0\\ 0\amp 1\\ 1\amp 1 \end{bmatrix}, \qquad \mathbf{b} = \begin{bmatrix} 1\\ 2\\ 4 \end{bmatrix}. \end{equation*} 
import sympy as sp

# Augmented matrix [A | b]
M = sp.Matrix([
    [1, 0, 1],
    [0, 1, 2],
    [1, 1, 4],
])

M.rref()
Output:
🔗
(Matrix([
[1, 0, 0],
[0, 1, 0],
[0, 0, 1]]), (0, 1, 2))
Read the displayed matrix as
🔗
\begin{equation*} \left[ \begin{array}{cc|c} 1\amp0\amp0\\ 0\amp1\amp0\\ 0\amp0\amp1 \end{array} \right]. \end{equation*}
  1. Which column is to the right of the vertical line?
    🔗
    🔗
  2. What does the last row represent?
    🔗
    🔗
  3. Is \(\mathbf{b}\) reachable as an output of \(A\text{?}\)
    🔗
    🔗
  4. What does the pivot to the right of the vertical line mean?
    🔗
    🔗
Solution.
The third column is to the right of the vertical line. It is the right-hand side column, not a variable column.
🔗
The last row is
🔗
\begin{equation*} [0\ 0\mid 1], \end{equation*}
which represents the equation
🔗
\begin{equation*} 0=1. \end{equation*}
That equation is impossible, so the system is inconsistent. Therefore \(\mathbf{b}\) is not reachable as an output of \(A\text{.}\)
🔗
A pivot to the right of the vertical line means the system has no solution.
🔗
🔗
🔗

Activity 2.6.4. Redundant square-footage features in code.

Rows represent houses. The columns are first-level area, second-level area, and total area, measured in hundreds of square feet.
🔗
import numpy as np

X = np.array([
    [9, 7, 16],
    [11, 9, 20],
    [14, 0, 14],
    [8, 8, 16],
])

z = np.array([1, 1, -1])
c = np.array([3, 1, 0])
c_alt = np.array([2, 0, 1])

rank = int(np.linalg.matrix_rank(X))

X @ z, rank, X @ c, X @ c_alt
Output:
🔗
(array([0, 0, 0, 0]), 2, array([34, 42, 42, 32]), array([34, 42, 42, 32]))
  1. What does \(X\mathbf{z}=\mathbf{0}\) say about the three feature columns?
    🔗
    🔗
  2. What does \(\operatorname{rank}(X)=2\) say about the three features?
    🔗
    🔗
  3. Why do \(X\mathbf{c}\) and \(X\mathbf{c}_{\mathrm{alt}}\) agree?
    🔗
    🔗
  4. What warning does this give about interpreting individual coefficients?
    🔗
    🔗
Solution.
The equation
🔗
\begin{equation*} X \begin{bmatrix} 1\\ 1\\ -1 \end{bmatrix} = \mathbf{0} \end{equation*}
means that
🔗
\begin{equation*} \text{first-level area}+\text{second-level area}-\text{total area}=0 \end{equation*}
for every row. Equivalently,
🔗
\begin{equation*} \text{total area} = \text{first-level area} + \text{second-level area}. \end{equation*}
The three columns contain only two independent feature directions, so \(\operatorname{rank}(X)=2\text{.}\)
🔗
The coefficient vectors differ by the null-space direction:
🔗
\begin{equation*} \mathbf{c}-\mathbf{c}_{\mathrm{alt}} = \begin{bmatrix} 3\\ 1\\ 0 \end{bmatrix} - \begin{bmatrix} 2\\ 0\\ 1 \end{bmatrix} = \begin{bmatrix} 1\\ 1\\ -1 \end{bmatrix} = \mathbf{z}. \end{equation*}
Since \(X\mathbf{z}=\mathbf{0}\text{,}\)
🔗
\begin{equation*} X\mathbf{c} = X(\mathbf{c}_{\mathrm{alt}}+\mathbf{z}) = X\mathbf{c}_{\mathrm{alt}}+X\mathbf{z} = X\mathbf{c}_{\mathrm{alt}}. \end{equation*}
The predictions agree, but the coefficient stories are different. With redundant features, individual coefficient importance may not be uniquely determined.
🔗
🔗
🔗

Activity 2.6.5. Debugging a column-space basis from code.

A student computes an rref and pivot columns.
🔗
import sympy as sp

A = sp.Matrix([
    [1, 2, 3],
    [2, 4, 6],
    [0, 1, 1],
])

R, pivots = A.rref()
R, pivots
Output:
🔗
(Matrix([
[1, 0, 1],
[0, 1, 1],
[0, 0, 0]]), (0, 1))
The student says:
🔗
\begin{equation*} \text{``Columns 1 and 2 of }R\text{ are a basis for }\operatorname{col}(A).\text{''} \end{equation*}
  1. What do the pivot indices (0, 1) mean in mathematical column numbering?
    🔗
    🔗
  2. What is wrong with using columns of \(R\) as a basis for \(\operatorname{col}(A)\text{?}\)
    🔗
    🔗
  3. Which columns should be used instead?
    🔗
    🔗
  4. Write a basis for \(\operatorname{col}(A)\text{.}\)
    🔗
    🔗
Solution.
The pivot indices (0, 1) mean mathematical columns 1 and 2.
🔗
The mistake is that row operations do not preserve the original columns of \(A\text{.}\) Row reduction identifies pivot positions, but a basis for \(\operatorname{col}(A)\) must use columns of the original matrix \(A\text{.}\)
🔗
Therefore we use columns 1 and 2 of \(A\text{:}\)
🔗
\begin{equation*} \left\{ \begin{bmatrix} 1\\ 2\\ 0 \end{bmatrix}, \begin{bmatrix} 2\\ 4\\ 1 \end{bmatrix} \right\}. \end{equation*}
These vectors form a basis for \(\operatorname{col}(A)\text{.}\)
🔗
🔗
🔗

Activity 2.6.6. Determinant and inverse diagnostics in code.

The following code checks determinant and rank for two square matrices.
🔗
import numpy as np

A = np.array([
    [1., 1.],
    [0., 1.],
])

B = np.array([
    [1., 0.],
    [0., 0.],
])

detA = float(np.linalg.det(A))
detB = float(np.linalg.det(B))

rankA = int(np.linalg.matrix_rank(A))
rankB = int(np.linalg.matrix_rank(B))

detA, rankA, detB, rankB
Output:
🔗
(1.0, 2, 0.0, 1)
  1. Which matrix is invertible?
    🔗
    🔗
  2. Which matrix forgets a nonzero input direction?
    🔗
    🔗
  3. What do determinant and rank each say?
    🔗
    🔗
  4. Should \(B\mathbf{x}=\mathbf{b}\) have a unique solution for every \(\mathbf{b}\in\mathbb{R}^2\text{?}\)
    🔗
    🔗
Solution.
Matrix \(A\) is invertible because
🔗
\begin{equation*} \det(A)=1\ne 0 \end{equation*}
\begin{equation*} \operatorname{rank}(A)=2. \end{equation*}
Matrix \(B\) is singular because
🔗
\begin{equation*} \det(B)=0 \end{equation*}
\begin{equation*} \operatorname{rank}(B)=1\lt 2. \end{equation*}
The matrix \(B\) forgets the nonzero direction
🔗
\begin{equation*} \begin{bmatrix} 0\\ 1 \end{bmatrix}, \end{equation*}
\begin{equation*} B \begin{bmatrix} 0\\ 1 \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}. \end{equation*}
The system \(B\mathbf{x}=\mathbf{b}\) should not be expected to have a unique solution for every \(\mathbf{b}\text{.}\) The map is not invertible.
🔗
🔗
🔗
Exam skill. Given a short Unit 2 code cell or printed output, identify the linear-algebra object being computed, read the output using Unit 2 language, and connect it to reachable outputs, forgotten directions, rank, nullity, redundancy, determinant tests, or inverse checks.
🔗
🔗

Subsection Linked notebook

Run Lab U2: Understanding a Linear Map
 1 
sebroc.github.io/MATH345-Course-Materials/labs/#lab-u2
. The core path practices projection, reachable outputs, row reduction, plane intersections, difference matrices, rank, null vectors, redundant features, determinants, and inverse checks.
🔗
🔗
For a quick reference on arrays, matrix products, numerical rank, determinants, solves, row reduction, and null spaces, see the programming appendix sections B.2, B.4, B.10, and B.11.
🔗
🔗
PrevTopNext