Skip to main content

MATH 345: Linear Algebra and Optimization

Sebastien Roch with Jacob Denson and Kaitlyn Phillipson

Section 2.6 Coding recap

Subsection Linked notebook

Run Lab U2: Understanding a Linear Map
 1 
sebroc.github.io/MATH345-Course-Materials/labs/#lab-u2
. The main sequence practices projection, reachable outputs, row reduction, plane intersections, difference matrices, rank, null vectors, redundant features, determinants, and inverse checks.
🔗
For a quick reference on arrays, matrix products, numerical rank, determinants, solves, row reduction, and null spaces, see the programming appendix sections B.2, B.4, B.10, and B.11.
🔗
🔗

Subsection Review activities

Activity 2.6.1. A sensor that drops height (U2-LO3, U2-LO6).

A sensor records only horizontal position. It keeps the first two coordinates and drops height.
🔗
import numpy as np

P = np.array([
    [1., 0., 0.],
    [0., 1., 0.],
])

x = np.array([2., 1., 5.])
z = np.array([0., 0., 1.])

P @ x, P @ (x + 6*z), P @ z
Output:
🔗
(array([2., 1.]), array([2., 1.]), array([0., 0.]))
  1. What does \(P\) do to an input vector?
    🔗
    🔗
  2. Why do \(P\mathbf{x}\) and \(P(\mathbf{x}+6\mathbf{z})\) agree?
    🔗
    🔗
  3. What does \(P\mathbf{z}=\mathbf{0}\) say about the direction \(\mathbf{z}\text{?}\)
    🔗
    🔗
  4. Why can the original input not be recovered uniquely from the output?
    🔗
    🔗
Solution.
The matrix \(P\) keeps the first two coordinates and forgets the third.
🔗
The vectors
🔗
\begin{equation*} \mathbf{x} = \begin{bmatrix} 2\\ 1\\ 5 \end{bmatrix} \qquad\text{and}\qquad \mathbf{x}+6\mathbf{z} = \begin{bmatrix} 2\\ 1\\ 11 \end{bmatrix} \end{equation*}
have different heights, but the same first two coordinates. Therefore their outputs agree.
🔗
Since
🔗
\begin{equation*} P\mathbf{z}=\mathbf{0}, \end{equation*}
the direction
🔗
\begin{equation*} \mathbf{z} = \begin{bmatrix} 0\\ 0\\ 1 \end{bmatrix} \end{equation*}
is a forgotten direction. Inputs that differ by a multiple of \(\mathbf{z}\) have the same output.
🔗
🔗
🔗

Activity 2.6.2. Reading a null-space basis in code (U2-LO3, U2-LO6).

The following code asks SymPy for a basis of a null space. Recall from Activity 2.4.56 how to select a vector from the returned list and multiply it by the matrix.
🔗
import sympy as sp

A = sp.Matrix([
    [1, 2, 3],
    [2, 4, 6],
    [0, 1, 1],
])

A.nullspace()
Output:
🔗
[Matrix([
[-1],
[-1],
[ 1]])]
  1. What kind of object does nullspace() return?
    🔗
    🔗
  2. Is the displayed vector an input direction or an output direction?
    🔗
    🔗
  3. Verify from the code output that \(\mathbf{z} = \begin{bmatrix} -1\\ -1\\ 1 \end{bmatrix}\) is in \(\operatorname{null}(A)\text{.}\)
    🔗
    🔗
  4. If \(A\mathbf{x}=\mathbf{y}\text{,}\) what is \(A(\mathbf{x}+t\mathbf{z})\text{?}\)
    🔗
    🔗
Solution.
The command returns a list of basis vectors for \(\operatorname{null}(A)\text{.}\)
🔗
The vector
🔗
\begin{equation*} \mathbf{z} = \begin{bmatrix} -1\\ -1\\ 1 \end{bmatrix} \end{equation*}
is an input direction. It satisfies
🔗
\begin{equation*} A\mathbf{z} = \mathbf{0}. \end{equation*}
If \(A\mathbf{x}=\mathbf{y}\text{,}\) then
🔗
\begin{equation*} A(\mathbf{x}+t\mathbf{z}) = A\mathbf{x}+tA\mathbf{z} = \mathbf{y}+t\mathbf{0} = \mathbf{y}. \end{equation*}
Thus every input of the form \(\mathbf{x}+t\mathbf{z}\) gives the same output.
🔗
🔗
🔗

Activity 2.6.3. Reading an augmented rref output (U2-LO1, U2-LO2, U2-LO3).

Recall from Activity 2.2.54 how to read row-reduction output. For unpacking the result and selecting entries using zero-based indices, see Activity 2.4.41.
🔗
The code row-reduces the augmented matrix for trying to solve
🔗
\begin{equation*} A\mathbf{x}=\mathbf{b}, \qquad A= \begin{bmatrix} 1\amp 0\\ 0\amp 1\\ 1\amp 1 \end{bmatrix}, \qquad \mathbf{b} = \begin{bmatrix} 1\\ 2\\ 4 \end{bmatrix}. \end{equation*} 
import sympy as sp

# Augmented matrix [A | b]
M = sp.Matrix([
    [1, 0, 1],
    [0, 1, 2],
    [1, 1, 4],
])

M.rref()
Output:
🔗
(Matrix([
[1, 0, 0],
[0, 1, 0],
[0, 0, 1]]), (0, 1, 2))
Read the displayed matrix as
🔗
\begin{equation*} \left[ \begin{array}{cc|c} 1\amp0\amp0\\ 0\amp1\amp0\\ 0\amp0\amp1 \end{array} \right]. \end{equation*}
  1. What does the last row represent?
    🔗
    🔗
  2. Is \(\mathbf{b}\) reachable as an output of \(A\text{?}\)
    🔗
    🔗
  3. What does the pivot to the right of the vertical line mean?
    🔗
    🔗
Solution.
The last row is
🔗
\begin{equation*} [0\ 0\mid 1], \end{equation*}
which represents the equation
🔗
\begin{equation*} 0=1. \end{equation*}
That equation is impossible, so the system is inconsistent. Therefore \(\mathbf{b}\) is not reachable as an output of \(A\text{.}\)
🔗
A pivot to the right of the vertical line means the system has no solution.
🔗
🔗
🔗

Activity 2.6.4. Redundant exam-score features in code (U2-LO3, U2-LO6).

Use the exam-parts-and-total example from Section 2.4. Rows represent students. The four columns record scores on part 1, part 2, part 3, and the exam total, all measured in points.
🔗
import numpy as np

X = np.array([
    [8., 6., 4., 18.],
    [5., 9., 7., 21.],
    [7., 4., 9., 20.],
    [6., 8., 5., 19.],
])

z = np.array([1., 1., 1., -1.])
c = np.array([2., 3., 1., 0.])
c_alt = np.array([1., 2., 0., 1.])

rank = np.linalg.matrix_rank(X)

X @ z, rank, X @ c, X @ c_alt
Output:
🔗
(array([0., 0., 0., 0.]),
 3,
 array([38., 44., 35., 41.]),
 array([38., 44., 35., 41.]))
  1. What does \(X\mathbf{z}=\mathbf{0}\) say about the four feature columns?
    🔗
    🔗
  2. What does \(\operatorname{rank}(X)=3\) say about the four features?
    🔗
    🔗
  3. Why do \(X\mathbf{c}\) and \(X\mathbf{c}_{\mathrm{alt}}\) agree?
    🔗
    🔗
  4. What warning does this give about interpreting individual coefficients?
    🔗
    🔗
Solution.
The equation
🔗
\begin{equation*} X \begin{bmatrix} 1\\ 1\\ 1\\ -1 \end{bmatrix} = \mathbf{0} \end{equation*}
means that
🔗
\begin{equation*} \text{total} = \text{part 1}+\text{part 2}+\text{part 3} \end{equation*}
for every student. The total column is a linear combination of the three part-score columns.
🔗
The four columns contain only three independent feature directions, so \(\operatorname{rank}(X)=3\text{.}\) The total adds no independent feature direction.
🔗
The coefficient vectors differ by the null-space direction:
🔗
\begin{equation*} \mathbf{c}-\mathbf{c}_{\mathrm{alt}} = \begin{bmatrix} 2\\ 3\\ 1\\ 0 \end{bmatrix} - \begin{bmatrix} 1\\ 2\\ 0\\ 1 \end{bmatrix} = \begin{bmatrix} 1\\ 1\\ 1\\ -1 \end{bmatrix} = \mathbf{z}. \end{equation*}
Since \(X\mathbf{z}=\mathbf{0}\text{,}\)
🔗
\begin{equation*} X\mathbf{c} = X(\mathbf{c}_{\mathrm{alt}}+\mathbf{z}) = X\mathbf{c}_{\mathrm{alt}}+X\mathbf{z} = X\mathbf{c}_{\mathrm{alt}}. \end{equation*}
The first coefficient vector puts weights \(2,3,1\) on the three parts and zero on the total. The second puts weights \(1,2,0\) on the parts and \(1\) on the total. They give the same predictions, so these coefficients alone do not identify a uniquely most important exam feature.
🔗
🔗
🔗

Activity 2.6.5. Debugging a column-space basis from code (U2-LO3, U2-LO4).

A student computes an rref and pivot columns. Recall the column slicing and list comprehension from Activity 2.4.41.
🔗
import sympy as sp

A = sp.Matrix([
    [1, 2, 3],
    [2, 4, 6],
    [0, 1, 1],
])

R, pivots = A.rref()
R, pivots
Output:
🔗
(Matrix([
[1, 0, 1],
[0, 1, 1],
[0, 0, 0]]), (0, 1))
The student says: “Columns 1 and 2 of \(R\) are a basis for \(\operatorname{col}(A)\text{.}\)”
🔗
  1. What do the pivot indices (0, 1) mean in mathematical column numbering?
    🔗
    🔗
  2. What is wrong with using columns of \(R\) as a basis for \(\operatorname{col}(A)\text{?}\)
    🔗
    🔗
  3. Which columns should be used instead?
    🔗
    🔗
  4. Write a basis for \(\operatorname{col}(A)\text{.}\)
    🔗
    🔗
Solution.
The pivot indices (0, 1) mean mathematical columns 1 and 2.
🔗
The mistake is that row operations do not preserve the original columns of \(A\text{.}\) Row reduction identifies pivot positions, but a basis for \(\operatorname{col}(A)\) must use columns of the original matrix \(A\text{.}\)
🔗
Therefore we use columns 1 and 2 of \(A\text{:}\)
🔗
\begin{equation*} \left\{ \begin{bmatrix} 1\\ 2\\ 0 \end{bmatrix}, \begin{bmatrix} 2\\ 4\\ 1 \end{bmatrix} \right\}. \end{equation*}
These vectors form a basis for \(\operatorname{col}(A)\text{.}\)
🔗
🔗
🔗

Activity 2.6.6. Determinant and inverse diagnostics in code (U2-LO3, U2-LO5, U2-LO7).

The following code checks determinant and rank for two square matrices.
🔗
import numpy as np

A = np.array([
    [1., 1.],
    [0., 1.],
])

B = np.array([
    [1., 0.],
    [0., 0.],
])

detA = float(np.linalg.det(A))
detB = float(np.linalg.det(B))

rankA = np.linalg.matrix_rank(A)
rankB = np.linalg.matrix_rank(B)

detA, rankA, detB, rankB
Output:
🔗
(1.0, 2, 0.0, 1)
  1. Which matrix is invertible?
    🔗
    🔗
  2. Which matrix forgets a nonzero input direction?
    🔗
    🔗
  3. What do determinant and rank each say?
    🔗
    🔗
  4. Should \(B\mathbf{x}=\mathbf{b}\) have a unique solution for every \(\mathbf{b}\in\mathbb{R}^2\text{?}\)
    🔗
    🔗
Solution.
Matrix \(A\) is invertible because
🔗
\begin{equation*} \det(A)=1\ne 0 \end{equation*}
and
🔗
\begin{equation*} \operatorname{rank}(A)=2. \end{equation*}
Matrix \(B\) is singular because
🔗
\begin{equation*} \det(B)=0 \end{equation*}
and
🔗
\begin{equation*} \operatorname{rank}(B)=1\lt 2. \end{equation*}
The matrix \(B\) forgets the nonzero direction
🔗
\begin{equation*} \begin{bmatrix} 0\\ 1 \end{bmatrix}, \end{equation*}
since
🔗
\begin{equation*} B \begin{bmatrix} 0\\ 1 \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}. \end{equation*}
The system \(B\mathbf{x}=\mathbf{b}\) should not be expected to have a unique solution for every \(\mathbf{b}\text{.}\) The map is not invertible.
🔗
🔗
🔗
🔗
🔗
PrevTopNext