Skip to main content
Contents Dark Mode Prev Up Next \(\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\dimens}{\operatorname{dim}}
\DeclareMathOperator{\row}{\operatorname{row}}
\DeclareMathOperator{\col}{\operatorname{col}}
\newcommand{\im}{\operatorname{im}}
\newcommand{\nulls}{\operatorname{null}}
\newcommand{\minor}{\operatorname{minor}}
\newcommand{\spans}{\operatorname{span}}
\newcommand{\nullity}{\operatorname{nullity}}
\newcommand{\kers}{\operatorname{ker}}
\newcommand{\proj}{\operatorname{proj}}
\newcommand{\diag}{\operatorname{diag}}
\newcommand{\Tr}{\operatorname{Tr}}
\newcommand{\rank}{\operatorname{rank}}
\newcommand{\lt}{<}
\newcommand{\gt}{>}
\newcommand{\amp}{&}
\definecolor{fillinmathshade}{gray}{0.9}
\newcommand{\fillinmath}[1]{\mathchoice{\colorbox{fillinmathshade}{$\displaystyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\textstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptscriptstyle\phantom{\,#1\,}$}}}
\)
Section B.10 NumPy Linear Algebra Commands
Use this section when code calls common NumPy linear-algebra commands: rank, determinant, solve, least squares, eigenvalues, QR, or SVD.
Basic computations. These commands return single quantities or solutions for common linear-algebra tasks.
Norm.
Read as. The norm
\(\|x\|\text{.}\)
Used for. Lengths, distances, and residuals.
Numerical rank.
Read as. The numerical rank of
\(A\text{.}\)
Shape/return. An integer.
Used for. Independent directions.
Determinant.
Read as. The determinant of square
\(A\text{.}\)
Used for. Invertibility checks.
Linear solve.
Read as. Solve
\(Ax=b\) numerically.
Shape/return. A vector or matrix.
Used for. Square full-rank systems.
Least-squares coefficients. np.linalg.lstsq(A, b, rcond=None)[0]
Read as. The least-squares coefficient vector.
Used for. Regression, projection, and sampled polynomial approximation.
Eigenvalue commands. These commands read square or symmetric matrices through eigenvalues and eigenvectors.
General eigenvalues.
Read as. Eigenvalues and eigenvectors of a square matrix.
Shape/return. A pair: eigenvalue array and eigenvector matrix.
Used for. General eigenvalue computations.
Watch for. For symmetric real matrices,
eigvalsh or
eigh is usually cleaner.
Symmetric eigenvalues.
Read as. Eigenvalues of a symmetric matrix.
Shape/return. A one-dimensional array of real eigenvalues.
Used for. Hessian classification and symmetric quadratic forms.
Symmetric eigenvectors.
Read as. Eigenvalues and orthonormal eigenvectors of a symmetric matrix.
Shape/return. A pair: eigenvalue array and eigenvector matrix.
Used for. Symmetric quadratic forms, covariance matrices, and
\(A^TA\text{.}\)
Watch for. Eigenvalues are returned in increasing order.
Outer product.
Read as. The outer product
\(\mathbf{g}\mathbf{h}^T\text{.}\)
Shape/return. Shape
(m, d) when
g has length
m and
h has length
d.
Used for. PCA reconstructions and rank-one matrices.
Watch for. This is different from
g @ h, which is a dot product when both are one-dimensional arrays of the same length.
Factorizations. These commands split a matrix into pieces that can be reused for interpretation or reconstruction.
QR factorization.
Read as. QR factorization.
Used for. Least-squares checks.
Compact SVD.
np.linalg.svd(A, full_matrices=False)
Read as. Compact singular value decomposition.
Shape/return. U,
s, and
Vt.
Used for. Matrix stretch, PCA, rank, and singular-vector identities.
Full SVD.
np.linalg.svd(A, full_matrices=True)
Read as. Full singular value decomposition.
Shape/return. U,
s, and
Vt; the columns of
U and the rows of
Vt give complete orthonormal bases.
Used for. The four fundamental subspaces, especially the left null space.
Watch for. The length of
s is
\(\min(m,n)\text{.}\)
result = np.linalg.lstsq(A, b, rcond=None)
xhat = result[0]
The command
np.linalg.lstsq returns several pieces of information. The coefficient vector is the first item, so the textbook often writes
[0].
U, s, Vt = np.linalg.svd(Z, full_matrices=False)
V = Vt.T
T = Z @ V
covariance_eigenvalues = s**2 / len(Z)
V stores the principal directions as columns.
T stores the principal scores, with
\begin{equation*}
T=ZV=U\Sigma.
\end{equation*}
s**2 / len(Z) gives the covariance eigenvalues.
full_matrices=False returns compact arrays that are convenient for PCA and reconstruction.
Additional exercise B.10.1 . Check Yourself: Least Squares Output.
In the line that defines
xhat using
np.linalg.lstsq, what does
xhat represent? Why is
[0] used? What should be close to zero if the residual is orthogonal to the columns of
A?
Additional exercise B.10.2 . Check Yourself: PCA and SVD.
If
V = Vt.T and
T = Z @ V, what do the columns of
V represent, and what does the matrix
T store? Why do the covariance eigenvalues equal
s**2 / len(Z)?
