Skip to main content
Contents Dark Mode Prev Up Next \(\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\dimens}{\operatorname{dim}}
\DeclareMathOperator{\row}{\operatorname{row}}
\DeclareMathOperator{\col}{\operatorname{col}}
\newcommand{\im}{\operatorname{im}}
\newcommand{\nulls}{\operatorname{null}}
\newcommand{\minor}{\operatorname{minor}}
\newcommand{\spans}{\operatorname{span}}
\newcommand{\nullity}{\operatorname{nullity}}
\newcommand{\kers}{\operatorname{ker}}
\newcommand{\proj}{\operatorname{proj}}
\newcommand{\diag}{\operatorname{diag}}
\newcommand{\Tr}{\operatorname{Tr}}
\newcommand{\rank}{\operatorname{rank}}
\newcommand{\lt}{<}
\newcommand{\gt}{>}
\newcommand{\amp}{&}
\definecolor{fillinmathshade}{gray}{0.9}
\newcommand{\fillinmath}[1]{\mathchoice{\colorbox{fillinmathshade}{$\displaystyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\textstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptscriptstyle\phantom{\,#1\,}$}}}
\)
Section 1.6 Exercises
Learning outcomes. The labels below identify the learning outcomes for each exercise group. Individual problems may also involve earlier outcomes.
Subsection Matrices
Learning outcomes. U1-LO3.
Subsection Matrix-vector multiplication
Learning outcomes. U1-LO4.
Subsection Matrix multiplication
Learning outcomes. U1-LO6.
Subsection Additional exercises
These exercises connect the main Unit 1 ideas: vectors as data, cosine similarity, weighted averages, matrix-vector products, geometric matrix actions, composition, affine maps, and code interpretation. Solutions are collected in Appendix
C.1 .
Additional exercise 1.6.1 . Word-count vectors and cosine similarity.
Use the dictionary linear, matrix, data. Let
\begin{equation*}
\mathbf{q}=\begin{bmatrix}1\\0\\1\end{bmatrix},
\end{equation*}
\begin{equation*}
\mathbf{D}_1=\begin{bmatrix}2\\0\\2\end{bmatrix},\qquad \mathbf{D}_2=\begin{bmatrix}1\\1\\0\end{bmatrix},\qquad \mathbf{D}_3=\begin{bmatrix}0\\2\\0\end{bmatrix}.
\end{equation*}
Compute the cosine similarity between
\(\mathbf{q}\) and each document vector.
Rank the documents by cosine similarity.
Explain why
\(\mathbf{D}_1\) has cosine similarity
\(1\) with
\(\mathbf{q}\text{.}\)
Which document is orthogonal to
\(\mathbf{q}\text{?}\)
What does that orthogonality mean in terms of the dictionary words?
Compute the Euclidean distance from
\(\mathbf{q}\) to each
\(\mathbf{D}_i\text{.}\) Does the distance ranking match the cosine ranking?
Learning outcomes. U1-LO1, U1-LO2.
Additional exercise 1.6.2 . Weighted average of value vectors.
Let
\begin{equation*}
\mathbf{v}_1=\begin{bmatrix}10\\0\end{bmatrix},\qquad \mathbf{v}_2=\begin{bmatrix}0\\10\end{bmatrix},\qquad \mathbf{v}_3=\begin{bmatrix}10\\10\end{bmatrix},
\end{equation*}
and let
\begin{equation*}
\boldsymbol{\alpha}=\begin{bmatrix}\frac14\\\frac14\\\frac12\end{bmatrix}.
\end{equation*}
Compute
\(\alpha_1\mathbf{v}_1+\alpha_2\mathbf{v}_2+\alpha_3\mathbf{v}_3\text{.}\)
Explain why this is a weighted average.
Which value vector receives the largest weight?
Learning outcomes. U1-LO8.
Additional exercise 1.6.3 . Attention bridge: hand scores versus \(K\mathbf{q}\) .
Use the token attention activity with tokens “small”, “red”, “bird”. Let
\begin{equation*}
\mathbf{q}=\begin{bmatrix}1\\1\end{bmatrix},
\end{equation*}
\begin{equation*}
\mathbf{k}_{\mathrm{small}}=\begin{bmatrix}1\\0\end{bmatrix},\qquad \mathbf{k}_{\mathrm{red}}=\begin{bmatrix}0\\1\end{bmatrix},\qquad \mathbf{k}_{\mathrm{bird}}=\begin{bmatrix}1\\1\end{bmatrix}.
\end{equation*}
Compute the three dot products
\begin{equation*}
\mathbf{q}\cdot \mathbf{k}_{\mathrm{small}},\qquad \mathbf{q}\cdot \mathbf{k}_{\mathrm{red}},\qquad \mathbf{q}\cdot \mathbf{k}_{\mathrm{bird}}.
\end{equation*}
Put the key vectors as rows of the matrix
\begin{equation*}
K=\begin{bmatrix}1\amp 0\\ 0\amp 1\\ 1\amp 1\end{bmatrix}.
\end{equation*}
Compute \(K\mathbf{q}\text{.}\)
Explain why the entries of
\(K\mathbf{q}\) are the same scores from part (a).
Normalize the scores by dividing by their sum. What is the weight vector
\(\boldsymbol{\alpha}\text{?}\)
Learning outcomes. U1-LO1, U1-LO4, U1-LO8.
Additional exercise 1.6.4 . Rows measure; columns contribute.
Let
\begin{equation*}
A=\begin{bmatrix}-1\amp 4\amp -5\\ 3\amp 1\amp -2\end{bmatrix},
\qquad
\mathbf{x}=\begin{bmatrix}2\\ -3\\ 4\end{bmatrix}.
\end{equation*}
Compute
\(A\mathbf{x}\) using row dot products.
Compute
\(A\mathbf{x}\) as a linear combination of the columns of
\(A\text{.}\)
In this example, what does the first row of
\(A\) measure?
In this example, what do the coordinates
\(2,-3,4\) tell you in the column-combination view?
Learning outcomes. U1-LO4.
Additional exercise 1.6.5 . The unit-square visualization.
Let
\begin{equation*}
A=\begin{bmatrix}2\amp -1\\ 1\amp 1\end{bmatrix}.
\end{equation*}
Compute
\(A\mathbf{e}_1\) and
\(A\mathbf{e}_2\text{.}\)
Compute
\(A\begin{bmatrix}1\\1\end{bmatrix}\text{.}\)
Apply \(A\) to the four corners of the unit square:
\begin{equation*}
\begin{bmatrix}0\\0\end{bmatrix},\qquad \begin{bmatrix}1\\0\end{bmatrix},\qquad \begin{bmatrix}1\\1\end{bmatrix},\qquad \begin{bmatrix}0\\1\end{bmatrix}.
\end{equation*}
Sketch the image of the unit square.
Which two transformed edges leaving the origin are the columns of
\(A\text{?}\)
Learning outcomes. U1-LO5, U1-LO8.
Additional exercise 1.6.6 . Transformation matching.
Additional exercise 1.6.7 . Composition order: \(RS\) versus \(SR\) .
Let
\begin{equation*}
S=\begin{bmatrix}2\amp 0\\ 0\amp 1\end{bmatrix},
\qquad
R=\begin{bmatrix}0\amp -1\\ 1\amp 0\end{bmatrix}.
\end{equation*}
Are
\(RS\) and
\(SR\) equal?
Which product represents “apply
\(S\) first, then apply
\(R\) ”?
Explain geometrically why the two orders produce different results.
Learning outcomes. U1-LO6.
Additional exercise 1.6.8 . Affine but not linear.
Consider the transformation
\begin{equation*}
T\begin{bmatrix}x\\y\end{bmatrix}=\begin{bmatrix}x+1\\y\end{bmatrix}.
\end{equation*}
Compute
\(T(0,0)\text{.}\)
Explain why
\(T\) cannot be written in the form
\(T(\mathbf{x})=A\mathbf{x}\) for any matrix
\(A\text{.}\)
Write \(T\) in the affine form
\begin{equation*}
T(\mathbf{x})=A\mathbf{x}+\mathbf{b}.
\end{equation*}
Is
\(T\) linear, affine but not linear, or neither?
Learning outcomes. U1-LO7.
Additional exercise 1.6.9 . Code interpretation: sorting similarity scores.
import numpy as np
doc_names = np.array(["D1", "D2", "D3"])
scores = np.array([1.0, 0.5, 0.0])
ranking = doc_names[np.argsort(scores)[::-1]]
ranking
What is the value of
ranking?
What does
np.argsort(scores) return?
Is this ranking based on direction, distance, or alphabetical order?
Learning outcomes. U1-LO2, U1-LO8.
Additional exercise 1.6.10 . Transformer shape check.
