Skip to main content
Contents Dark Mode Prev Up Next \(\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\dimens}{\operatorname{dim}}
\DeclareMathOperator{\row}{\operatorname{row}}
\DeclareMathOperator{\col}{\operatorname{col}}
\newcommand{\im}{\operatorname{im}}
\newcommand{\nulls}{\operatorname{null}}
\newcommand{\minor}{\operatorname{minor}}
\newcommand{\spans}{\operatorname{span}}
\newcommand{\nullity}{\operatorname{nullity}}
\newcommand{\kers}{\operatorname{ker}}
\newcommand{\proj}{\operatorname{proj}}
\newcommand{\diag}{\operatorname{diag}}
\newcommand{\Tr}{\operatorname{Tr}}
\newcommand{\rank}{\operatorname{rank}}
\newcommand{\lt}{<}
\newcommand{\gt}{>}
\newcommand{\amp}{&}
\definecolor{fillinmathshade}{gray}{0.9}
\newcommand{\fillinmath}[1]{\mathchoice{\colorbox{fillinmathshade}{$\displaystyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\textstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptscriptstyle\phantom{\,#1\,}$}}}
\)
Section 1.6 Exercises
Exercise tags. The tags below identify the primary outcomes and competency types for each exercise group. Individual problems may also involve earlier outcomes.
Subsection Matrices
Primary outcomes. U1-LO3.
Competencies. P.
Subsection Matrix-vector multiplication
Primary outcomes. U1-LO4.
Competencies. P+R.
Subsection Matrix multiplication
Primary outcomes. U1-LO6.
Competencies. P+R.
Subsection Additional applied and computational problems
These exercises connect the main Unit 1 ideas: vectors as data, cosine similarity, weighted averages, matrix-vector products, geometric matrix actions, composition, affine maps, and code interpretation. Solutions are collected in Appendix
C.1 .
Checkpoint 1.6.1 . Word-count vectors and cosine similarity.
Use the dictionary linear, matrix, data. Let
\begin{equation*}
\mathbf{q}=\begin{bmatrix}1\\0\\1\end{bmatrix},
\end{equation*}
\begin{equation*}
\mathbf{D}_1=\begin{bmatrix}2\\0\\2\end{bmatrix},\qquad \mathbf{D}_2=\begin{bmatrix}1\\1\\0\end{bmatrix},\qquad \mathbf{D}_3=\begin{bmatrix}0\\2\\0\end{bmatrix}.
\end{equation*}
Compute the cosine similarity between
\(\mathbf{q}\) and each document vector.
Rank the documents by cosine similarity.
Explain why
\(\mathbf{D}_1\) has cosine similarity
\(1\) with
\(\mathbf{q}\text{.}\)
Which document is orthogonal to
\(\mathbf{q}\text{?}\)
What does that orthogonality mean in terms of the dictionary words?
Compute the Euclidean distance from
\(\mathbf{q}\) to each
\(\mathbf{D}_i\text{.}\) Does the distance ranking match the cosine ranking?
Tags. [U1-LO1, U1-LO2 | C+R | Core]
Checkpoint 1.6.2 . Weighted average of value vectors.
Let
\begin{equation*}
\mathbf{v}_1=\begin{bmatrix}10\\0\end{bmatrix},\qquad \mathbf{v}_2=\begin{bmatrix}0\\10\end{bmatrix},\qquad \mathbf{v}_3=\begin{bmatrix}10\\10\end{bmatrix},
\end{equation*}
and let
\begin{equation*}
\boldsymbol{\alpha}=\begin{bmatrix}\frac14\\\frac14\\\frac12\end{bmatrix}.
\end{equation*}
Compute
\(\alpha_1\mathbf{v}_1+\alpha_2\mathbf{v}_2+\alpha_3\mathbf{v}_3\text{.}\)
Explain why this is a weighted average.
Which value vector receives the largest weight?
Tags. [U1-LO8 | P+C | Core]
Checkpoint 1.6.3 . Attention bridge: hand scores versus \(K\mathbf{q}\) .
Use the token attention activity with tokens “small”, “red”, “bird”. Let
\begin{equation*}
\mathbf{q}=\begin{bmatrix}1\\1\end{bmatrix},
\end{equation*}
\begin{equation*}
\mathbf{k}_{\mathrm{small}}=\begin{bmatrix}1\\0\end{bmatrix},\qquad \mathbf{k}_{\mathrm{red}}=\begin{bmatrix}0\\1\end{bmatrix},\qquad \mathbf{k}_{\mathrm{bird}}=\begin{bmatrix}1\\1\end{bmatrix}.
\end{equation*}
Compute the three dot products
\begin{equation*}
\mathbf{q}\cdot \mathbf{k}_{\mathrm{small}},\qquad \mathbf{q}\cdot \mathbf{k}_{\mathrm{red}},\qquad \mathbf{q}\cdot \mathbf{k}_{\mathrm{bird}}.
\end{equation*}
Put the key vectors as rows of the matrix
\begin{equation*}
K=\begin{bmatrix}1\amp 0\\ 0\amp 1\\ 1\amp 1\end{bmatrix}.
\end{equation*}
Compute \(K\mathbf{q}\text{.}\)
Explain why the entries of
\(K\mathbf{q}\) are the same scores from part (a).
Normalize the scores by dividing by their sum. What is the weight vector
\(\boldsymbol{\alpha}\text{?}\)
Tags. [U1-LO1, U1-LO4, U1-LO8 | C+R | Core]
Checkpoint 1.6.4 . Rows measure; columns contribute.
Let
\begin{equation*}
A=\begin{bmatrix}-1\amp 4\amp -5\\ 3\amp 1\amp -2\end{bmatrix},
\qquad
\mathbf{x}=\begin{bmatrix}2\\ -3\\ 4\end{bmatrix}.
\end{equation*}
Compute
\(A\mathbf{x}\) using row dot products.
Compute
\(A\mathbf{x}\) as a linear combination of the columns of
\(A\text{.}\)
In this example, what does the first row of
\(A\) measure?
In this example, what do the coordinates
\(2,-3,4\) tell you in the column-combination view?
Tags. [U1-LO4 | P+R | Core]
Checkpoint 1.6.5 . The unit-square visualization.
Let
\begin{equation*}
A=\begin{bmatrix}2\amp -1\\ 1\amp 1\end{bmatrix}.
\end{equation*}
Compute
\(A\mathbf{e}_1\) and
\(A\mathbf{e}_2\text{.}\)
Compute
\(A\begin{bmatrix}1\\1\end{bmatrix}\text{.}\)
Apply \(A\) to the four corners of the unit square:
\begin{equation*}
\begin{bmatrix}0\\0\end{bmatrix},\qquad \begin{bmatrix}1\\0\end{bmatrix},\qquad \begin{bmatrix}1\\1\end{bmatrix},\qquad \begin{bmatrix}0\\1\end{bmatrix}.
\end{equation*}
Sketch the image of the unit square.
Which two transformed edges leaving the origin are the columns of
\(A\text{?}\)
Tags. [U1-LO5, U1-LO8 | C+R | Core]
Checkpoint 1.6.6 . Transformation matching.
Checkpoint 1.6.7 . Composition order: \(RS\) versus \(SR\) .
Let
\begin{equation*}
S=\begin{bmatrix}2\amp 0\\ 0\amp 1\end{bmatrix},
\qquad
R=\begin{bmatrix}0\amp -1\\ 1\amp 0\end{bmatrix}.
\end{equation*}
Are
\(RS\) and
\(SR\) equal?
Which product represents “apply
\(S\) first, then apply
\(R\) ”?
Explain geometrically why the two orders produce different results.
Tags. [U1-LO6 | P+R | Core]
Checkpoint 1.6.8 . Affine but not linear.
Consider the transformation
\begin{equation*}
T\begin{bmatrix}x\\y\end{bmatrix}=\begin{bmatrix}x+1\\y\end{bmatrix}.
\end{equation*}
Compute
\(T(0,0)\text{.}\)
Explain why
\(T\) cannot be written in the form
\(T(\mathbf{x})=A\mathbf{x}\) for any matrix
\(A\text{.}\)
Write \(T\) in the affine form
\begin{equation*}
T(\mathbf{x})=A\mathbf{x}+\mathbf{b}.
\end{equation*}
Is
\(T\) linear, affine but not linear, or neither?
Tags. [U1-LO7 | C+R | Core]
Checkpoint 1.6.9 . Code interpretation: sorting similarity scores.
import numpy as np
doc_names = np.array(["D1", "D2", "D3"])
scores = np.array([1.0, 0.5, 0.0])
ranking = doc_names[np.argsort(scores)[::-1]]
ranking
What is the value of
ranking?
What does
np.argsort(scores) return?
Is this ranking based on direction, distance, or alphabetical order?
Tags. [U1-LO2, U1-LO8 | R+T | Core]
Checkpoint 1.6.10 . Transformer shape check.
