Section 1.7 Unit 1 highlights
Big questions. How do vectors represent data, how do matrices act on vectors, and what is special about linear functions?
Notation. \(\|\mathbf{v}\|\text{,}\) \(\mathbf{u}\cdot \mathbf{v}\text{,}\) \(\operatorname{cosim}(\mathbf{u},\mathbf{v})\text{,}\) \(A\mathbf{x}\text{,}\) \(K\mathbf{q}\text{,}\) \(\boldsymbol{\alpha}^T V\text{,}\) \(S=QK^T\text{,}\) \(A_{\mathrm{att}}\text{,}\) \(H=A_{\mathrm{att}}V\text{,}\) \(T(\mathbf{x})=A\mathbf{x}\text{,}\) \(\mathbf{x}\mapsto W\mathbf{x}+\mathbf{b}\text{.}\)
Concepts. Vector, distance, dot product, cosine similarity, linear combination, convex combination, data vector, word-count vector, token vector, matrix, matrix-vector product, linear map, affine map.
Results. Dot products connect length, angle, orthogonality, and cosine similarity. The product \(A\mathbf{x}\) can be read as row dot products or as a linear combination of columns. The columns of a \(2\times 2\) matrix show where \(\mathbf{e}_1\) and \(\mathbf{e}_2\) go. Every linear map between Euclidean spaces is represented by a matrix. An affine map \(\mathbf{x}\mapsto A\mathbf{x}+\mathbf{b}\) is linear exactly when \(\mathbf{b}=\mathbf{0}\text{.}\)
Main applications. Vector representations of colors, time series, images, features, customers, documents, and tokens; document ranking by cosine similarity; attention as token matrices, dot-product scores, normalized weights, and weighted averages; geometric matrix actions; composition of linear maps; affine layers.
Connections. Unit 2 uses vectors and matrices to describe reachable outputs and forgotten directions through systems. Unit 3 reuses the unit-square visualization as square-grid visualizations and uses affine approximations locally. Least squares in Unit 4 and gradients in Unit 5 both reuse dot products.
Study anchors. U1-LO1 and U1-LO2: Vectors. U1-LO3: Matrices. U1-LO4 and U1-LO5: Matrix-vector product and linear maps. U1-LO6: Matrix multiplication and map composition. U1-LO7: Linear and affine maps. U1-LO8: Applications and computation recap and Linked notebook.
Applications and computations readiness checklist. Given a short Unit 1 calculation, code snippet, table, or diagram, I can identify whether
@ is a dot product, matrix-vector product, or matrix product; decide whether a ranking is based on cosine similarity or distance; read \(K\mathbf{q}\) as token-query scores; read \(\boldsymbol{\alpha}^T V\) as a weighted average; explain the columns of \(X\) and \(Y\) in \(Y=AX\text{;}\) use the unit-square visualization for a \(2\times 2\) matrix; check dimensions before multiplying; and decide whether \(\mathbf{x}\mapsto W\mathbf{x}+\mathbf{b}\) is linear or affine.
Common mistakes. Confusing dot product with distance, forgetting to normalize cosine similarity, mixing up rows and columns, multiplying matrices in the wrong order, or calling an affine map linear when the bias is nonzero.
