Checkpoint B.7.1. Check Yourself: Axis.
For a two-row, three-column array
A, how many entries are in A.sum(axis=0)? How many are in A.sum(axis=1)?
Section B.7 Reductions, Axes, and Row-Wise Computations
This section helps you read reductions, axes, and entry-by-entry scalar functions in array computations.
A = np.array([[1, 2, 3],
[4, 5, 6]])
A.sum()
A.sum(axis=0) # one sum per column
A.sum(axis=1) # one sum per row
np.sum(A)
S = np.array([[1.0, 2.0, 3.0],
[2.0, 4.0, 1.0]])
S_shifted = S - np.max(S, axis=1, keepdims=True)
e = np.exp(S_shifted)
A = e / e.sum(axis=1, keepdims=True)
Axes and reductions. These commands collapse all entries, one column at a time, or one row at a time.
All entries.
A.sum()
Read as. Sum all entries.
Shape/return. A scalar.
Used for. Totals.
Column sums.
A.sum(axis=0)
Read as. One sum per column.
Shape/return. A one-dimensional array.
Used for. Column summaries.
Row sums.
A.sum(axis=1)
Read as. One sum per row.
Shape/return. A one-dimensional array.
Used for. Row summaries.
Row maximum.
np.max(S, axis=1, keepdims=True)
Read as. Row maxima with the reduced dimension preserved.
Shape/return. A column-shaped array.
Used for. Stable softmax-style code.
Row sum with shape.
e.sum(axis=1, keepdims=True)
Read as. One row sum, with the reduced dimension preserved.
Shape/return. A column-shaped array.
Used for. Row normalization.
Keeping the reduced dimension makes the result line up with the original array for row-by-row subtraction or division. This is why the softmax-style example uses
keepdims=True.
NumPy also applies many familiar scalar functions entry-by-entry to arrays.
x = np.array([-1.0, 0.0, 1.0])
np.exp(x) # entry-by-entry exponential
np.log(x + 2) # entry-by-entry logarithm
np.sqrt(x + 1) # entry-by-entry square root
np.tanh(x) # entry-by-entry hyperbolic tangent
Entrywise functions. These commands apply the same scalar function to each entry of an array.
Exponential.
np.exp(x)
Read as. Entry-by-entry exponential.
Shape/return. An array.
Used for. Softmax.
Logarithm.
np.log(x)
Read as. Entry-by-entry logarithm.
Shape/return. An array.
Used for. Losses and log probabilities.
Square root.
np.sqrt(x)
Read as. Entry-by-entry square root.
Shape/return. An array.
Used for. Norms and scales.
Hyperbolic tangent.
np.tanh(x)
Read as. Entry-by-entry hyperbolic tangent.
Shape/return. An array.
Used for. Nonlinear hidden layer.
Coordinatewise maximum.
np.maximum(z, 0)
Read as. Replace negative entries by 0.
Shape/return. Same shape as
z.
Used for. Coordinatewise ReLU.
Cumulative energy.
np.cumsum(s**2) / np.sum(s**2)
Read as. Cumulative fraction of squared singular-value energy.
Output. A one-dimensional array.
Used for. Choosing how many SVD terms to keep.
These commands do not perform matrix multiplication. They apply the same scalar function to each entry of the array.
The same idea appears in cumulative SVD energy calculations.
