$$ \mathbf{A}\mathbf{x}

\begin{bmatrix} \uparrow & & \uparrow \\ \mathbf{a}_1 & \cdots & \mathbf{a}_n \\ \downarrow & & \downarrow \\ \end{bmatrix}

\left\{ \begin{matrix} x_1 \\ \vdots \\ x_n \end{matrix} \right\}

x_1\mathbf{a}_1 + \dots+x_n\mathbf{a}_n $$

Column Space, $C(\mathbf{A})$ - all possible linear combinations of independent columns.

Column Rank, $\text{rank}(\mathbf{A})$ or $\text{dim }C(\mathbf{A})$ - number of independent columns, or dimension of column space.

Linear Independence

$$ \sum_{i=1}^{n} k_i\mathbf{a}_i = \mathbf{0} \iff k_1 = \dots = k_n = 0 $$

Equal Ranks

$$ \text{Column Rank} = \text{Row Rank} \\ \text{dim }C(\mathbf{A}) = \text{dim }C(\mathbf{A^T}) \\ C(A) \subseteq \mathbb{R}^m,\; C(A^T) \subseteq \mathbb{R}^n $$

Row-Reduced Echelon Form, $\mathbf{R}=\text{rref}(\mathbf{A})$

$$ \mathbf{A} = \mathbf{C}\mathbf{R} $$

where $\mathbf{C}$ contains the independent columns of $\mathbf{A}$,

and $\mathbf{R}$ contains the derived independent rows of $\mathbf{A}$.

<aside> 💡

The row-reduced echelon form can be used to show that $\mathbf{A}$ contains equal ranks because $\mathbf{C}$ is $(m \times r)$ and $\mathbf{R}$ is $(r \times n)$, where $r$ is the rank of $\mathbf{A}$.

</aside>

Section I.2

Matrix-Matrix Multiplication - sum of rank one matrices.

$$ \mathbf{A}\mathbf{B}

\begin{bmatrix} \uparrow & & \uparrow \\ \mathbf{a}_1 & \cdots & \mathbf{a}_n \\ \downarrow & & \downarrow \\ \end{bmatrix} \begin{bmatrix} \leftarrow & \mathbf{b}_1^* & \rightarrow \\ & \vdots & \\ \leftarrow & \mathbf{b}_n^* & \rightarrow \\ \end{bmatrix}

\mathbf{a}_1\mathbf{b}_1^* + \dots + \mathbf{a}_n\mathbf{b}_n^* $$

<aside> 💡

If $\mathbf{A}$ is $(m \times n)$ and $\mathbf{B}$ is $(n \times p)$, then the total number of element-wise multiplications will be $mnp$.

</aside>