Basic matrix arithmetic

14. Basic matrix arithmetic#

An \(m\times n\)-matrix is an array of \(m\) rows and \(n\) columns containing (usually real or complex) numbers called the entries (or coefficients) of the matrix. The entries of a matrix \(A\) are denoted by \(A_{i,j}\), \(a_{ij}\) or similar, where the first index refers to the row and the second to the column. That is, the entries are laid out like

(14.1)#\[\begin{split} A = \begin{pmatrix} a_{1,1} & a_{1,2} & \cdots & a_{1,n}\\ a_{2,1} & a_{2,2} & \cdots & a_{2,n}\\ \vdots& \vdots& \ddots & \vdots\\ a_{m,1} & a_{m,2} & \cdots & a_{m,n} \end{pmatrix}. \end{split}\]

You will often see notation like \(A=(a_{i,j})\) or \(A=(a_{i,j})_{1\le i\le m\atop 1\le j\le n}\) to indicate how the entries are denoted and/or how many rows and columns the matrix has.

In this section we focus on the basic operations that can be performed with matrices.

14.1. Addition and scalar multiplication#

Like vectors, matrices can be added or subtracted entrywise. They can also be multiplied entrywise by a given scalar. For example,

\[\begin{split} 4\begin{pmatrix}1& 0\\ 3& -1\end{pmatrix} - 2\begin{pmatrix}0& 2\\ 1& -1\end{pmatrix} = \begin{pmatrix}4& 0\\ 12& -4\end{pmatrix} - \begin{pmatrix}0& 4\\ 2& -2\end{pmatrix} = \begin{pmatrix}4& -4\\ 10& -2\end{pmatrix}. \end{split}\]

14.2. Special matrices#

A few (types of) matrices occur very often and have their own names:

  • the \(m\times n\) zero matrix \(\begin{pmatrix} 0 & 0 & \cdots & 0\\ 0 & 0 & \cdots & 0\\ \vdots& \vdots& \ddots & \vdots\\ 0 & 0 & \cdots & 0 \end{pmatrix}\)

  • the \(n\times n\) identity matrix \(\begin{pmatrix} 1 & 0 & \cdots & 0\\ 0 & 1 & \cdots & 0\\ \vdots& \vdots& \ddots & \vdots\\ 0 & 0 & \cdots & 1 \end{pmatrix}\)

  • diagonal matrices \(\begin{pmatrix} a_{1,1} & 0 & \cdots & 0\\ 0 & a_{2,2} & \cdots & 0\\ \vdots& \vdots& \ddots & \vdots\\ 0 & 0 & \cdots & a_{n,n} \end{pmatrix}\)

  • upper triangular matrices \(\begin{pmatrix} a_{1,1} & a_{1,2} & \cdots & a_{1,n}\\ 0 & a_{2,2} & \cdots & a_{2,n}\\ \vdots& \ddots& \ddots & \vdots\\ 0 & \cdots & 0 & a_{n,n} \end{pmatrix}\)

  • lower triangular matrices \(\begin{pmatrix} a_{1,1} & 0 & \cdots & 0\\ a_{2,1} & a_{2,2} & \ddots & \vdots\\ \vdots& \vdots& \ddots & 0\\ a_{n,1} & a_{n,2} & \cdots & a_{n,n} \end{pmatrix}\)

14.3. Inner product of vectors#

Definition 14.1

Given two vectors \(\vv=(v_1,\ldots,v_n)\) and \(\vw=(w_1,\ldots,w_n)\), the inner product of \(\vv\) and \(\vw\) is the scalar

\[ \vv\cdot\vw = v_1 w_1 + \cdots + v_n w_n. \]

Property 14.1 (Properties of the inner product)

For vectors \(\vv\), \(\vw\), \(\vx\) (all of the same length) and scalars \(c\) the following hold:

  • \(\vv\cdot(\vw+\vx) = \vv\cdot\vw + \vv\cdot\vx\)

  • \((\vv+\vw)\cdot\vx = \vv\cdot\vx + \vw\cdot\vx\)

  • \(\vv\cdot(c\vw) = c(\vv\cdot\vw)=(c\vv)\cdot\vw\)

  • \(\vv\cdot\vw = \vw\cdot\vv\)

14.4. Matrix-vector multiplication#

Matrices can be multiplied by vectors.

Definition 14.2

Given a matrix \(A\) as in (14.1) and a vector

\[\begin{split} \vv = \begin{pmatrix}v_1\\ \vdots\\ v_n\end{pmatrix} \in\mathbb{R}^n \end{split}\]

the (matrix-vector) product of \(A\) and \(\vv\) is the vector

\[\begin{split} A\vv=\begin{pmatrix}a_{1,1}v_1 + \cdots + a_{1,n}v_n\\ \vdots\\ a_{m,1}v_1 + \cdots + a_{m,n}v_n\end{pmatrix} \in\mathbb{R}^m. \end{split}\]

Equivalently, we can write

(14.2)#\[ (A\vv)_i = \sum_{j=1}^n A_{i,j} v_j\quad(1\le i\le m). \]

Property 14.2 (Properties of matrix-vector multiplication)

For matrices \(A\) and \(B\), vectors \(\vv\) and \(\vw\) and scalars \(c\) the following hold whenever the expressions are defined:

  • \(A(\vv+\vw)= A\vv + A\vw\)

  • \(A(c\vv) = c(A\vv)\)

  • \((A+B)\vv = A\vv+B\vv\)

  • \((cA)\vv = c(A\vv)\)

Note

Here are two other useful ways to think about matrix-vector products.

First, in terms of inner products (see Definition 14.1): the vector \(A\vv\) consists of the inner products of each of the rows of \(A\) with the vector \(\vv\). Namely, if

\[\begin{split} A = \begin{pmatrix}\va_1\\ \vdots\\ \va_m\end{pmatrix}, \end{split}\]

then

\[\begin{split} A\vv = \begin{pmatrix}\va_1\cdot\vv\\ \vdots\\ \va_m\cdot\vv\end{pmatrix}. \end{split}\]

Second, in terms of linear combinations: the vector \(A\vv\) is a linear combination of the columns of \(A\) where the coefficients are the entries of \(\vv\). Namely, if

\[\begin{split} A = (\va_1\ \va_2\ \cdots\ \va_n)\quad\text{and}\quad \vv = \begin{pmatrix}v_1\\ \vdots\\ v_n\end{pmatrix}, \end{split}\]

then

\[ A\vv = v_1 \va_1 + v_2 \va_2 + \cdots + v_n \va_m. \]

14.5. Matrix multiplication#

Two matrices \(A\) and \(B\) can be multiplied if the number of columns of \(A\) equals the number of rows of \(B\).

Definition 14.3

If \(A\) is an \(m\times n\)-matrix and \(B\) is an \(n\times p\)-matrix, then the (matrix) product of \(A\) and \(B\) is the \(m\times p\)-matrix \(AB\) defined as follows:

\[ (AB)_{i,k} = \sum_{j=1}^n A_{i,j} B_{j,k} \quad(1\le i\le m,1\le k\le p). \]

The product \(AB\) can be viewed as \(p\) matrix-vector multiplications: if \(B=(\vb_1\ \vb_2\ \cdots \vb_p)\), then

\[ AB = (A\vb_1\ A\vb_2\ \cdots A\vb_p). \]

Property 14.3 (Properties of matrix multiplication)

For matrices \(A\), \(B\) and \(C\) and scalars \(c\) the following hold whenever the expressions are defined:

  • \(A(B + C)= AB + AC\)

  • \((A + B)C = AC + BC\)

  • \((AB)C = A(BC)\)

  • \((cA)B = c(AB) = A(cB)\)

Warning

Matrix multiplication is not commutative! That is, in general we have

\[AB\ne BA.\]
Contents