2.2. Linear Combinations

Definition 2.2.1

Let \(\mathbf{v}_1, \ldots, \mathbf{v}_n\) be vectors in \(\mathbb{R}^m\). Any expression of the form

\[ x_1 \mathbf{v_1}+\cdots+x_n \mathbf{v_n}, \]

where \(x_1, \ldots, x_n\) are real numbers, is called a linear combination of the vectors \(\mathbf{v}_1, \ldots, \mathbf{v}_n\).

Example 2.2.1

The vectors \(\mathbf{v}_1\) and \(\mathbf{v}_2\) are two vectors in the plane \(\mathbb{R}^2\). As we can see in Figure 2.2.1, the vector \(\mathbf{u}\) is a linear combination of \(\mathbf{v}_1\) and \(\mathbf{v}_2\) since it can be written as \(\mathbf{u}=2\mathbf{v}_1+\mathbf{v}_2\). The vector \(\mathbf{w}\) is a linear combination of these two vectors as well. It can be written as \(\mathbf{w}=-3\mathbf{v}_1+2\mathbf{v}_2\).

Fig. 2.2.1 Linear combinations of vectors in the plane.

If we want to determine whether a given vector is a linear combination of other vectors, then we can do that using systems of equations.

Example 2.2.2

\[\begin{split} \mathbf{v_1}= \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix} \quad \mathbf{v_2}= \begin{bmatrix} 3 \\ 1 \\ 2 \end{bmatrix} \quad \mathbf{b}= \begin{bmatrix} -1 \\ 3 \\ 0 \end{bmatrix} \end{split}\]

Is the vector \(\mathbf{b}\) a linear combination of \(\mathbf{v}_1\) and \(\mathbf{v}_2\)? We can use the definition of a linear combination to solve this problem. If \(\mathbf{b}\) is in fact a linear combination of the two other vectors, then it can be written as \(x_1 \mathbf{v}_1+x_2 \mathbf{v}_2\). This means that we should verify whether the system of equations \(x_1 \mathbf{v}_1+x_2 \mathbf{v}_2=\mathbf{b}\) has a solution.

The equation

\[\begin{split} x_1 \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}+x_2 \begin{bmatrix} 3 \\ 1 \\ 2 \end{bmatrix}= \begin{bmatrix} -1 \\ 3 \\ 0 \end{bmatrix} \end{split}\]

is equivalent to the system

\[\begin{split} \left\{\begin{array}{l} x_1+3x_2=-1 \\ 2x_1+x_2=3 \\ x_1+2x_2=0\end{array} \right. \end{split}\]

The augmented matrix of this system of equations is equal to

\[\begin{split} \left[\begin{array}{cc|c} 1 & 3 & -1 \\ 2 & 1 & 3 \\ 1 & 2 & 0 \end{array}\right] \end{split}\]

and its reduced echelon form is equal to

\[\begin{split} \left[\begin{array}{cc|c} 1 & 0 & 2 \\ 0 & 1 & -1 \\ 0 & 0 & 0 \end{array}\right]. \end{split}\]

This means that \(\mathbf{b}\) is indeed a linear combination of \(\mathbf{v}_1\) and \(\mathbf{v}_2\).

\[\begin{split} 2 \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}- \begin{bmatrix} 3 \\ 1 \\ 2 \end{bmatrix}= \begin{bmatrix} -1 \\ 3 \\ 0 \end{bmatrix} \end{split}\]

We have found that \(\mathbf{b}\) can be written as \(2\mathbf{v}_1-\mathbf{v_2}\).

Example 2.2.3

\[\begin{split} \mathbf{v_1}= \begin{bmatrix} 1 \\ 0 \\ 2 \end{bmatrix} \quad \mathbf{v_2}= \begin{bmatrix} 3 \\ 0 \\ 1 \end{bmatrix} \quad \mathbf{b}= \begin{bmatrix} 2 \\ 1 \\ 1 \end{bmatrix} \end{split}\]

In this case it is a lot easier to decide whether \(\mathbf{b}\) is a linear combination of \(\mathbf{v}_1\) and \(\mathbf{v}_2\). Since the second component of both \(\mathbf{v}_1\) and \(\mathbf{v}_2\) is equal to zero, we know that the second component of each linear combination of those vectors will be zero. This means that \(\mathbf{b}\) can never be a linear combination of \(\mathbf{v}_1\) and \(\mathbf{v}_2\).

Grasple Exercise 2.2.1

Expressing a vector as a linear combination of other vectors

Show/Hide Content

Grasple Exercise 2.2.2

Expressing a vector as a linear combination of other vectors

Show/Hide Content

2.2.1. Span

In linear algebra it is often important to know whether each vector in \(\mathbb{R}^n\) can be written as a linear combination of a set of given vectors. In order to investigate when it is possible to write any given vector as a linear combination of a set of given vectors we introduce the notion of a span.

Definition 2.2.2

Let \(S\) be a set of vectors. The set of all linear combinations \(a_1\mathbf{v}_1+a_2\mathbf{v}_2+ \cdots +a_k \mathbf{v}_k\), where \(\mathbf{v}_1, \ldots, \mathbf{v}_k\) are vectors in \(S\), will be called the span of those vectors and will be denoted as \(\Span{S}\).

When \(S\) is equal to a finite set \(\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}\), then we will simply write \(\Span{\mathbf{v}_1, \ldots, \mathbf{v}_k}\).

The span of an empty collection of vectors will be defined as the set that only contains the zero vector \(\mathbf{0}\).

Remark 2.2.1

The collection \(\Span{\mathbf{v}_1, \ldots, \mathbf{v}_k}\) always contains all of the vectors \(\mathbf{v}_1, \ldots, \mathbf{v}_k\). This is true since each vector \(\mathbf{v}_i\) can be written as the linear combination

\[ 0\mathbf{v}_1+\cdots+1\mathbf{v}_i+\cdots +0\mathbf{v}_k. \]

Moreover, the span of any set of vectors always contains the zero vector. Whatever set of vectors we start with, we can always write

\[ \mathbf{0}=0\mathbf{v}_1+0\mathbf{v}_2+\cdots +0\mathbf{v}_k. \]

The following examples will give us a bit of an idea what spans look like.

Example 2.2.4

What does the span of a single non-zero vector look like? A linear combination of a vector \(\mathbf{v}\) is of the form \(x\mathbf{v}\), where \(x\) is some real number. Linear combinations of a single vector \(\mathbf{v}\) are thus just multiples of that vector. This means that \(\Span{\mathbf{v}}\) is simply the collection of all vectors on the line through the origin and with directional vector \(\mathbf{v}\) as we can see in Figure 2.2.2.

Fig. 2.2.2 The span of a single non-zero vector.

Example 2.2.5

Let \(\mathbf{u}\) and \(\mathbf{v}\) be two non-zero vectors in \(\mathbb{R}^3\), as depicted in Figure 2.2.3. What does the span of these vectors look like? By definition, \(\Span{\mathbf{u}, \mathbf{v}}\) contains all linear combinations of \(\mathbf{u}\) and \(\mathbf{v}\). Each of these linear combinations is of the form

\[ x_1\mathbf{u}+x_2\mathbf{v} \quad \textrm{$x_1$, $x_2$ in $\mathbb{R}$}. \]

This looks like the parametric vector equation of a plane. Since the span must contain the zero vector we find that we obtain a plane through the origin like in Figure 2.2.3.

Fig. 2.2.3 The span of two non-zero, non-parallel vectors.

Example 2.2.6

The span of two non-zero vectors does not need to be a plane through the origin. If \(\mathbf{u}\) and \(\mathbf{v}\) are parallel, as in Figure 2.2.4, then the span is actually a line through the origin.

Fig. 2.2.4 The span of two non-zero, parallel vectors.

If two non-zero vectors \(\mathbf{u}\) and \(\mathbf{v}\) are parallel, then \(\mathbf{v}\) can be written as a multiple of \(\mathbf{u}\). Assume for example that \(\mathbf{v}=2\mathbf{u}\). Any linear combination \(x_1\mathbf{u}+x_2\mathbf{v}\) can then be written as \(x_1\mathbf{u}+2x_2\mathbf{u}\) or \((x_1+2x_2)\mathbf{u}\). This means that in this case each vector in the span of \(\mathbf{u}\) and \(\mathbf{v}\) is a multiple of \(\mathbf{u}\). Therefore, the span will be a line through the origin.

Example 2.2.7

If we start with three non-zero vectors in \(\mathbb{R}^3\), then the resulting span may take on different forms. The span of the three vectors in Figure 2.2.5, for example, is equal to the entire space \(\mathbb{R}^3\). In Section 4.2 we will see why this is the case.

Fig. 2.2.5 The span of three vectors.

On the other hand, if we start with the three vectors that you can see in Figure 2.2.6, then the span is equal to a plane through the origin.

Fig. 2.2.6 The span of three vectors lying in the same plane.

There is also a possibility where the span of three non-zero vectors in \(\mathbb{R}^3\) is equal to a line through the origin. Can you figure out when this happens?

Grasple Exercise 2.2.3

Interpretation of Span\(\{\vect{v}_1,\vect{v_2},\vect{v}_3\}\).

Show/Hide Content

We will now look at a very specific set of vectors in \(\mathbb{R}^n\) of which the span is always the entire space \(\mathbb{R}^n\).

Definition 2.2.3

Suppose we are working in \(\mathbb{R}^n\). Let \(\mathbf{e}_k\) be the vector of which all components are equal to 0, with the exception that the entry on place \(k\) is equal to 1. The vectors \((\mathbf{e}_1, \ldots, \mathbf{e}_n)\) will be called the standard basis of \(\mathbb{R}^n\).

Example 2.2.8

The following vectors form the standard basis for \(\mathbb{R}^2\).

\[\begin{split} \mathbf{e}_1= \begin{bmatrix} 1 \\ 0 \end{bmatrix} \quad \mathbf{e}_2= \begin{bmatrix} 0 \\ 1 \end{bmatrix} \nonumber \end{split}\]

Each vector \(\mathbf{v}\) can be written as a linear combination of the vectors \(\mathbf{e}_1\) and \(\mathbf{e}_2\) in a unique way. Later on we will call each collection of vectors with this property a basis for \(\mathbb{R}^2\). If

\[\begin{split} \mathbf{v}= \begin{bmatrix} a \\ b \end{bmatrix}, \nonumber \end{split}\]

then clearly we have that

\[\begin{split} \mathbf{v}=a \begin{bmatrix} 1 \\ 0 \end{bmatrix}+b \begin{bmatrix} 0 \\ 1 \end{bmatrix}. \nonumber \end{split}\]

It is easy to see that this is the only linear combination of \(\mathbf{e}_1\) and \(\mathbf{e}_2\) that is equal to \(\mathbf{v}\).

Example 2.2.9

The three vectors below form the standard basis for \(\mathbb{R}^3\).

\[\begin{split} \mathbf{e}_1= \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \quad \mathbf{e}_2= \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \quad \mathbf{e}_3= \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \nonumber \end{split}\]

Here too, it is true that each vector in \(\mathbb{R}^3\) can be written as a unique linear combination of these three vectors.

Proposition 2.2.1

If \((\mathbf{e}_1, \ldots, \mathbf{e}_n)\) is the standard basis for \(\mathbb{R}^n\), then \(\Span{\mathbf{e}_1, \ldots, \mathbf{e}_n}\) is equal to \(\mathbb{R}^n\).

Proof. Take an arbitrary vector \(\mathbf{v}\) in \(\mathbb{R}^n\) with

\[\begin{split} \mathbf{v}= \begin{bmatrix} a_1 \\ \vdots \\ a_n \end{bmatrix}.\nonumber \end{split}\]

The vector \(\mathbf{v}\) can be written as

\[\begin{align*} \mathbf{v} &= a_1 \begin{bmatrix} 1 \\ 0 \\ \vdots \\ 0 \end{bmatrix}+a_2 \begin{bmatrix} 0 \\ 1 \\ \vdots \\ 0 \end{bmatrix}+ \ldots a_n \begin{bmatrix} 0 \\ 0 \\ \vdots \\ 1 \end{bmatrix} \\ &= a_n\mathbf{e}\_1+a_2\mathbf{v}\_2+\ldots +a_n\mathbf{e}\_n. \end{align*}\]

This means that \(\mathbf{v}\) is in the span of \(\mathbf{e}_1, \ldots, \mathbf{e}_n\).

On the other hand, each vector in \(\Span{\mathbf{e}_1, \ldots, \mathbf{e}_n}\) is a linear combination of vectors in \(\mathbb{R}^n\) and thus itself a vector in \(\mathbb{R}^n\).

In Proposition 2.2.1 we saw that the span of the standard basis of \(\mathbb{R}^n\) is equal to the entire space. In Section 2.4, we will find out when, for an arbitrary set of vectors \(\mathbf{v}_1, \ldots, \mathbf{v}_k\), the collection \(\Span{\mathbf{v}_1, \ldots, \mathbf{v}_k}\) contains every vector in \(\mathbb{R}^n\).

2.2.2. Grasple Exercises

Grasple Exercise 2.2.4

Is \(\vect{b}\) an element of Span\(\{\vect{a}_1,\vect{a}_2,\vect{a}_3\}\)?

Show/Hide Content

Grasple Exercise 2.2.5

Is \(\vect{b}\) an element of Span\(\{\vect{a}_1,\vect{a}_2,\vect{a}_3\}\)?

Show/Hide Content

Grasple Exercise 2.2.6

Generate your own linear combinations

Show/Hide Content

Grasple Exercise 2.2.7

Is the span of two vectors always a plane?

Show/Hide Content

Grasple Exercise 2.2.8

Checking whether a vector is in the span of other vectors.

Show/Hide Content

Grasple Exercise 2.2.9

About the interpretation of Span\(\{\vect{a}_1,\vect{a}_2\}\).

Show/Hide Content

Grasple Exercise 2.2.10

Checking whether a vector is a linear combination of the columns of a matrix \(A\).

Show/Hide Content

Grasple Exercise 2.2.11

About the difference between \(\{\vect{a}_1,\vect{a}_2,\vect{a}_3\}\) and Span\(\{\vect{a}_1,\vect{a}_2,\vect{a}_3\}\).

Show/Hide Content

Grasple Exercise 2.2.12

When do the columns of a matrix span all of \(\R^m\)?

Show/Hide Content

Grasple Exercise 2.2.13

About removing vectors without reducing the span.

Show/Hide Content

Grasple Exercise 2.2.14

Conversion between vector equation and linear system.

Show/Hide Content