[latexpage]

Introduction

In the study of systems of linear equations in Chapter 1, we found it convenient to manipulate the augmented matrix of the system. Our aim was to reduce it to row-echelon form (using elementary row operations) and hence to write down all solutions to the system. In the present chapter we consider matrices for their own sake. While some of the motivation comes from linear equations, it turns out that matrices can be multiplied and added and so form an algebraic system somewhat analogous to the real numbers. This “matrix algebra” is useful in ways that are quite different from the study of linear equations. For example, the geometrical transformations obtained by rotating the euclidean plane about the origin can be viewed as multiplications by certain $2 \times 2$ matrices. These “matrix transformations” are an important tool in geometry and, in turn, the geometry provides a “picture” of the matrices. Furthermore, matrix algebra has many other applications, some of which will be explored in this chapter. This subject is quite old and was first studied systematically in 1858 by Arthur Cayley.

2.1 Matrix Addition, Scalar Multiplication, and Transposition

A rectangular array of numbers is called a matrix (the plural is matrices), and the numbers are called the entries of the matrix. Matrices are usually denoted by uppercase letters: $A$, $B$, $C$, and so on. Hence,
\begin{equation*}
A = \left[ \begin{array}{rrr}
1 & 2 & -1 \\
0 & 5 & 6
\end{array} \right] \quad
B = \left[ \begin{array}{rr}
1 & -1 \\
0 & 2
\end{array} \right] \quad
C = \left[ \begin{array}{r}
1 \\
3 \\
2
\end{array} \right]
\end{equation*}
are matrices. Clearly matrices come in various shapes depending on the number of rows and columns. For example, the matrix $A$ shown has $2$ rows and $3$ columns. In general, a matrix with $m$ rows and $n$ columns is referred to as an $\bm{m} \times \bm{n}$ matrix or as having size $\bm{m} \times \bm{n}$. Thus matrices $A$, $B$, and $C$ above have sizes $2 \times 3$, $2 \times 2$, and $3 \times 1$, respectively. A matrix of size $1 \times n$ is called a row matrix, whereas one of size $m \times 1$ is called a column matrix. Matrices of size $n \times n$ for some $n$ are called square matrices.

Each entry of a matrix is identified by the row and column in which it lies. The rows are numbered from the top down, and the columns are numbered from left to right. Then the $\bm{(i, j)}$-entry of a matrix is the number lying simultaneously in row $i$ and column $j$. For example,
\begin{align*}
\mbox{The } (1, 2) \mbox{-entry of } &\left[ \begin{array}{rr}
1 & -1 \\
0 & 1
\end{array}\right] \mbox{ is } -1. \\
\mbox{The } (2, 3) \mbox{-entry of } &\left[ \begin{array}{rrr}
1 & 2 & -1 \\
0 & 5 & 6
\end{array}\right] \mbox{ is } 6.
\end{align*}

A special notation is commonly used for the entries of a matrix. If $A$ is an $m \times n$ matrix, and if the $(i, j)$-entry of $A$ is denoted as $a_{ij}$, then $A$ is displayed as follows:
\begin{equation*}
A = \left[ \begin{array}{ccccc}
a_{11} & a_{12} & a_{13} & \cdots & a_{1n} \\
a_{21} & a_{22} & a_{23} & \cdots & a_{2n} \\
\vdots & \vdots & \vdots & & \vdots \\
a_{m1} & a_{m2} & a_{m3} & \cdots & a_{mn}
\end{array} \right]
\end{equation*}
This is usually denoted simply as $A = \left[ a_{ij} \right]$. Thus $a_{ij}$ is the entry in row $i$ and column $j$ of $A$. For example, a $3 \times 4$ matrix in this notation is written
\begin{equation*}
A = \left[ \begin{array}{cccc}
a_{11} & a_{12} & a_{13} & a_{14} \\
a_{21} & a_{22} & a_{23} & a_{24} \\
a_{31} & a_{32} & a_{33} & a_{34}
\end{array} \right]
\end{equation*}
It is worth pointing out a convention regarding rows and columns: Rows are mentioned before columns. For example:

• If a matrix has size $m \times n$, it has $m$ rows and $n$ columns.
• If we speak of the $(i, j)$-entry of a matrix, it lies in row $i$ and column $j$.
• If an entry is denoted $a_{ij}$, the first subscript $i$ refers to the row and the second subscript $j$ to the column in which $a_{ij}$ lies.

Two points $(x_{1}, y_{1})$ and $(x_{2}, y_{2})$ in the plane are equal if and only if they have the same coordinates, that is $x_{1} = x_{2}$ and $y_{1} = y_{2}$. Similarly, two matrices $A$ and $B$ are called equal (written $A = B$) if and only if:

1. They have the same size.
2. Corresponding entries are equal.

If the entries of $A$ and $B$ are written in the form $A = \left[ a_{ij} \right]$, $B = \left[ b_{ij} \right]$, described earlier, then the second condition takes the following form:
\begin{equation*}
A = \left[ a_{ij} \right] = \left[ b_{ij} \right] \mbox{ means } a_{ij} = b_{ij} \mbox{ for all } i \mbox{ and } j
\end{equation*}

Solution:

$A = B$ is impossible because $A$ and $B$ are of different sizes: $A$ is $2 \times 2$ whereas $B$ is $2 \times 3$. Similarly, $B = C$ is impossible. But $A = C$ is possible provided that corresponding entries are equal:
$\left[ \begin{array}{cc}
a & b \\
c & d
\end{array} \right] =
\left[ \begin{array}{rr}
1 & 0 \\
-1 & 2
\end{array} \right]$
means $a = 1$, $b = 0$, $c = -1$, and $d = 2$.

Matrix Addition

If $A = \left[ a_{ij} \right]$ and $B = \left[ b_{ij} \right]$, this takes the form
\begin{equation*}
A + B = \left[ a_{ij} + b_{ij} \right]
\end{equation*}
Note that addition is not defined for matrices of different sizes.

Solution:

\begin{equation*}
A + B = \left[ \begin{array}{rrr}
2 + 1 & 1 + 1 & 3 – 1 \\
-1 + 2 & 2 + 0 & 0 + 6
\end{array} \right]
=
\left[ \begin{array}{rrr}
3 & 2 & 2 \\
1 & 2 & 6
\end{array} \right]
\end{equation*}

Solution:

Add the matrices on the left side to obtain
\begin{equation*}
\left[ \begin{array}{ccc}
a + c & b + a & c + b
\end{array} \right]
=
\left[ \begin{array}{rrr}
3 & 2 & -1
\end{array} \right]
\end{equation*}
Because corresponding entries must be equal, this gives three equations: $a + c = 3$, $b + a = 2$, and $c + b = -1$. Solving these yields $a = 3$, $b = -1$, $c = 0$.

If $A$, $B$, and $C$ are any matrices of the same size, then
\begin{align}
A + B &= B + A \tag{commutative law} \\
A + (B + C) &= (A + B) + C \tag{associative law}
\end{align}
In fact, if $A = \left[ a_{ij} \right]$ and $B = \left[ b_{ij} \right]$, then the $(i, j)$-entries of $A + B$ and $B + A$ are, respectively, $a_{ij} + b_{ij}$ and $b_{ij} + a_{ij}$. Since these are equal for all $i$ and $j$, we get
\begin{equation*}
A + B = \left[ \begin{array}{c}
a_{ij} + b_{ij}
\end{array} \right]
=
\left[ \begin{array}{c}
b_{ij} + a_{ij}
\end{array} \right]
= B + A
\end{equation*}
The associative law is verified similarly.

The $m \times n$ matrix in which every entry is zero is called the $m \times n$ zero matrix and is denoted as $0$ (or $0_{mn}$ if it is important to emphasize the size). Hence,
\begin{equation*}
0 + X = X
\end{equation*}
holds for all $m \times n$ matrices $X$. The negative of an $m \times n$ matrix $A$ (written $-A$) is defined to be the $m \times n$ matrix obtained by multiplying each entry of $A$ by $-1$. If $A = \left[ a_{ij} \right]$, this becomes $-A = \left[ -a_{ij} \right]$. Hence,
\begin{equation*}
A + (-A) = 0
\end{equation*}
holds for all matrices $A$ where, of course, $0$ is the zero matrix of the same size as $A$.

A closely related notion is that of subtracting matrices. If $A$ and $B$ are two $m \times n$ matrices, their difference $A – B$ is defined by
\begin{equation*}
A – B = A + (-B)
\end{equation*}
Note that if $A = \left[ a_{ij} \right]$ and $B = \left[ b_{ij} \right]$, then
\begin{equation*}
A – B = \left[ a_{ij} \right] + \left[ -b_{ij} \right] = \left[ a_{ij} – b_{ij} \right]
\end{equation*}
is the $m \times n$ matrix formed by subtracting corresponding entries.

Solution:

\begin{align*}
-A &= \left[ \begin{array}{rrr}
-3 & 1 & 0 \\
-1 & -2 & 4
\end{array} \right] \\
A – B &= \left[ \begin{array}{lcr}
3 – 1 & -1 – (-1) & 0 – 1 \\
1 – (-2) & 2 – 0 & -4 – 6
\end{array} \right] = \left[ \begin{array}{rrr}
2 & 0 & -1 \\
3 & 2 & -10
\end{array} \right] \\
A + B – C &= \left[ \begin{array}{rrl}
3 + 1 – 1 & -1 – 1 – 0 & 0 + 1 -(-2) \\
1 – 2 – 3 & 2 + 0 – 1 & -4 + 6 -1
\end{array} \right] = \left[ \begin{array}{rrr}
3 & -2 & 3 \\
-4 & 1 & 1
\end{array} \right]
\end{align*}

We solve a numerical equation $a + x = b$ by subtracting the number $a$ from both sides to obtain $x = b – a$. This also works for matrices. To solve
$\left[ \begin{array}{rr}
3 & 2 \\
-1 & 1
\end{array} \right] + X = \left[ \begin{array}{rr}
1 & 0 \\
-1 & 2
\end{array} \right]$
simply subtract the matrix
$\left[ \begin{array}{rr}
3 & 2 \\
-1 & 1
\end{array} \right]$
from both sides to get
\begin{equation*}
X = \left[ \begin{array}{rr}
1 & 0 \\
-1 & 2
\end{array} \right]- \left[ \begin{array}{rr}
3 & 2 \\
-1 & 1
\end{array} \right] = \left[ \begin{array}{cr}
1 – 3 & 0 – 2 \\
-1 – (-1) & 2 – 1
\end{array} \right] = \left[ \begin{array}{rr}
-2 & -2 \\
0 & 1
\end{array} \right]
\end{equation*}
The reader should verify that this matrix $X$ does indeed satisfy the original equation.

The solution in Example 2.1.5 solves the single matrix equation $A + X = B$ directly via matrix subtraction: $X = B – A$. This ability to work with matrices as entities lies at the heart of matrix algebra.

It is important to note that the sizes of matrices involved in some calculations are often determined by the context. For example, if
\begin{equation*}
A + C = \left[ \begin{array}{rrr}
1 & 3 & -1 \\
2 & 0 & 1
\end{array} \right]
\end{equation*}
then $A$ and $C$ must be the same size (so that $A + C$ makes sense), and that size must be $2 \times 3$ (so that the sum is $2 \times 3$). For simplicity we shall often omit reference to such facts when they are clear from the context.

Scalar Multiplication

In gaussian elimination, multiplying a row of a matrix by a number $k$ means multiplying every entry of that row by $k$.

The term scalar arises here because the set of numbers from which the entries are drawn is usually referred to as the set of scalars. We have been using real numbers as scalars, but we could equally well have been using complex numbers.

Solution:

\begin{align*}
5A &= \left[ \begin{array}{rrr}
15 & -5 & 20 \\
10 & 0 & 30
\end{array} \right], \quad \frac{1}{2}B = \left[ \begin{array}{rrr}
\frac{1}{2} & 1 & -\frac{1}{2} \\
0 & \frac{3}{2} & 1
\end{array} \right] \\
3A – 2B &= \left[ \begin{array}{rrr}
9 & -3 & 12 \\
6 & 0 & 18
\end{array} \right] – \left[ \begin{array}{rrr}
2 & 4 & -2 \\
0 & 6 & 4
\end{array} \right] = \left[ \begin{array}{rrr}
7 & -7 & 14 \\
6 & -6 & 14
\end{array} \right]
\end{align*}

If $A$ is any matrix, note that $kA$ is the same size as $A$ for all scalars $k$. We also have
\begin{equation*}
0A = 0 \quad \mbox{ and } \quad k0 = 0
\end{equation*}
because the zero matrix has every entry zero. In other words, $kA = 0$ if either $k = 0$ or $A = 0$. The converse of this statement is also true, as Example 2.1.7 shows.

Solution:

Write $A = \left[ a_{ij} \right]$ so that $kA = 0$ means $ka_{ij} = 0$ for all $i$ and $j$. If $k = 0$, there is nothing to do. If $k \neq 0$, then $ka_{ij} = 0$ implies that $a_{ij} = 0$ for all $i$ and $j$; that is, $A = 0$.

For future reference, the basic properties of matrix addition and scalar multiplication are listed in Theorem 2.1.1.

Proof:
Properties 1–4 were given previously. To check Property 5, let $A = \left[ a_{ij} \right]$ and $B = \left[ b_{ij} \right]$ denote matrices of the same size. Then $A + B = \left[ a_{ij} + b_{ij} \right]$, as before, so the $(i, j)$-entry of $k(A + B)$ is
\begin{equation*}
k(a_{ij} + b_{ij}) = ka_{ij} + kb_{ij}
\end{equation*}
But this is just the $(i, j)$-entry of $kA + kB$, and it follows that $k(A + B) = kA + kB$. The other Properties can be similarly verified; the details are left to the reader.

The Properties in Theorem 2.1.1 enable us to do calculations with matrices in much the same way that
numerical calculations are carried out. To begin, Property 2 implies that the sum
\begin{equation*}
(A + B) + C = A + (B + C)
\end{equation*}
is the same no matter how it is formed and so is written as $A + B + C$. Similarly, the sum
\begin{equation*}
A + B + C + D
\end{equation*}
is independent of how it is formed; for example, it equals both $(A + B) + (C + D)$ and $A + \left[ B + (C + D) \right]$. Furthermore, property 1 ensures that, for example,
\begin{equation*}
B + D + A + C = A + B + C + D
\end{equation*} In other words, the order in which the matrices are added does not matter. A similar remark applies to sums of five (or more) matrices.

Properties 5 and 6 in Theorem 2.1.1  are called distributive laws for scalar multiplication, and they extend to sums of more than two terms. For example,
\begin{equation*}
k(A + B – C) = kA + kB – kC
\end{equation*}
\begin{equation*}
(k + p – m)A = kA + pA -mA
\end{equation*}
Similar observations hold for more than three summands. These facts, together with properties 7 and 8, enable us to simplify expressions by collecting like terms, expanding, and taking common factors in exactly the same way that algebraic expressions involving variables and real numbers are manipulated. The following example illustrates these techniques.

Solution:

The reduction proceeds as though $A$, $B$, and $C$ were variables.
\begin{align*}
2(A &+ 3C) – 3(2C – B) – 3 \left[ 2(2A + B – 4C) – 4(A – 2C) \right] \\
&= 2A + 6C – 6C + 3B – 3 \left[ 4A + 2B – 8C – 4A + 8C \right] \\
&= 2A + 3B – 3 \left[ 2B \right] \\
&= 2A – 3B
\end{align*}

Transpose of a Matrix

Many results about a matrix $A$ involve the rows of $A$, and the corresponding result for columns is derived in an analogous way, essentially by replacing the word row by the word column throughout. The following definition is made with such applications in mind.

In other words, the first row of $A^{T}$ is the first column of $A$ (that is it consists of the entries of column 1 in order). Similarly the second row of $A^{T}$ is the second column of $A$, and so on.

Solution:

\begin{equation*}
A^{T} = \left[ \begin{array}{rrr}
1 & 3 & 2
\end{array} \right],\ B^{T} = \left[ \begin{array}{r}
5 \\
2 \\
6
\end{array} \right],\ C^{T} = \left[ \begin{array}{rrr}
1 & 3 & 5 \\
2 & 4 & 6
\end{array} \right], \mbox{ and } D^{T} = D.
\end{equation*}

If $A = \left[ a_{ij} \right]$ is a matrix, write $A^{T} = \left[ b_{ij} \right]$. Then $b_{ij}$ is the $j$th element of the $i$th row of $A^{T}$ and so is the $j$th element of the $i$th column of $A$. This means $b_{ij} = a_{ji}$, so the definition of $A^{T}$ can be stated as follows:
\begin{equation*}
\mbox{If } A = \left[ a_{ij} \right] \mbox{, then } A^{T} = \left[ a_{ji} \right]. \tag{2.1}

\end{equation*}
This is useful in verifying the following properties of transposition.

Proof:

Property 1 is part of the definition of $A^{T}$, and Property 2 follows from (2.1). As to Property 3: If $A = \leftB a_{ij} \rightB$, then $kA = \leftB ka_{ij} \rightB$, so (2.1) gives
\begin{equation*}
(kA)^{T} = \left[ ka_{ji} \right] = k \left[ a_{ji} \right] = kA^{T}
\end{equation*}
Finally, if $B = \left[ b_{ij} \right]$, then $A + B = \left[ c_{ij} \right]$ where $c_{ij} = a_{ij} + b_{ij}$ Then (2.1) gives Property 4:
\begin{equation*}
(A + B)^{T} = \left[ c_{ij} \right]^{T} = \left[ c_{ji} \right] = \left[ a_{ji} + b_{ji} \right] = \left[ a_{ji} \right] + \left[ b_{ji} \right] = A^{T} + B^{T}
\end{equation*}

There is another useful way to think of transposition. If $A = \left[ a_{ij} \right]$ is an $m \times n$ matrix, the elements $a_{11}, a_{22}, a_{33}, \dots$ are called the main diagonal of $A$. Hence the main diagonal extends down and to the right from the upper left corner of the matrix $A$; it is shaded in the following examples:

Thus forming the transpose of a matrix $A$ can be viewed as “flipping” $A$ about its main diagonal, or as “rotating” $A$ through $180^{\circ}$ about the line containing the main diagonal. This makes Property 2 in Theorem~\ref{thm:002240} transparent.

Solution:

Using Theorem 2.1.2, the left side of the equation is
\begin{equation*}
\left(2A^{T} – 3 \left[ \begin{array}{rr}
1 & 2 \\
-1 & 1
\end{array} \right]\right)^{T} = 2\left(A^{T}\right)^{T} – 3 \left[ \begin{array}{rr}
1 & 2 \\
-1 & 1
\end{array} \right]^{T} = 2A – 3 \left[ \begin{array}{rr}
1 & -1 \\
2 & 1
\end{array} \right]
\end{equation*}
Hence the equation becomes
\begin{equation*}
2A – 3 \left[ \begin{array}{rr}
1 & -1 \\
2 & 1
\end{array} \right] = \left[ \begin{array}{rr}
2 & 3 \\
-1 & 2
\end{array} \right]
\end{equation*}
Thus
$2A = \left[ \begin{array}{rr}
2 & 3 \\
-1 & 2
\end{array} \right] + 3 \left[ \begin{array}{rr}
1 & -1 \\
2 & 1
\end{array} \right] = \left[ \begin{array}{rr}
5 & 0 \\
5 & 5
\end{array} \right]$, so finally
$A = \frac{1}{2} \left[ \begin{array}{rr}
5 & 0 \\
5 & 5
\end{array} \right] = \frac{5}{2} \left[ \begin{array}{rr}
1 & 0 \\
1 & 1
\end{array} \right]$.

Note that Example 2.1.10 can also be solved by first transposing both sides, then solving for $A^{T}$, and so obtaining $A = (A^{T})^{T}$. The reader should do this.

The matrix $D = \left[ \begin{array}{rr}
1 & 2 \\
2 & 5 \end{array}\right]$ in Example 2.1.9 has the property that $D = D^{T}$. Such matrices are important; a matrix $A$ is called symmetric if $A = A^{T}$. A symmetric matrix $A$ is necessarily square (if $A$ is $m \times n$, then $A^{T}$ is $n \times m$, so $A = A^{T}$ forces $n = m$). The name comes from the fact that these matrices exhibit a symmetry about the main diagonal. That is, entries that are directly across the main diagonal from each other are equal.

For example, $\left[ \begin{array}{ccc}
a & b & c \\
b^\prime & d & e \\
c^\prime & e^\prime & f
\end{array} \right]$ is symmetric when $b = b^\prime$, $c = c^\prime$, and $e = e^\prime$.

Solution:

We have $A^{T} = A$ and $B^{T} = B$, so, by Theorem 2.1.2, we have $(A + B)^{T} = A^{T} + B^{T} = A + B$. Hence $A + B$ is symmetric.

Solution:

If we iterate the given equation, Theorem 2.1.2 gives
\begin{equation*}
A = 2A^{T} = 2 {\left[ 2A^{T} \right]}^T = 2 \left[ 2(A^{T})^{T} \right] = 4A
\end{equation*}
Subtracting $A$ from both sides gives $3A = 0$, so $A = \frac{1}{3}(0) = 0$.

2.2 Matrix-Vector Multiplication

Up to now we have used matrices to solve systems of linear equations by manipulating the rows of the augmented matrix. In this section we introduce a different way of describing linear systems that makes more use of the coefficient matrix of the system and leads to a useful way of “multiplying” matrices.

Vectors

It is a well-known fact in analytic geometry that two points in the plane with coordinates $(a_{1}, a_{2})$ and $(b_{1}, b_{2})$ are equal if and only if $a_{1} = b_{1}$ and $a_{2} = b_{2}$. Moreover, a similar condition applies to points $(a_{1}, a_{2}, a_{3})$ in space. We extend this idea as follows.

An ordered sequence $(a_{1}, a_{2}, \dots, a_{n})$ of real numbers is called an ordered $\bm{n}$-tuple. The word “ordered” here reflects our insistence that two ordered $n$-tuples are equal if and only if corresponding entries are the same. In other words,
\begin{equation*}
(a_{1}, a_{2}, \dots, a_{n}) = (b_{1}, b_{2}, \dots, b_{n}) \quad \mbox{if and only if} \quad a_{1} = b_{1}, a_{2} = b_{2}, \dots, \mbox{ and } a_{n} = b_{n}.
\end{equation*}
Thus the ordered $2$-tuples and $3$-tuples are just the ordered pairs and triples familiar from geometry.

There are two commonly used ways to denote the $n$-tuples in $\mathbb{R}^{n}$: As rows $(r_{1}, r_{2}, \dots, r_{n})$ or columns $\left[ \begin{array}{c}
r_{1} \\
r_{2} \\
\vdots \\
r_{n}
\end{array} \right]$;
the notation we use depends on the context. In any event they are called vectors or $n$-vectors and will be denoted using bold type such as x or v. For example, an $m \times n$ matrix $A$ will be written as a row of columns:
\begin{equation*}
A = \left[ \begin{array}{cccc}
\textbf{a}_{1} & \textbf{a}_{2} & \cdots & \textbf{a}_{n}
\end{array} \right] \mbox{ where } \textbf{a}_{j} \mbox{ denotes column } j \mbox{ of } A \mbox{ for each } j.
\end{equation*}

If $\textbf{x}$ and $\textbf{y}$ are two $n$-vectors in $\mathbf{R}^n$, it is clear that their matrix sum $\textbf{x} + \textbf{y}$ is also in $\mathbf{R}^n$ as is the scalar multiple $k\textbf{x}$ for any real number $k$. We express this observation by saying that $\mathbf{R}^n$ is closed under addition and scalar multiplication. In particular, all the basic properties in Theorem 2.1.1 are true of these $n$-vectors. These properties are fundamental and will be used frequently below without comment. As for matrices in general, the $n \times 1$ zero matrix is called the zero $\bm{n}$-vector in $\mathbf{R}^n$ and, if $\textbf{x}$ is an $n$-vector, the $n$-vector $-\textbf{x}$ is called the negative $\textbf{x}$.

Of course, we have already encountered these $n$-vectors in Section 1.3 as the solutions to systems of linear equations with $n$ variables. In particular we defined the notion of a linear combination of vectors and showed that a linear combination of solutions to a homogeneous system is again a solution. Clearly, a linear combination of $n$-vectors in $\mathbf{R}^n$ is again in $\mathbf{R}^n$, a fact that we will be using.

Matrix-Vector Multiplication

Given a system of linear equations, the left sides of the equations depend only on the coefficient matrix $A$ and the column $\textbf{x}$ of variables, and not on the constants. This observation leads to a fundamental idea in linear algebra: We view the left sides of the equations as the “product” $A\textbf{x}$ of the matrix $A$ and the vector $\textbf{x}$. This simple change of perspective leads to a completely new way of viewing linear systems—one that is very useful and will occupy our attention throughout this book.

To motivate the definition of the “product” $A\textbf{x}$, consider first the following system of two equations in three variables:
\begin{equation*}\tag{2.2}
\arraycolsep=1pt
\begin{array}{rrrrrrr}
ax_{1} & + & bx_{2} & + & cx_{3} & = & b_{1} \\
a^{\prime}x_{1} & + & b^{\prime}x_{2} & + & c^{\prime}x_{3} & = & b_{1}
\end{array}
\end{equation*}

and let $A = \left[ \begin{array}{ccc}
a & b & c \\
a^\prime & b^\prime & c^\prime
\end{array} \right]$, $\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3}
\end{array} \right]$, $\textbf{b} = \left[ \begin{array}{c}
b_{1} \\
b_{2}
\end{array} \right]$ denote the coefficient matrix, the variable matrix, and the constant matrix, respectively. The system (2.2) can be expressed as a single vector equation
\begin{equation*}
\left[ \arraycolsep=1pt \begin{array}{rrrrr}
ax_{1} & + & bx_{2} & + & cx_{3} \\
a^\prime x_{1} & + & b^\prime x_{2} & + & c^\prime x_{3}
\end{array} \right] = \left[ \begin{array}{c}
b_{1} \\
b_{2}
\end{array} \right]
\end{equation*}
which in turn can be written as follows:
\begin{equation*}
x_{1} \left[ \begin{array}{c}
a \\
a^\prime
\end{array} \right] +
x_{2} \left[ \begin{array}{c}
b \\
b^\prime
\end{array} \right] +
x_{3} \left[ \begin{array}{c}
c \\
c^\prime
\end{array} \right] = \left[ \begin{array}{c}
b_{1} \\
b_{2}
\end{array} \right]
\end{equation*}
Now observe that the vectors appearing on the left side are just the columns
\begin{equation*}
\textbf{a}_{1} = \left[ \begin{array}{c}
a \\
a^\prime
\end{array} \right],
\textbf{a}_{2} = \left[ \begin{array}{c}
b \\
b^\prime
\end{array} \right], \mbox{ and }
\textbf{a}_{3} = \left[ \begin{array}{c}
c \\
c^\prime
\end{array} \right]
\end{equation*}
of the coefficient matrix $A$. Hence the system (2.2) takes the form
\begin{equation*} \tag{2.3}
x_{1}\textbf{a}_{1} + x_{2}\textbf{a}_{2} + x_{3}\textbf{a}_{3} = \textbf{b}
\end{equation*}
This shows that the system (2.2) has a solution if and only if the constant matrix $\textbf{b}$ is a linear combination of the columns of $A$, and that in this case the entries of the solution are the coefficients $x_{1}$, $x_{2}$, and $x_{3}$ in this linear combination.

Moreover, this holds in general. If $A$ is any $m \times n$ matrix, it is often convenient to view $A$ as a row of columns. That is, if $\textbf{a}_{1}, \textbf{a}_{2}, \dots, \textbf{a}_{n}$ are the columns of $A$, we write
\begin{equation*}
A = \left[ \begin{array}{cccc}
\textbf{a}_{1} & \textbf{a}_{2} & \cdots & \textbf{a}_{n}
\end{array} \right]
\end{equation*}
and say that $A = \left[ \begin{array}{cccc}\textbf{a}_{1} & \textbf{a}_{2} & \cdots & \textbf{a}_{n}
\end{array} \right]$ is given in terms of its columns.

Now consider any system of linear equations with $m \times n$ coefficient matrix $A$. If $\textbf{b}$ is the constant matrix of the system, and if $\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
\vdots \\
x_{n}
\end{array} \right]$
is the matrix of variables then, exactly as above, the system can be written as a single vector equation
\begin{equation*} \tag{2.4}
x_{1}\textbf{a}_{1} + x_{2}\textbf{a}_{2} + \dots + x_{n}\textbf{a}_{n} = \textbf{b}
\end{equation*}

Solution:

\begin{equation*}
x_{1} \left[ \begin{array}{r}
3 \\
1 \\
0
\end{array} \right] +
x_{2} \left[ \begin{array}{r}
2 \\
-3 \\
1
\end{array} \right] +
x_{3} \left[ \begin{array}{r}
-4 \\
1 \\
-5
\end{array} \right] =
\left[ \begin{array}{r}
0 \\
3 \\
-1
\end{array} \right]
\end{equation*}

As mentioned above, we view the left side of (2.4) as the product of the matrix $A$ and the vector $\textbf{x}$. This basic idea is formalized in the following definition:

In other words, if $A$ is $m \times n$ and $\textbf{x}$ is an $n$-vector, the product $A\textbf{x}$ is the linear combination of the columns of $A$ where the coefficients are the entries of $\textbf{x}$ (in order).

Note that if $A$ is an $m \times n$ matrix, the product $A\textbf{x}$ is only defined if $\textbf{x}$ is an $n$-vector and then the vector $A\textbf{x}$ is an $m$-vector because this is true of each column $\textbf{a}_{j}$ of $A$. But in this case the system of linear equations with coefficient matrix $A$ and constant vector $\textbf{b}$ takes the form of a single matrix equation
\begin{equation*}
A\textbf{x} = \textbf{b}
\end{equation*}
The following theorem combines Definition 2.5 and equation (2.4) and summarizes the above discussion. Recall that a system of linear equations is said to be consistent if it has at least one solution.

A system of linear equations in the form $A\textbf{x} = \textbf{b}$ as in (1) of Theorem 2.2.1 is said to be written in matrix form. This is a useful way to view linear systems as we shall see.

Theorem 2.2.1 transforms the problem of solving the linear system $A\textbf{x} = \textbf{b}$ into the problem of expressing the constant matrix $B$ as a linear combination of the columns of the coefficient matrix $A$. Such a change in perspective is very useful because one approach or the other may be better in a particular situation; the importance of the theorem is that there is a choice.

Solution:

By Definition 2.5:
$A\textbf{x} = 2 \left[ \begin{array}{r}
2 \\
0 \\
-3
\end{array} \right] + 1
\left[ \begin{array}{r}
-1 \\
2 \\
4
\end{array} \right] + 0
\left[ \begin{array}{r}
3 \\
-3 \\
1
\end{array} \right] – 2
\left[ \begin{array}{r}
5 \\
1 \\
2
\end{array} \right] =
\left[ \begin{array}{r}
-7 \\
0 \\
-6
\end{array} \right]$.

Solution:

Here the column of coefficients is
$
\textbf{x} = \left[ \begin{array}{r}
2 \\
-3 \\
5 \\
1
\end{array} \right]. $
Hence Definition 2.5 gives
\begin{equation*}
A\textbf{x} = 2\textbf{a}_{1} – 3\textbf{a}_{2} + 5\textbf{a}_{3} + \textbf{a}_{4}
\end{equation*}
where $A = \left[ \begin{array}{cccc}
\textbf{a}_{1} & \textbf{a}_{2} & \textbf{a}_{3} & \textbf{a}_{4}
\end{array} \right]$ is the matrix with $\textbf{a}_{1}$, $\textbf{a}_{2}$, $\textbf{a}_{3}$, and $\textbf{a}_{4}$ as its columns.

Solution:

By Theorem 2.2.1, $\textbf{b}$ is a linear combination of $\textbf{a}_{1}$, $\textbf{a}_{2}$, $\textbf{a}_{3}$, and $\textbf{a}_{4}$ if and only if the system $A\textbf{x} = \textbf{b}$ is consistent (that is, it has a solution). So in each case we carry the augmented matrix $\left[ A|\textbft{b} \right]$ of the system $A\textbf{x} = \textbf{b}$ to reduced form.

1.  Here
$
\left[ \begin{array}{rrrr|r}
2 & 1 & 3 & 3 & 1 \\
0 & 1 & -1 & 1 & 2 \\
-1 & 1 & -3 & 0 & 3
\end{array} \right]
\rightarrow
\left[ \begin{array}{rrrr|r}
1 & 0 & 2 & 1 & 0 \\
0 & 1 & -1 & 1 & 0 \\
0 & 0 & 0 & 0 & 1
\end{array} \right]$, so the system $A\textbf{x} = \textbf{b}$ has no solution in this case. Hence $\textbf{b}$ is \textit{not} a linear combination of $\textbf{a}_{1}$, $\textbf{a}_{2}$, $\textbf{a}_{3}$, and $\textbf{a}_{4}$.

2.  Now
$
\left[ \begin{array}{rrrr|r}
2 & 1 & 3 & 3 & 4 \\
0 & 1 & -1 & 1 & 2 \\
-1 & 1 & -3 & 0 & 1
\end{array} \right]
\rightarrow
\left[ \begin{array}{rrrr|r}
1 & 0 & 2 & 1 & 1 \\
0 & 1 & -1 & 1 & 2 \\
0 & 0 & 0 & 0 & 0
\end{array} \right]$, so the system $A\textbf{x} = \textbf{b}$ is consistent.

Thus $\textbf{b}$ is a linear combination of $\textbf{a}_{1}$, $\textbf{a}_{2}$, $\textbf{a}_{3}$, and $\textbf{a}_{4}$ in this case. In fact the general solution is $x_{1} = 1 – 2s – t$, $x_{2} = 2 + s – t$, $x_{3} = s$, and $x_{4} = t$ where $s$ and $t$ are arbitrary parameters. Hence $x_{1}\textbf{a}_{1} + x_{2}\textbf{a}_{2} + x_{3}\textbf{a}_{3} + x_{4}\textbf{a}_{4} = \textbf{b} = \left[ \begin{array}{r}
4 \\
2 \\
1
\end{array} \right]$
for any choice of $s$ and $t$. If we take $s = 0$ and $t = 0$, this becomes $\textbf{a}_{1} + 2\textbf{a}_{2} = \textbf{b}$, whereas taking $s = 1 = t$ gives $-2\textbf{a}_{1} + 2\textbf{a}_{2} + \textbf{a}_{3} + \textbf{a}_{4} = \textbf{b}$.

Solution:

If $\textbf{x} = \left[ \begin{array}{r}
x_{1} \\
x_{2} \\
x_{3}
\end{array} \right],$ then Definition 2.5 gives
\begin{equation*}
I\textbf{x} = x_{1} \left[ \begin{array}{r}
1 \\
0 \\
0 \\
\end{array} \right] +
x_{2} \left[ \begin{array}{r}
0 \\
1 \\
0 \\
\end{array} \right] +
x_{3} \left[ \begin{array}{r}
0 \\
0 \\
1 \\
\end{array} \right] =
\left[ \begin{array}{r}
x_{1} \\
0 \\
0 \\
\end{array} \right] +
\left[ \begin{array}{r}
0 \\
x_{2} \\
0 \\
\end{array} \right] +
\left[ \begin{array}{r}
0 \\
0 \\
x_{3} \\
\end{array} \right] =
\left[ \begin{array}{r}
x_{1} \\
x_{2} \\
x_{3} \\
\end{array} \right] = \textbf{x}
\end{equation*}

The matrix $I$ in Example 2.2.6 is called the $3 \times 3$ identity matrix, and we will encounter such matrices again in future. Before proceeding, we develop some algebraic properties of matrix-vector multiplication that are used extensively throughout linear algebra.

Proof:

We prove (3); the other verifications are similar and are left as exercises. Let $A = \left[ \begin{array}{cccc}
\textbf{a}_{1} & \textbf{a}_{2} & \cdots & \textbf{a}_{n}
\end{array} \right]$ and $B = \left[ \begin{array}{cccc}
\textbf{b}_{1} & \textbf{b}_{2} & \cdots & \textbf{b}_{n}
\end{array} \right]$ be given in terms of their columns. Since adding two matrices is the same as adding their columns, we have
\begin{equation*}
A + B = \left[ \begin{array}{cccc}
\textbf{a}_{1} + \textbf{b}_{1} & \textbf{a}_{2} + \textbf{b}_{2} & \cdots & \textbf{a}_{n} + \textbf{b}_{n}
\end{array} \right]
\end{equation*}
If we write $\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
\vdots \\
x_{n}
\end{array} \right].$
Definition 2.5 gives
\begin{align*}
(A + B)\textbf{x} &= x_{1}(\textbf{a}_{1} + \textbf{b}_{1}) + x_{2}(\textbf{a}_{2} + \textbf{b}_{2}) + \dots + x_{n}(\textbf{a}_{n} + \textbf{b}_{n}) \\
&= (x_{1}\textbf{a}_{1} + x_{2}\textbf{a}_{2} + \dots + x_{n}\textbf{a}_{n}) + (x_{1}\textbf{b}_{1} + x_{2}\textbf{b}_{2} + \dots + x_{n}\textbf{b}_{n})\\
&= A\textbf{x} + B\textbf{x}
\end{align*}

Theorem 2.2.2 allows matrix-vector computations to be carried out much as in ordinary arithmetic. For example, for any $m \times n$ matrices $A$ and $B$ and any $n$-vectors $\textbf{x}$ and $\textbf{y}$, we have:
\begin{equation*}
A(2 \textbf{x} – 5 \textbf{y}) = 2A\textbf{x} – 5A\textbf{y} \quad \mbox{ and } \quad (3A -7B)\textbf{x} = 3A\textbf{x} – 7B\textbf{x}
\end{equation*}
We will use such manipulations throughout the book, often without mention.

Linear Equations

Theorem 2.2.2 also gives a useful way to describe the solutions to a system
\begin{equation*}
A\textbf{x} = \textbf{b}
\end{equation*}
of linear equations. There is a related system
\begin{equation*}
A\textbf{x} = \textbf{0}
\end{equation*}
called the associated homogeneous system, obtained from the original system $A\textbf{x} = \textbf{b}$ by replacing all the constants by zeros. Suppose $\textbf{x}_{1}$ is a solution to $A\textbf{x} = \textbf{b}$ and $\textbf{x}_{0}$ is a solution to $A\textbf{x} = \textbf{0}$ (that is $A\textbf{x}_{1} = \textbf{b}$ and $A\textbf{x}_{0} = \textbf{0}$). Then $\textbf{x}_{1} + \textbf{x}_{0}$ is another solution to $A\textbf{x} = \textbf{b}$. Indeed, Theorem 2.2.2 gives
\begin{equation*}
A(\textbf{x}_{1} + \textbf{x}_{0}) = A\textbf{x}_{1} + A\textbf{x}_{0} = \textbf{b} + \textbf{0} = \textbf{b}
\end{equation*}
This observation has a useful converse.

Proof:
Suppose $\textbf{x}_{2}$ is also a solution to $A\textbf{x} = \textbf{b}$, so that $A\textbf{x}_{2} = \textbf{b}$. Write $\textbf{x}_{0} = \textbf{x}_{2} – \textbf{x}_{1}$. Then $\textbf{x}_{2} = \textbf{x}_{0} + \textbf{x}_{1}$ and, using Theorem 2.2.2, we compute
\begin{equation*}
A\textbf{x}_{0} = A(\textbf{x}_{2} – \textbf{x}_{1}) = A\textbf{x}_{2} – A\textbf{x}_{1} = \textbf{b} – \textbf{b} = \textbf{0}
\end{equation*}
Hence $\textbf{x}_{0}$ is a solution to the associated homogeneous system $A\textbf{x} = \textbf{0}$.

Note that gaussian elimination provides one such representation.

Solution:

Gaussian elimination gives $x_{1} = 4 + 2s – t$, $x_{2} = 2 + s + 2t$, $x_{3} = s$, and $x_{4} = t$ where $s$ and $t$ are arbitrary parameters. Hence the general solution can be written
\begin{equation*}
\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4}
\end{array} \right] =
\left[ \begin{array}{c}
4 + 2s – t \\
2 + s + 2t \\
s \\
t
\end{array} \right] =
\left[ \begin{array}{r}
4 \\
2 \\
0 \\
0
\end{array} \right] +
\left( s \left[ \begin{array}{r}
2 \\
1 \\
1 \\
0
\end{array} \right] + t \left[ \begin{array}{r}
-1 \\
2 \\
0 \\
1
\end{array} \right] \right)
\end{equation*}
Thus
$\textbf{x}_1 = \left[ \begin{array}{r}
4 \\
2 \\
0 \\
0
\end{array} \right]$
is a particular solution (where $s = 0 = t$), and
$
\textbf{x}_{0} = s \left[ \begin{array}{r}
2 \\
1 \\
1 \\
0
\end{array} \right] + t \left[ \begin{array}{r}
-1 \\
2 \\
0 \\
1
\end{array} \right]$ gives all solutions to the associated homogeneous system. (To see why this is so, carry out the gaussian elimination again but with all the constants set equal to zero.)

The following useful result is included with no proof.

The Dot Product

Definition 2.5 is not always the easiest way to compute a matrix-vector product $A\textbf{x}$ because it requires that the columns of $A$ be explicitly identified. There is another way to find such a product which uses the matrix $A$ as a whole with no reference to its columns, and hence is useful in practice. The method depends on the following notion.

To see how this relates to matrix products, let $A$ denote a $3 \times 4$ matrix and let $\textbf{x}$ be a $4$-vector. Writing
\begin{equation*}
\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4}
\end{array} \right] \quad \mbox{ and } \quad
A = \left[ \begin{array}{cccc}
a_{11} & a_{12} & a_{13} & a_{14} \\
a_{21} & a_{22} & a_{23} & a_{24} \\
a_{31} & a_{32} & a_{33} & a_{34}
\end{array} \right]
\end{equation*}
in the notation of Section 2.1, we compute
\begin{align*}
A\textbf{x} = \left[ \begin{array}{cccc}
a_{11} & a_{12} & a_{13} & a_{14} \\
a_{21} & a_{22} & a_{23} & a_{24} \\
a_{31} & a_{32} & a_{33} & a_{34}
\end{array} \right] \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4}
\end{array} \right] &= x_{1} \left[ \begin{array}{c}
a_{11} \\
a_{21} \\
a_{31}
\end{array} \right] + x_{2} \left[ \begin{array}{c}
a_{12} \\
a_{22} \\
a_{32}
\end{array} \right] + x_{3} \left[ \begin{array}{c}
a_{13} \\
a_{23} \\
a_{33}
\end{array} \right] + x_{4} \left[ \begin{array}{c}
a_{14} \\
a_{24} \\
a_{34}
\end{array} \right] \\
&= \left[ \begin{array}{c}
a_{11}x_{1} + a_{12}x_{2} + a_{13}x_{3} + a_{14}x_{4} \\
a_{21}x_{1} + a_{22}x_{2} + a_{23}x_{3} + a_{24}x_{4} \\
a_{31}x_{1} + a_{32}x_{2} + a_{33}x_{3} + a_{34}x_{4}
\end{array} \right]
\end{align*}
From this we see that each entry of $A\textbf{x}$ is the dot product of the corresponding row of $A$ with $\textbf{x}$. This computation goes through in general, and we record the result in Theorem 2.2.5.

This result is used extensively throughout linear algebra.

If $A$ is $m \times n$ and $\textbf{x}$ is an $n$-vector, the computation of $A\textbf{x}$ by the dot product rule is simpler than using Definition 2.5 because the computation can be carried out directly with no explicit reference to the columns of $A$ (as in Definition 2.5. The first entry of $A\textbf{x}$ is the dot product of row 1 of $A$ with $\textbf{x}$. In hand calculations this is computed by going across row one of $A$, going down the column $\textbf{x}$, multiplying corresponding entries, and adding the results. The other entries of $A\textbf{x}$ are computed in the same way using the other rows of $A$ with the column $\textbf{x}$.

In general, compute entry $i$ of $A\textbf{x}$ as follows (see the diagram):

Go across row $i$ of $A$ and down column $\textbf{x}$, multiply corresponding entries, and add the results.

As an illustration, we rework Example 2.2.2 using the dot product rule instead of Definition 2.5.

Solution:
The entries of $A\textbf{x}$ are the dot products of the rows of $A$ with $\textbf{x}$:
\begin{equation*}
A\textbf{x} = \left[ \begin{array}{rrrr}
2 & -1 & 3 & 5 \\
0 & 2 & -3 & 1 \\
-3 & 4 & 1 & 2
\end{array} \right] \left[ \begin{array}{r}
2 \\
1 \\
0 \\
-2
\end{array} \right] = \left[ \begin{array}{rrrrrrr}
2 \cdot 2 & + & (-1)1 & + & 3 \cdot 0 & + & 5(-2) \\
0 \cdot 2 & + & 2 \cdot 1 & + & (-3)0 & + & 1(-2) \\
(-3)2 & + & 4 \cdot 1 & + & 1 \cdot 0 & + & 2(-2)
\end{array} \right] = \left[ \begin{array}{r}
-7 \\
0 \\
-6
\end{array} \right]
\end{equation*}
Of course, this agrees with the outcome in Example 2.2.2.

Solution:

Write $A = \left[ \begin{array}{rrrrr}
5 & -1 & 2 & 1 & -3 \\
1 & 1 & 3 & -5 & 2 \\
-1 & 1 & -2 & 0 & -3
\end{array} \right]$, $\textbf{b} = \left[ \begin{array}{r}
8 \\
-2 \\
0
\end{array} \right]$, and $\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4} \\
x_{5}
\end{array} \right]$. Then the dot product rule gives $A\textbf{x} = \left[ \arraycolsep=1pt \begin{array}{rrrrrrrrr}
5x_{1} & – & x_{2} & + & 2x_{3} & + & x_{4} & – & 3x_{5} \\
x_{1} & + & x_{2} & + & 3x_{3} & – & 5x_{4} & + & 2x_{5} \\
-x_{1} & + & x_{2} & – & 2x_{3} & & & – & 3x_{5}
\end{array} \right]$, so the entries of $A\textbf{x}$ are the left sides of the equations in the linear system. Hence the system becomes $A\textbf{x} = \textbf{b}$ because matrices are equal if and only corresponding entries are equal.

Solution:

For each $k$, entry $k$ of $A\textbf{x}$ is the dot product of row $k$ of $A$ with $\textbf{x}$, and this is zero because row $k$ of $A$ consists of zeros.

The first few identity matrices are
\begin{equation*}
I_{2} = \left[ \begin{array}{rr}
1 & 0 \\
0 & 1
\end{array} \right], \quad I_{3} = \left[ \begin{array}{rrr}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{array} \right], \quad I_{4} = \left[ \begin{array}{rrrr}
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1
\end{array} \right], \quad \dots
\end{equation*}
In Example 2.2.6 we showed that $I_{3}\textbf{x} = \textbf{x}$ for each $3$-vector $\textbf{x}$ using Definition 2.5. The following result shows that this holds in general, and is the reason for the name.

Solution:

We verify the case $n = 4$. Given the $4$-vector $\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4}
\end{array} \right]$
the dot product rule gives
\begin{equation*}
I_{4}\textbf{x} = \left[\begin{array}{rrrr}
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1
\end{array} \right] \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4}
\end{array} \right] = \left[ \begin{array}{c}
x_{1} + 0 + 0 + 0 \\
0 + x_{2} + 0 + 0 \\
0 + 0 + x_{3} + 0 \\
0 + 0 + 0 + x_{4}
\end{array} \right] = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
x_{3} \\
x_{4}
\end{array} \right] = \vect{x}
\end{equation*}
In general, $I_{n}\textbf{x} = \textbf{x}$ because entry $k$ of $I_{n}\textbf{x}$ is the dot product of row $k$ of $I_{n}$ with $\textbf{x}$, and row $k$ of $I_{n}$ has $1$ in position $k$ and zeros elsewhere.

Solution:

Write $\textbf{e}_{j} = \left[ \begin{array}{c}
t_{1} \\
t_{2} \\
\vdots \\
t_{n}
\end{array} \right]$
where $t_{j} = 1$, but $t_{i} = 0$ for all $i \neq j$. Then Theorem 2.2.5 gives
\begin{equation*}
A\textbf{e}_{j} = t_{1}\textbf{a}_{1} + \dots + t_{j}\textbf{a}_{j} + \dots + t_{n}\textbf{a}_{n} = 0 + \dots + \textbf{a}_{j} + \dots + 0 = \textbf{a}_{j}
\end{equation*}

Example 2.2.12will be referred to later; for now we use it to prove:

Proof:

Write $A = \left[ \begin{array}{cccc}
\textbf{a}_{1} & \textbf{a}_{2} & \cdots & \textbf{a}_{n}
\end{array} \right]$ and $B = \left[ \begin{array}{cccc}
\textbf{b}_{1} & \textbf{b}_{2} & \cdots & \textbf{b}_{n}
\end{array} \right]$ and in terms of their columns. It is enough to show that $\textbf{a}_{k} = \textbf{b}_{k}$ holds for all $k$. But we are assuming that $A\textbf{e}_{k} = B\textbf{e}_{k}$, which gives $\textbf{a}_{k} = \textbf{b}_{k}$ by Example 2.2.12.

We have introduced matrix-vector multiplication as a new way to think about systems of linear equations. But it has several other uses as well. It turns out that many geometric operations can be described using matrix multiplication, and we now investigate how this happens. As a bonus, this description provides a geometric “picture” of a matrix by revealing the effect on a vector when it is multiplied by $A$. This “geometric view” of matrices is a fundamental tool in understanding them.

2.3 Matrix Multiplication

In Section 2.2 matrix-vector products were introduced. If $A$ is an $m \times n$ matrix, the product $A\textbf{x}$ was defined for any $n$-column $\vect{x}$ in $\mathbf{R}^n$ as follows: If $A = \left[ \begin{array}{cccc}
\textbf{a}_{1} & \textbf{a}_{2} & \cdots & \textbf{a}_{n}
\end{array} \right]$ where the $\textbf{a}_{j}$ are the columns of $A$, and if $\textbf{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2} \\
\vdots \\
x_{n}
\end{array} \right]$,
Definition 2.5 reads
\begin{equation*} \tag{2.5}
A\textbf{x} = x_{1}\textbf{a}_{1} + x_{2}\textbf{a}_{2} + \cdots + x_{n}\textbf{a}_{n}
\end{equation*}
This was motivated as a way of describing systems of linear equations with coefficient matrix $A$. Indeed every such system has the form $A\textbf{x} = \textbf{b}$ where $\textbf{b}$ is the column of constants.

In this section we extend this matrix-vector multiplication to a way of multiplying matrices in general, and then investigate matrix algebra for its own sake. While it shares several properties of ordinary arithmetic, it will soon become clear that matrix arithmetic is different in a number of ways.

Thus the product matrix $AB$ is given in terms of its columns $A\textbf{b}_{1}, A\textbf{b}_{2}, \dots, A\textbf{b}_{n}$: Column $j$ of $AB$ is the matrix-vector product $A\textbf{b}_{j}$ of $A$ and the corresponding column $\textbf{b}_{j}$ of $B$. Note that each such product $A\textbf{b}_{j}$ makes sense by Definition 2.5 because $A$ is $m \times n$ and each $\vect{b}_{j}$ is in $\mathbf{R}^n$ (since $B$ has $n$ rows). Note also that if $B$ is a column matrix, this definition reduces to Definition 2.5 for matrix-vector multiplication.

Given matrices $A$ and $B$, Definition 2.9 and the above computation give
\begin{equation*}
A(B\vec{x}) = \left[ \begin{array}{cccc}
A\vec{b}_{1} & A\vec{b}_{2} & \cdots & A\vec{b}_{n}
\end{array} \right] \vec{x} = (AB)\vec{x}
\end{equation*}
for all $\vec{x}$ in $\mathbf{R}^k$. We record this for reference.

Here is an example of how to compute the product $AB$ of two matrices using Definition 2.9.

Solution:

The columns of $B$ are
$\vec{b}_{1} = \left[ \begin{array}{r}
8 \\
7 \\
6
\end{array} \right]$ and $\vec{b}_{2} = \left[ \begin{array}{r}
9 \\
2 \\
1
\end{array} \right]$, so Definition 2.5 gives
\begin{equation*}
A\vec{b}_{1} = \left[ \begin{array}{rrr}
2 & 3 & 5 \\
1 & 4 & 7 \\
0 & 1 & 8
\end{array} \right] \left[ \begin{array}{r}
8 \\
7 \\
6
\end{array} \right] = \left[ \begin{array}{r}
67 \\
78 \\
55
\end{array} \right] \mbox{ and } A\vec{b}_{2} = \left[ \begin{array}{rrr}
2 & 3 & 5 \\
1 & 4 & 7 \\
0 & 1 & 8
\end{array} \right] \left[ \begin{array}{r}
9 \\
2 \\
1
\end{array} \right] = \left[\begin{array}{r}
29 \\
24 \\
10
\end{array} \right]
\end{equation*}
Hence Definition 2.9 above gives $AB = \left[ \begin{array}{cc}
A\vec{b}_{1} & A\vec{b}_{2}
\end{array} \right] = \left[ \begin{array}{rr}
67 & 29 \\
78 & 24 \\
55 & 10
\end{array} \right]$.

While Definition 2.9 is important, there is another way to compute the matrix product $AB$ that gives a way to calculate each individual entry. In Section 2.2 we defined the dot product of two $n$-tuples to be the sum of the products of corresponding entries. We went on to show (Theorem 2.2.5) that if $A$ is an $m \times n$ matrix and $\vec{x}$ is an $n$-vector, then entry $j$ of the product $A\vec{x}$ is the dot product of row $j$ of $A$ with $\vec{x}$. This observation was called the “dot product rule” for matrix-vector multiplication, and the next theorem shows that it extends to matrix multiplication in general.

Proof:

Write $B = \left[ \begin{array}{cccc}
\vec{b}_{1} & \vec{b}_{2} & \cdots & \vec{b}_{n}
\end{array} \right]$ in terms of its columns. Then $A\vec{b}_{j}$ is column $j$ of $AB$ for each $j$. Hence the $(i, j)$-entry of $AB$ is entry $i$ of $A\vec{b}_{j}$, which is the dot product of row $i$ of $A$ with $\vec{b}_{j}$. This proves the theorem.

Thus to compute the $(i, j)$-entry of $AB$, proceed as follows (see the diagram):

Go across row $i$ of $A$, and down column $j$ of $B$, multiply corresponding entries, and add the results.

Note that this requires that the rows of $A$ must be the same length as the columns of $B$. The following rule is useful for remembering this and for deciding the size of the product matrix $AB$.

Compatibility Rule

Let $A$ and $B$ denote matrices. If $A$ is $m \times n$ and $B$ is $n^\prime \times k$, the product $AB$ can be formed if and only if $n=n^\prime$. In this case the size of the product matrix $AB$ is $m \times k$, and we say that $AB$ is defined, or that $A$ and $B$ are compatible for multiplication.

The diagram provides a useful mnemonic for remembering this. We adopt the following convention:

Whenever a product of matrices is written, it is tacitly assumed that the sizes of the factors are such that the product is defined.

To illustrate the dot product rule, we recompute the matrix product in Example 2.3.1.

Solution:

Here $A$ is $3 \times 3$ and $B$ is $3 \times 2$, so the product matrix $AB$ is defined and will be of size $3 \times 2$. Theorem 2.3.2 gives each entry of $AB$ as the dot product of the corresponding row of $A$ with the corresponding column of $B_{j}$ that is,
\begin{equation*}
AB = \left[ \begin{array}{rrr}
2 & 3 & 5 \\
1 & 4 & 7 \\
0 & 1 & 8
\end{array} \right] \left[ \begin{array}{rr}
8 & 9 \\
7 & 2 \\
6 & 1
\end{array} \right]= \left[ \arraycolsep=8pt \begin{array}{cc}
2 \cdot 8 + 3 \cdot 7 + 5 \cdot 6 & 2 \cdot 9 + 3 \cdot 2 + 5 \cdot 1 \\
1 \cdot 8 + 4 \cdot 7 + 7 \cdot 6 & 1 \cdot 9 + 4 \cdot 2 + 7 \cdot 1 \\
0 \cdot 8 + 1 \cdot 7 + 8 \cdot 6 & 0 \cdot 9 + 1 \cdot 2 + 8 \cdot 1
\end{array} \right] = \left[ \begin{array}{rr}
67 & 29 \\
78 & 24 \\
55 & 10
\end{array} \right]
\end{equation*}
Of course, this agrees with Example 2.3.1.

Solution:

The $(1, 3)$-entry of $AB$ is the dot product of row 1 of $A$ and column 3 of $B$ (highlighted in the following display), computed by multiplying corresponding entries and adding the results.

Similarly, the $(2, 4)$-entry of $AB$ involves row 2 of $A$ and column 4 of $B$.

Since $A$ is $2 \times 3$ and $B$ is $3 \times 4$, the product is $2 \times 4$.
\begin{equation*}
AB = \left[ \begin{array}{rrr}
3 & -1 & 2 \\
0 & 1 & 4
\end{array} \right] \left[ \begin{array}{rrrr}
2 & 1 & 6 & 0 \\
0 & 2 & 3 & 4 \\
-1 & 0 & 5 & 8
\end{array} \right] = \left[ \begin{array}{rrrr}
4 & 1 & 25 & 12 \\
-4 & 2 & 23 & 36
\end{array} \right]
\end{equation*}

Solution:

Here, $A$ is a $1 \times 3$ matrix and $B$ is a $3 \times 1$ matrix, so $A^{2}$ and $B^{2}$ are not defined. However, the compatibility rule reads
\begin{equation*}
\begin{array}{ccc}
\begin{array}{cc}
A & B\\
1 \times 3 & 3 \times 1
\end{array}
& \mbox{ and } &
\begin{array}{cc}
B & A \\
3 \times 1 & 1 \times 3
\end{array}
\end{array}
\end{equation*}
so both $AB$ and $BA$ can be formed and these are $1 \times 1$ and $3 \times 3$ matrices, respectively.

\begin{equation*}
AB = \left[ \begin{array}{rrr}
1 & 3 & 2
\end{array} \right] \left[ \begin{array}{r}
5 \\
6 \\
4
\end{array} \right] = \left[ \begin{array}{c} 1 \cdot 5 + 3 \cdot 6 + 2 \cdot 4 \end{array} \right] = \arraycolsep=1.5pt \left[ \begin{array}{c} 31 \end{array}\right]
\end{equation*}
\begin{equation*}
BA = \left[ \begin{array}{r}
5 \\
6 \\
4
\end{array} \right] \left[ \begin{array}{rrr}
1 & 3 & 2
\end{array} \right] = \left[ \begin{array}{rrr}
5 \cdot 1 & 5 \cdot 3 & 5 \cdot 2 \\
6 \cdot 1 & 6 \cdot 3 & 6 \cdot 2 \\
4 \cdot 1 & 4 \cdot 3 & 4 \cdot 2
\end{array} \right] = \left[ \begin{array}{rrr}
5 & 15 & 10 \\
6 & 18 & 12 \\
4 & 12 & 8
\end{array} \right
\end{equation*}

Unlike numerical multiplication, matrix products $AB$ and $BA$ need not be equal. In fact they need not even be the same size, as Example 2.3.5 shows. It turns out to be rare that $AB = BA$ (although it is by no means impossible), and $A$ and $B$ are said to commute when this happens.

Solution:
$A^{2} = \left[ \begin{array}{rr}
6 & 9 \\
-4 & -6
\end{array} \right] \left[ \begin{array}{rr}
6 & 9 \\
-4 & -6
\end{array} \right] = \left[ \begin{array}{rr}
0 & 0 \\
0 & 0
\end{array} \right]$, so $A^{2} = 0$ can occur even if $A \neq 0$. Next,
\begin{align*}
AB & = \left[ \begin{array}{rr}
6 & 9 \\
-4 & -6
\end{array} \right] \left[ \begin{array}{rr}
1 & 2 \\
-1 & 0
\end{array} \right] = \left[ \begin{array}{rr}
-3 & 12 \\
2 & -8
\end{array} \right] \\
BA & = \left[ \begin{array}{rr}
1 & 2 \\
-1 & 0
\end{array} \right] \left[ \begin{array}{rr}
6 & 9 \\
-4 & -6
\end{array} \right] = \left[ \begin{array}{rr}
-2 & -3 \\
-6 & -9
\end{array} \right]
\end{align*}
Hence $AB \neq BA$, even though $AB$ and $BA$ are the same size.

Solution:

These both follow from the dot product rule as the reader should verify. For a more formal proof, write $A = \left[ \begin{array}{rrrr}
\vec{a}_{1} & \vec{a}_{2} & \cdots & \vec{a}_{n}
\end{array} \right]$ where $\vec{a}_{j}$ is column $j$ of $A$. Then Definition 2.9 and Example 2.2.1 give
\begin{equation*}
IA = \left[ \begin{array}{rrrr}
I\vec{a}_{1} & I\vec{a}_{2} & \cdots & I\vec{a}_{n}
\end{array} \right] = \left[ \begin{array}{rrrr}
\vect{a}_{1} & \vect{a}_{2} & \cdots & \vect{a}_{n}
\end{array} \right] = A
\end{equation*}
If $\vec{e}_{j}$ denotes column $j$ of $I$, then $A\vec{e}_{j} = \vec{a}_{j}$ for each $j$ by Example 2.2.12. Hence Definition 2.9 gives:
\begin{equation*}
AI = A \left[ \begin{array}{rrrr}
\vec{e}_{1} & \vec{e}_{2} & \cdots & \vec{e}_{n}
\end{array} \right] = \left[ \begin{array}{rrrr}
A\vec{e}_{1} & A\vec{e}_{2} & \cdots & A\vec{e}_{n}
\end{array} \right] = \left[ \begin{array}{rrrr}
\vec{a}_{1} & \vec{a}_{2} & \cdots & \vec{a}_{n}
\end{array} \right] = A
\end{equation*}

The following theorem collects several results about matrix multiplication that are used everywhere in linear algebra.

Proof:

Condition (1) is Example 2.3.7; we prove (2), (4), and (6) and leave (3) and (5) as exercises.
1. If $C = \left[ \begin{array}{cccc}
\vec{c}_{1} & \vec{c}_{2} & \cdots & \vec{c}_{k}
\end{array} \right]$ in terms of its columns, then $BC = \left[ \begin{array}{cccc}
B\vec{c}_{1} & B\vec{c}_{2} & \cdots & B\vec{c}_{k}
\end{array} \right]$ by Definition 2.9, so
\begin{equation*}
\begin{array}{lllll}
A(BC) & = & \left[ \begin{array}{rrrr}
A(B\vec{c}_{1}) & A(B\vec{c}_{2}) & \cdots & A(B\vec{c}_{k})
\end{array} \right] & & \mbox{Definition 2.9} \\
& & & & \\
& = & \left[ \begin{array}{rrrr}
(AB)\vec{c}_{1} & (AB)\vec{c}_{2} & \cdots & (AB)\vec{c}_{k})
\end{array} \right] & & \mbox{Theorem 2.3.1} \\
& & & & \\
& = & (AB)C & & \mbox{Definition 2.9}
\end{array}
\end{equation*}

4. We know (Theorem 2.2.) that $(B + C)\vec{x} = B\vec{x} + C\vec{x}$ holds for every column $\vec{x}$. If we write $A = \left[ \begin{array}{rrrr}
\vec{a}_{1} & \vec{a}_{2} & \cdots & \vec{a}_{n}
\end{array} \right]$ in terms of its columns, we get
\begin{equation*}
\begin{array}{lllll}
(B + C)A & = & \left[ \begin{array}{rrrr}
(B + C)\vec{a}_{1} & (B + C)\vec{a}_{2} & \cdots & (B + C)\vec{a}_{n}
\end{array} \right] & & \mbox{Definition 2.9} \\
& & & & \\
& = & \left[ \begin{array}{rrrr}
B\vec{a}_{1} + C\vec{a}_{1} & B\vec{a}_{2} + C\vec{a}_{2} & \cdots & B\vec{a}_{n} + C\vec{a}_{n}
\end{array} \right] & & \mbox{Theorem 2.2.2} \\
& & & & \\
& = & \left[ \begin{array}{rrrr}
B\vec{a}_{1} & B\vec{a}_{2} & \cdots & B\vec{a}_{n}
\end{array} \right] + \left[ \begin{array}{rrrr}
C\vec{a}_{1} & C\vec{a}_{2} & \cdots & C\vec{a}_{n}
\end{array} \right] & & \mbox{Adding Columns} \\
& & & & \\
& = & BA + CA & & \mbox{Definition 2.9}
\end{array}
\end{equation*}

6. As in Section 2.1, write $A = [a_{ij}]$ and $B = [b_{ij}]$, so that $A^{T} = [a^\prime_{ij}]$ and $B^{T} = [b^\prime_{ij}]$ where $a^\prime_{ij} = a_{ji}$ and $b^\prime_{ji} = b_{ij}$ for all $i$ and $j$. If $c_{ij}$ denotes the $(i, j)$-entry of $B^{T}A^{T}$, then $c_{ij}$ is the dot product of row $i$ of $B^{T}$ with column $j$ of $A^{T}$. Hence
\begin{align*}
c_{ij} = b_{i1}^\prime a_{1j}^\prime + b_{i2}^\prime a_{2j}^\prime + \cdots + b_{im}^\prime a_{mj}^\prime &= b_{1i} a_{j1} + b_{2i} a_{j2} + \cdots + b_{mi} a_{jm} \\
&= a_{j1}b_{1i} + a_{j2}b_{2i} + \cdots + a_{jm}b_{mi}
\end{align*}
But this is the dot product of row $j$ of $A$ with column $i$ of $B$; that is, the $(j, i)$-entry of $AB$; that is, the $(i, j)$-entry of $(AB)^{T}$. This proves (6).

Property 2 in Theorem 2.3.3 is called the associative law of matrix multiplication. It asserts that the equation $A(BC) = (AB)C$ holds for all matrices (if the products are defined). Hence this product is the same no matter how it is formed, and so is written simply as $ABC$. This extends: The product $ABCD$ of four matrices can be formed several ways—for example, $(AB)(CD)$, $[A(BC)]D$, and $A[B(CD)]$—but the associative law implies that they are all equal and so are written as $ABCD$. A similar remark applies in general: Matrix products can be written unambiguously with no parentheses.

However, a note of caution about matrix multiplication must be taken: The fact that $AB$ and $BA$ need not be equal means that the order of the factors is important in a product of matrices. For example $ABCD$ and $ADCB$ may not be equal.

Warning:
If the order of the factors in a product of matrices is changed, the product matrix may change (or may not be defined). Ignoring this warning is a source of many errors by students of linear algebra!}

Properties 3 and 4 in Theorem 2.3.3 are called distributive laws. They assert that $A(B + C) = AB + AC$ and $(B + C)A = BA + CA$ hold whenever the sums and products are defined. These rules extend to more than two terms and, together with Property 5, ensure that many manipulations familiar from ordinary algebra extend to matrices. For example
\begin{align*}
A(2B – 3C + D – 5E) & = 2AB – 3AC + AD – 5AE \\
(A + 3C – 2D)B & = AB + 3CB – 2DB
\end{align*}
Note again that the warning is in effect: For example $A(B – C)$ need not equal $AB – CA$. These rules make possible a lot of simplification of matrix expressions.

Solution:

\begin{align*}
A(BC – CD) + A(C – B)D – AB(C – D) &= A(BC) – A(CD) + (AC-AB)D – (AB)C + (AB)D \\
&= ABC – ACD + ACD – ABD – ABC + ABD \\
&= 0
\end{align*}

Example 2.3.9 and Example 2.3.10 below show how we can use the properties in Theorem 2.3.2to deduce other facts about matrix multiplication. Matrices $A$ and $B$ are said to commute if $AB = BA$.

Solution:
Showing that $AB$ commutes with $C$ means verifying that $(AB)C = C(AB)$. The computation uses the associative law several times, as well as the given facts that $AC = CA$ and $BC = CB$.
\begin{equation*}
(AB)C = A(BC) = A(CB) = (AC)B = (CA)B = C(AB)
\end{equation*}

Solution:
The following always holds:
\begin{equation*} \tag{2.6}
(A – B)(A + B) = A(A + B) – B(A + B) = A^{2} + AB – BA -B^{2}
\end{equation*}
Hence if $AB = BA$, then $(A – B)(A + B) = A^{2} – B^{2}$ follows. Conversely, if this last equation holds, then equation (2.6 becomes
\begin{equation*}
A^{2} – B^{2} = A^{2} + AB – BA – B^{2}
\end{equation*}
This gives $0 = AB – BA$, and $AB = BA$ follows.

In Section 2.2 we saw (in Theorem 2.2.1 ) that every system of linear equations has the form
\begin{equation*}
A\vec{x} = \vec{b}
\end{equation*}
where $A$ is the coefficient matrix, $\vec{x}$ is the column of variables, and $\vec{b}$ is the constant matrix. Thus the system of linear equations becomes a single matrix equation. Matrix multiplication can yield information about such a system.

Solution:

Suppose that $\vec{x}$ is any solution to the system, so that $A\vec{x} = \vec{b}$. Multiply both sides of this matrix equation by $C$ to obtain, successively,
\begin{equation*}
C(A\vec{x}) = C\vec{b}, \quad (CA)\vec{x} = C\vec{b}, \quad I_{n}\vec{x} = C\vec{b}, \quad \vec{x} = C\vec{b}
\end{equation*}
This shows that if the system has a solution $\vec{x}$, then that solution must be $\vec{x} = C\vec{b}$, as required. But it does not guarantee that the system has a solution. However, if we write $\vec{x}_{1} = C\vec{b}$, then
\begin{equation*}
A\vec{x}_{1} = A(C\vec{b}) = (AC)\vec{b}
\end{equation*}
Thus $\vec{x}_{1} = C\vec{b}$ will be a solution if the condition $AC = I_{m}$ is satisfied.

The ideas in Example 2.3.11 lead to important information about matrices; this will be pursued in the next section.

2.4 Matrix Inverse

Three basic operations on matrices, addition, multiplication, and subtraction, are analogs for matrices of the same operations for numbers. In this section we introduce the matrix analog of numerical division.

To begin, consider how a numerical equation $ax = b$ is solved when $a$ and $b$ are known numbers. If $a = 0$, there is no solution (unless $b = 0$). But if $a \neq 0$, we can multiply both sides by the inverse $a^{-1} = \frac{1}{a}$ to obtain the solution $x = a^{-1}b$. Of course multiplying by $a^{-1}$ is just dividing by $a$, and the property of $a^{-1}$ that makes this work is that $a^{-1}a = 1$. Moreover, we saw in Section~\ref{sec:2_2} that the role that $1$ plays in arithmetic is played in matrix algebra by the identity matrix $I$. This suggests the following definition.

Note that only square matrices have inverses. Even though it is plausible that nonsquare matrices $A$ and $B$ could exist such that $AB = I_{m}$ and $BA = I_{n}$, where $A$ is $m \times n$ and $B$ is $n \times m$, we claim that this forces $n = m$. Indeed, if $m < n$ there exists a nonzero column $\vec{x}$ such that $A\vec{x} = \vec{0}$ (by Theorem 1.3.1), so $\vec{x} = I_{n}\vec{x} = (BA)\vec{x} = B(A\vec{x}) = B(\vec{0}) = \vec{0}$, a contradiction. Hence $m \geq n$. Similarly, the condition $AB = I_{m}$ implies that $n \geq m$. Hence $m = n$ so $A$ is square.}

Solution:

Compute $AB$ and $BA$.
\begin{equation*}
AB = \left[ \begin{array}{rr}
0 & 1 \\
1 & 1
\end{array} \right] \left[ \begin{array}{rr}
-1 & 1 \\
1 & 0
\end{array} \right] = \left[ \begin{array}{rr}
1 & 0 \\
0 & 1
\end{array} \right] \quad
BA = \left[ \begin{array}{rr}
-1 & 1 \\
1 & 0
\end{array} \right] \left[ \begin{array}{rr}
0 & 1 \\
1 & 1
\end{array} \right] = \left[ \begin{array}{rr}
1 & 0 \\
0 & 1
\end{array} \right]
\end{equation*}
Hence $AB = I = BA$, so $B$ is indeed an inverse of $A$.

Solution:
Let $B = \left[ \begin{array}{rr}
a & b \\
c & d
\end{array} \right]$
denote an arbitrary $2 \times 2$ matrix. Then
\begin{equation*}
AB = \left[ \begin{array}{rr}
0 & 0 \\
1 & 3
\end{array} \right] \left[ \begin{array}{rr}
a & b \\
c & d
\end{array} \right] = \left[ \begin{array}{cc}
0 & 0 \\
a + 3c & b + 3d
\end{array} \right]
\end{equation*}
so $AB$ has a row of zeros. Hence $AB$ cannot equal $I$ for any $B$.

The argument in Example 2.4.2 shows that no zero matrix has an inverse. But Example 2.4.2 also shows that, unlike arithmetic, it is possible for a nonzero matrix to have no inverse. However, if a matrix does have an inverse, it has only one.

Proof:

Since $B$ and $C$ are both inverses of $A$, we have $CA = I = AB$. Hence
\begin{equation*}
B = IB = (CA)B = C(AB) = CI = C
\end{equation*}

If $A$ is an invertible matrix, the (unique) inverse of $A$ is denoted $A^{-1}$. Hence $A^{-1}$ (when it exists) is a square matrix of the same size as $A$ with the property that
\begin{equation*}
AA^{-1} = I \quad \mbox{ and } \quad A^{-1}A = I
\end{equation*}
These equations characterize $A^{-1}$ in the following sense:

Inverse Criterion: If somehow a matrix $B$ can be found such that $AB = I$ and $BA = I$, then $A$ is invertible and $B$ is the inverse of $A$; in symbols, $B = A^{-1}$.}

This is a way to verify that the inverse of a matrix exists. Example 2.3.3 and Example 2.3.4 offer illustrations.

Solution:

We have $A^{2} = \left[ \begin{array}{rr}
0 & -1 \\
1 & -1
\end{array} \right] \left[ \begin{array}{rr}
0 & -1 \\
1 & -1
\end{array} \right] = \left[ \begin{array}{rr}
-1 & 1 \\
-1 & 0
\end{array} \right]$, and so
\begin{equation*}
A^{3} = A^{2}A = \left[ \begin{array}{rr}
-1 & 1 \\
-1 & 0
\end{array} \right] \left[ \begin{array}{rr}
0 & -1 \\
1 & -1
\end{array} \right] = \left[ \begin{array}{rr}
1 & 0 \\
0 & 1
\end{array} \right] = I
\end{equation*}
Hence $A^{3} = I$, as asserted. This can be written as $A^{2}A = I = AA^{2}$, so it shows that $A^{2}$ is the inverse of $A$. That is, $A^{-1} = A^{2} = \left[ \begin{array}{rr}
-1 & 1 \\
-1 & 0
\end{array} \right]$.

The next example presents a useful formula for the inverse of a $2 \times 2$ matrix $A = \left[ \begin{array}{cc}
a & b \\
c & d
\end{array} \right]$ when it exists. To state it, we define the $\textbf{determinant}$ $\func{det }A$ and the $\textbf{adjugate}$  $\func{adj }A$ of the matrix $A$ as follows:
\begin{equation*}
\func{det}\left[ \begin{array}{cc}
a & b \\
c & d
\end{array} \right] = ad – bc, \quad \mbox{ and } \quad \func{adj} \left[ \begin{array}{cc}
a & b \\
c & d
\end{array} \right] = \left[ \begin{array}{rr}
d & -b \\
-c & a
\end{array} \right]
\end{equation*}

Solution:
For convenience, write $e = \func{det } A = ad – bc$ and
$B = \func{adj } A = \left[ \begin{array}{rr}
d & -b \\
-c & a
\end{array} \right]$. Then $AB = eI = BA$ as the reader can verify. So if $e \neq 0$, scalar multiplication by $\frac{1}{e}$ gives
\begin{equation*}
A(\frac{1}{e}B) = I = (\frac{1}{e}B)A
\end{equation*}
Hence $A$ is invertible and $A^{-1} = \frac{1}{e}B$. Thus it remains only to show that if $A^{-1}$ exists, then $e \neq 0$.

We prove this by showing that assuming $e = 0$ leads to a contradiction. In fact, if $e = 0$, then $AB = eI = 0$, so left multiplication by $A^{-1}$ gives $A^{-1}AB = A^{-1}0$; that is, $IB = 0$, so $B = 0$. But this implies that $a$, $b$, $c$, and $d$ are all zero, so $A = 0$, contrary to the assumption that $A^{-1}$ exists.

As an illustration, if $A = \left[ \begin{array}{rr}
2 & 4 \\
-3 & 8
\end{array} \right]$
then $\func{det } A = 2 \cdot 8 – 4 \cdot (-3) = 28 \neq 0$. Hence $A$ is invertible and $A^{-1} = \frac{1}{\func{det } A} \func{adj } A = \frac{1}{28} \left[ \begin{array}{rr}
8 &-4 \\
3 & 2
\end{array} \right]$, as the reader is invited to verify.

The determinant and adjugate will be defined in Chapter 3 for any square matrix, and the conclusions in Example 2.4.4 will be proved in full generality.

Inverse and Linear systems

Matrix inverses can be used to solve certain systems of linear equations. Recall that a $\textit{system}$ of linear equations can be written as a $\textit{single}$ matrix equation
\begin{equation*}
A\vec{x} = \vec{b}
\end{equation*}
where $A$ and $\vec{b}$ are known and $\vec{x}$ is to be determined. If $A$ is invertible, we multiply each side of the equation on the left by $A^{-1}$ to get
\begin{align*}
A^{-1}A\vec{x} &= A^{-1}\vec{b} \\
I\vec{x} &= A^{-1}\vec{b} \\
\vec{x} &= A^{-1}\vec{b}
\end{align*}
This gives the solution to the system of equations (the reader should verify that $\vec{x} = A^{-1}\vec{b}$ really does satisfy $A\vec{x} = \vec{b}$). Furthermore, the argument shows that if $\vec{x}$ is $\textit{any} $solution, then necessarily $\vec{x} = A^{-1}\vec{b}$, so the solution is unique. Of course the technique works only when the coefficient matrix $A$ has an inverse. This proves Theorem 2.4.2.

Solution:

In matrix form this is $A\vec{x} = \vec{b}$ where $A = \left[ \begin{array}{rr}
5 & -3 \\
7 & 4
\end{array} \right]$, $\vec{x} = \left[ \begin{array}{c}
x_{1} \\
x_{2}
\end{array} \right]$, and $\vec{b} = \left[ \begin{array}{r}
-4 \\
8
\end{array} \right]$. Then $\func{det } A = 5 \cdot 4 – (-3) \cdot 7 = 41$, so $A$ is invertible and $A^{-1} = \frac{1}{41} \left[ \begin{array}{rr}
4 & 3 \\
-7 & 5
\end{array} \right]$
by Example 2.4.4. Thus Theorem 2.4.2 gives
\begin{equation*}
\vec{x} = A^{-1}\vec{b} = \frac{1}{41} \left[ \begin{array}{rr}
4 & 3 \\
-7 & 5
\end{array} \right] \left[ \begin{array}{r}
-4 \\
8
\end{array} \right] = \frac{1}{41} \left[ \begin{array}{r}
8 \\
68
\end{array} \right]
\end{equation*}
so the solution is $x_{1} = \frac{8}{41}$ and $x_{2} = \frac{68}{41}$.

An inversion method

If a matrix $A$ is $n \times n$ and invertible, it is desirable to have an efficient technique for finding the inverse. The following procedure will be justified in Section 2.5.

Solution:

Apply elementary row operations to the double matrix
\begin{equation*}
\left[ \begin{array}{rrr}
A & I
\end{array} \right] = \left[ \begin{array}{rrr|rrr}
2 & 7 & 1 & 1 & 0 & 0 \\
1 & 4 & -1 & 0 & 1 & 0 \\
1 & 3 & 0 & 0 & 0 & 1
\end{array} \right]
\end{equation*}
so as to carry $A$ to $I$. First interchange rows 1 and 2.
\begin{equation*}
\left[ \begin{array}{rrr|rrr}
1 & 4 & -1 & 0 & 1 & 0 \\
2 & 7 & 1 & 1 & 0 & 0 \\
1 & 3 & 0 & 0 & 0 & 1
\end{array} \right]
\end{equation*}
Next subtract $2$ times row 1 from row 2, and subtract row 1 from row 3.
\begin{equation*}
\left[ \begin{array}{rrr|rrr}
1 & 4 & -1 & 0 & 1 & 0 \\
0 & -1 & 3 & 1 & -2 & 0 \\
0 & -1 & 1 & 0 & -1 & 1
\end{array} \right]
\end{equation*}
Continue to reduced row-echelon form.
\begin{equation*}
\left[ \begin{array}{rrr|rrr}
1 & 0 & 11 & 4 & -7 & 0 \\
0 & 1 & -3 & -1 & 2 & 0 \\
0 & 0 & -2 & -1 & 1 & 1
\end{array} \right]
\end{equation*}
\begin{equation*}
\left[ \def\arraystretch{1.5} \begin{array}{rrr|rrr}
1 & 0 & 0 & \frac{-3}{2} & \frac{-3}{2} & \frac{11}{2} \\
0 & 1 & 0 & \frac{1}{2} & \frac{1}{2} & \frac{-3}{2} \\
0 & 0 & 1 & \frac{1}{2} & \frac{-1}{2} & \frac{-1}{2}
\end{array} \right]
\end{equation*}
Hence $A^{-1} = \frac{1}{2} \left[ \begin{array}{rrr}
-3 & -3 & 11 \\
1 & 1 & -3 \\
1 & -1 & -1
\end{array} \right]$, as is readily verified.

Given any $n \times n$ matrix $A$, Theorem 1.2.1 shows that $A$ can be carried by elementary row operations to a matrix $R$ in reduced row-echelon form. If $R = I$, the matrix $A$ is invertible (this will be proved in the next section), so the algorithm produces $A^{-1}$. If $R \neq I$, then $R$ has a row of zeros (it is square), so no system of linear equations $A\vecx} = \vect{b}$ can have a unique solution. But then $A$ is not invertible by Theorem 2.4.2. Hence, the algorithm is effective in the sense conveyed in Theorem 2.4.3.

Properties of inverses

The following properties of an invertible matrix are used everywhere.

Solution:

Given the equation $AB = AC$, left multiply both sides by $A^{-1}$ to obtain $A^{-1}AB = A^{-1}AC$. Thus $IB = IC$, that is $B = C$. This proves (1) and the proof of (2) is left to the reader.

Properties (1) and (2) in Example 2.4.7 are described by saying that an invertible matrix can be “left cancelled” and “right cancelled”, respectively. Note however that “mixed” cancellation does not hold in general: If $A$ is invertible and $AB = CA$, then $B$ and $C$ may $\textit{not} $ be equal, even if both are $2 \times 2$. Here is a specific example:
\begin{equation*}
A = \left[ \begin{array}{rr}
1 & 1 \\
0 & 1
\end{array} \right],\ B = \left[ \begin{array}{rr}
0 & 0 \\
1 & 2
\end{array} \right], C = \left[ \begin{array}{rr}
1 & 1 \\
1 & 1
\end{array} \right]
\end{equation*}
Sometimes the inverse of a matrix is given by a formula. Example 2.4.4 is one illustration; Example 2.4.8 and Example 2.4.9 provide two more. The idea is the $\textit{Inverse Criterion}$: If a matrix $B$ can be found such that $AB = I = BA$, then $A$ is invertible and $A^{-1} = B$.

Solution:

$A^{-1}$ exists (by assumption). Its transpose $(A^{-1})^{T}$ is the candidate proposed for the inverse of $A^{T}$. Using the inverse criterion, we test it as follows:
\begin{equation*}
\arraycolsep=1pt
\begin{array}{lllllll}
A^{T}(A^{-1})^{T} & = & (A^{-1}A)^{T} & = & I^{T} & = & I \\
(A^{-1})^{T}A^{T} & = & (AA^{-1})^{T} & = & I^{T} & = & I
\end{array}
\end{equation*}
Hence $(A^{-1})^{T}$ is indeed the inverse of $A^{T}$; that is, $(A^{T})^{-1} = (A^{-1})^{T}$.

Solution:

We are given a candidate for the inverse of $AB$, namely $B^{-1}A^{-1}$. We test it as follows:
\begin{align*}
(B^{-1}A^{-1})(AB) &= B^{-1}(A^{-1}A)B = B^{-1}IB = B^{-1}B = I \\
(AB)(B^{-1}A^{-1}) &= A(BB^{-1})A^{-1} = AIA^{-1} = AA^{-1} = I
\end{align*}
Hence $B^{-1}A^{-1}$ is the inverse of $AB$; in symbols, $(AB)^{-1} = B^{-1}A^{-1}$.

We now collect several basic properties of matrix inverses for reference.

Proof:
1. This is an immediate consequence of the fact that $I^{2} = I$.

2. The equations $AA^{-1} = I = A^{-1}A$ show that $A$ is the inverse of $A^{-1}$; in symbols, $(A^{-1})^{-1} = A$.

3. This is Example 2.4.9.

4. Use induction on $k$. If $k = 1$, there is nothing to prove, and if $k = 2$, the result is property 3. If $k > 2$, assume inductively that $(A_1A_2 \cdots A_{k-1})^{-1} = A_{k-1}^{-1} \cdots A_2^{-1}A_1^{-1}$. We apply this fact together with property 3 as follows:
\begin{align*}
\left[ A_{1}A_{2} \cdots A_{k-1}A_{k} \right]^{-1}
&= \left[ \left(A_{1}A_{2} \cdots A_{k-1}\right)A_{k} \right]^{-1} \\
&= A_{k}^{-1}\left(A_{1}A_{2} \cdots A_{k-1}\right)^{-1} \\
&= A_{k}^{-1}\left(A_{k-1}^{-1} \cdots A_{2}^{-1}A_{1}^{-1}\right)
\end{align*}
So the proof by induction is complete.

5. This is property 4 with $A_{1} = A_{2} = \cdots = A_{k} = A$.

6. The readers are invited to verify it.

7. This is Example 2.4.8.

The reversal of the order of the inverses in properties 3 and 4 of Theorem 2.4.4 is a consequence of the fact that matrix multiplication is not
commutative. Another manifestation of this comes when matrix equations are dealt with. If a matrix equation $B = C$ is given, it can be $\textit{left-multiplied}$ by a matrix $A$ to yield $AB = AC$. Similarly, $\textit{right-multiplication}$ gives $BA = CA$. However, we cannot mix the two: If $B = C$, it need $\textit{not} $be the case that $AB = CA$ even if $A$ is invertible, for example, $A = \left[ \begin{array}{rr}
1 & 1 \\
0 & 1
\end{array} \right]$, $B = \left[ \begin{array}{rr}
0 & 0 \\
1 & 0
\end{array} \right] = C$.

Part 7 of Theorem 2.4.4 together with the fact that $(A^{T})^{T} = A$ gives

Solution:

By Theorem 2.4.2 (2) and Example 2.4.4, we have
\begin{equation*}
(A^{T} – 2I) = \left[ \left(A^{T} – 2I\right)^{-1} \right]^{-1} = \left[ \begin{array}{rr}
2 & 1 \\
-1 & 0
\end{array} \right]^{-1} = \left[ \begin{array}{rr}
0 & -1 \\
1 & 2
\end{array} \right]
\end{equation*}
Hence $A^{T} = 2I + \left[ \begin{array}{rr}
0 & -1 \\
1 & 2
\end{array} \right] = \left[ \begin{array}{rr}
2 & -1 \\
1 & 4
\end{array} \right]$, so $A = \left[ \begin{array}{rr}
2 & 1 \\
-1 & 4
\end{array} \right]$
by Theorem 2.4.4(7).

The following important theorem collects a number of conditions all equivalent to invertibility. It will be referred to frequently below.

Proof:

We show that each of these conditions implies the next, and that (5) implies (1).

(1) $\Rightarrow$ (2). If $A^{-1}$ exists, then $A\vec{x} = \vec{0}$ gives $\vec{x} = I_{n}\vec{x} = A^{-1}A\vec{x} = A^{-1}\vect{0} = \vect{0}$.

(2) $\Rightarrow$ (3). Assume that (2) is true. Certainly $A \to R$ by row operations where $R$ is a reduced, row-echelon matrix. It suffices to show that $R = I_{n}$. Suppose that this is not the case. Then $R$ has a row of zeros (being square). Now consider the augmented matrix $\left[ \begin{array}{c|c}
A & \vec{0}
\end{array} \right]$ of the system $A\vec{x} = \vec{0}$. Then $\left[ \begin{array}{c|c}
A & \vec{0}
\end{array} \right] \to \left[ \begin{array}{c|c}
R & \vec{0}
\end{array} \right]$ is the reduced form, and $\left[ \begin{array}{c|c}
R & \vec{0}
\end{array} \right]$ also has a row of zeros. Since $R$ is square there must be at least one nonleading variable, and hence at least one parameter. Hence the system $A\vec{x} = \vec{0}$ has infinitely many solutions, contrary to (2). So $R = I_{n}$ after all.

(3) $\Rightarrow$ (4). Consider the augmented matrix $\left[ \begin{array}{c|c}
A & \vec{b}
\end{array} \right]$ of the system $A\vec{x} = \vec{b}$. Using (3), let $A \to I_{n}$ by a sequence of row operations. Then these same operations carry $\left[ \begin{array}{c|c}
A & \vec{b}
\end{array} \right] \to \left[ \begin{array}{c|c}
I_{n} & \vec{c}
\end{array} \right]$ for some column $\vec{c}$. Hence the system $A\vec{x} = \vec{b}$ has a solution (in fact unique) by gaussian elimination. This proves (4).

(4) $\Rightarrow$ (5). Write $I_{n} = \left[ \begin{array}{cccc}
\vec{e}_{1} & \vec{e}_{2} & \cdots & \vec{e}_{n}
\end{array} \right]$ where $\vec{e}_{1}, \vec{e}_{2}, \dots, \vec{e}_{n}$ are the columns of $I_{n}$. For each \newline $j = 1, 2, \dots, n$, the system $A\vec{x} = \vec{e}_{j}$ has a solution $\vec{c}_{j}$ by (4), so $A\vec{c}_{j} = \vec{e}_{j}$. Now let $C = \left[ \begin{array}{cccc}
\vec{c}_{1} & \vec{c}_{2} & \cdots & \vec{c}_{n}
\end{array} \right]$ be the $n \times n$ matrix with these matrices $\vec{c}_{j}$ as its columns. Then Definition 2.9 gives (5):
\begin{equation*}
AC = A \left[ \begin{array}{cccc}
\vec{c}_{1} & \vec{c}_{2} & \cdots & \vec{c}_{n}
\end{array} \right] = \left[ \begin{array}{cccc}
A\vec{c}_{1} & A\vec{c}_{2} & \cdots & A\vec{c}_{n}
\end{array} \right] = \left[ \begin{array}{cccc}
\vec{e}_{1} & \vec{e}_{2} & \cdots & \vec{e}_{n}
\end{array} \right] = I_{n}
\end{equation*}
(5) $\Rightarrow$ (1). Assume that (5) is true so that $AC = I_{n}$ for some matrix $C$. Then $C\vec{x} = 0$ implies $\vec{x} = \vect{0}$ (because $\vec{x} = I_{n}\vec{x} = AC\vec{x} = A\vec{0} = \vec{0}$). Thus condition (2) holds for the matrix $C$ rather than $A$. Hence the argument above that (2) $\Rightarrow$ (3) $\Rightarrow$ (4) $\Rightarrow$ (5) (with $A$ replaced by $C$) shows that a matrix $C^\prime$ exists such that $CC^\prime = I_{n}$. But then
\begin{equation*}
A = AI_{n} = A(CC^\prime) = (AC)C^\prime = I_{n}C^\prime = C^\prime
\end{equation*}
Thus $CA = CC^\prime = I_{n}$ which, together with $AC = I_{n}$, shows that $C$ is the inverse of $A$. This proves (1).

The proof of (5) $\Rightarrow$ (1) in Theorem 2.4.5 shows that if $AC = I$ for square matrices, then necessarily $CA = I$, and hence that $C$ and $A$ are inverses of each other. We record this important fact for reference.

Here is a quick way to remember Corollary 2.4.1. If $A$ is a square matrix, then

1. If $AC=I$ then $C=A^{-1}$.
2. If $CA=I$ then $C=A^{-1}$.

Observe that Corollary 2.4.1 is false if $A$ and $C$ are not square matrices. For example, we have
\begin{equation*}
\left[ \begin{array}{rrr}
1 & 2 & 1 \\
1 & 1 & 1
\end{array} \right] \left[ \begin{array}{rr}
-1 & 1 \\
1 & -1 \\
0 & 1
\end{array} \right] = I_{2} \quad \mbox{ but }
\left[ \begin{array}{rr}
-1 & 1 \\
1 & -1 \\
0 & 1
\end{array} \right] \left[ \begin{array}{rrr}
1 & 2 & 1 \\
1 & 1 & 1
\end{array} \right] \neq I_{3}
\end{equation*}
In fact, it can be verified that if $AB = I_{m}$ and $BA = I_{n}$, where $A$ is $m \times n$ and $B$ is $n \times m$, then $m = n$ and $A$ and $B$ are (square) inverses of each other.

An $n \times n$ matrix $A$ has $\func{rank }n$ if and only if (3) of Theorem 2.4.5 holds. Hence