4.3. Linear Transformations

In this section, we will be using symbols $\mathbb{V}$ and $\mathbb{W}$ to represent arbitrary vector spaces over a field $\mathbb{F}$. Unless otherwise specified, the two vector spaces won’t be related in any way. Following results can be restated for more general situations where $\mathbb{V}$ and $\mathbb{W}$ are defined over different fields, but we will assume that they are defined over the same field $\mathbb{F}$ for simplicity of discourse.

4.3.1. Operators

Operators are mappings from one vector space to another space. Normally, they are total functions.

In this section, we introduce different types of operators between vector spaces. Some operators are relevant only for real vector spaces.

Definition 4.44 (Homogeneous operator)

Let $\mathbb{V}$ and $\mathbb{W}$ be vector spaces (over some field $\mathbb{F}$). An operator $T:\mathbb{V}\to \mathbb{W}$ is called homogeneous if for every $\mathbf{x}\in \mathbb{V}$ and for every $\lambda \in \mathbb{F}$

$$T(\lambda \mathbf{x})=\lambda T(\mathbf{x}).$$

Definition 4.45 (Positively homogeneous operator)

Let $\mathbb{V}$ and $\mathbb{W}$ be real vector spaces (on field $\mathbb{R}$). An operator $T:\mathbb{V}\to \mathbb{W}$ is called positively homogeneous if for every $\mathbf{x}\in \mathbb{V}$ and for every $\lambda \in {\mathbb{R}}_{++}$

$$T(\lambda \mathbf{x})=\lambda T(\mathbf{x}).$$

Definition 4.46 (Additive operator)

Let $\mathbb{V}$ and $\mathbb{W}$ be vector spaces. An operator $T:\mathbb{V}\to \mathbb{W}$ is called additive if for every $\mathbf{x},\mathbf{y}\in \mathbb{V}$

$$T(\mathbf{x}+\mathbf{y})=T(\mathbf{x})+T(\mathbf{y}).$$

4.3.2. Linear Transformations

A linear operator is additive and homogeneous.

Definition 4.47 (Linear transformation)

We call a map $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ a linear transformation from $\mathbb{V}$ to $\mathbb{W}$ if for all $\mathbf{x},\mathbf{y}\in \mathbb{V}$ and $\alpha \in \mathbb{F}$, we have

1. $\mathbb{T}(\mathbf{x}+\mathbf{y})=\mathbb{T}(\mathbf{x})+\mathbb{T}(\mathbf{y})$ and

2. $\mathbb{T}(\alpha \mathbf{x})=\alpha \mathbb{T}(\mathbf{x})$

A linear transformation is also known as a linear map or a linear operator.

4.3.3. Properties

Proposition 4.3 (Zero maps to zero)

If $\mathbb{T}$ is linear then $\mathbb{T}(\mathbf{0})=\mathbf{0}$.

This is straightforward since

$$\mathbb{T}(\mathbf{0}+\mathbf{0})=\mathbb{T}(\mathbf{0})+\mathbb{T}(\mathbf{0})⟹\mathbb{T}(\mathbf{0})=\mathbb{T}(\mathbf{0})+\mathbb{T}(\mathbf{0})⟹\mathbb{T}(\mathbf{0})=\mathbf{0}.$$

Proposition 4.4

$\mathbb{T}$ is linear $⟺\mathbb{T}(\alpha \mathbf{x}+\mathbf{y})=\alpha \mathbb{T}(\mathbf{x})+\mathbb{T}(\mathbf{y})\mathrm{\forall }\mathbf{x},\mathbf{y}\in \mathbb{V},\alpha \in \mathbb{F}$

Proof. Assuming $\mathbb{T}$ to be linear we have

$$\mathbb{T}(\alpha \mathbf{x}+\mathbf{y})=\mathbb{T}(\alpha \mathbf{x})+\mathbb{T}(\mathbf{y})=\alpha \mathbb{T}(\mathbf{x})+\mathbb{T}(\mathbf{y}).$$

Now for the converse, assume

$$\mathbb{T}(\alpha \mathbf{x}+\mathbf{y})=\alpha \mathbb{T}(\mathbf{x})+\mathbb{T}(\mathbf{y})\mathrm{\forall }\mathbf{x},\mathbf{y}\in \mathbb{V},\alpha \in \mathbb{F}.$$

Choosing both $\mathbf{x}$ and $\mathbf{y}$ to be \bzero and $\alpha =1$ we get

$$\mathbb{T}(\mathbf{0}+\mathbf{0})=\mathbb{T}(\mathbf{0})+\mathbb{T}(\mathbf{0})⟹\mathbb{T}(\mathbf{0})=\mathbf{0}.$$

Choosing $\mathbf{y}=\mathbf{0}$ we get

$$\mathbb{T}(\alpha \mathbf{x}+\mathbf{0})=\alpha \mathbb{T}(\mathbf{x})+\mathbb{T}(\mathbf{0})=\alpha \mathbb{T}(\mathbf{x}).$$

Choosing $\alpha =1$ we get

$$\mathbb{T}(\mathbf{x}+\mathbf{y})=\mathbb{T}(\mathbf{x})+\mathbb{T}(\mathbf{y}).$$

Thus, $\mathbb{T}$ is a linear transformation.

Proposition 4.5

If $\mathbb{T}$ is linear then $\mathbb{T}(\mathbf{x}-\mathbf{y})=\mathbb{T}(\mathbf{x})-\mathbb{T}(\mathbf{y})$.

$$\mathbb{T}(\mathbf{x}-\mathbf{y})=\mathbb{T}(\mathbf{x}+(-1)\mathbf{y})=\mathbb{T}(\mathbf{x})+\mathbb{T}((-1)\mathbf{y})=\mathbb{T}(\mathbf{x})+(-1)\mathbb{T}(\mathbf{y})=\mathbb{T}(\mathbf{x})-\mathbb{T}(\mathbf{y}).$$

Proposition 4.6 (Linear transformation preserves linear combinations)

$\mathbb{T}$ is linear $⟺$ for ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n\in \mathbb{V}$ and ${\alpha }_1,\dots ,{\alpha }_n\in \mathbb{F}$,

$$\mathbb{T}\left(\sum _{i=1}^n{\alpha }_i{\mathbf{x}}_i\right)=\sum _{i=1}^n{\alpha }_i\mathbb{T}({\mathbf{x}}_i).$$

We can use mathematical induction to prove this.

Some special linear transformations need mention.

Definition 4.48 (Identity transformation)

The identity transformation ${\mathrm{I}}_{\mathbb{V}}:\mathbb{V}\to \mathbb{V}$ is defined as

$${\mathrm{I}}_{\mathbb{V}}(x)=x,\mathrm{\forall }x\in \mathbb{V}.$$

Definition 4.49

The zero transformation $\mathbf{O}:\mathbb{V}\to \mathbb{W}$ is defined as

$$\mathbf{O}(x)=\mathbf{0},\mathrm{\forall }x\in \mathbb{V}.$$

Note that $\mathbf{0}$ on the R.H.S. is the zero vector or $\mathbb{W}$.

In this definition $0$ is taking up multiple meanings: a linear transformation from $\mathbb{V}$ to $\mathbb{W}$ which maps every vector in $\mathbb{V}$ to the $\mathbf{0}$ vector in $\mathbb{W}$.

From the context usually it should be obvious whether we are talking about $0\in \mathbb{F}$ or $\mathbf{0}\in \mathbb{V}$ or $\mathbf{0}\in \mathbb{W}$ or $\mathbf{O}$ as a linear transformation from $\mathbb{V}$ to $\mathbb{W}$.

4.3.4. Null Space and Range

Definition 4.50 (Null space / Kernel)

The null space or kernel of a linear transformation $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ denoted by $\mathcal{N}(\mathbb{T})$ or $\ker (\mathbb{T})$ is defined as

$$\ker (\mathbb{T})=\mathcal{N}(\mathbb{T})\triangleq \{\mathbf{x}\in \mathbb{V}|\mathbb{T}(\mathbf{x})=\mathbf{0}\}.$$

Theorem 4.23

The null space of a linear transformation $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ is a subspace of $\mathbb{V}$.

Proof. Let ${\mathbf{v}}_1,{\mathbf{v}}_2\in \ker (\mathbb{T})$. Then

$$\mathbb{T}(\alpha {\mathbf{v}}_1+{\mathbf{v}}_2)=\alpha \mathbb{T}({\mathbf{v}}_1)+\mathbb{T}({\mathbf{v}}_2)=\alpha \mathbf{0}+\mathbf{0}=\mathbf{0}.$$

Thus $\alpha {\mathbf{v}}_1+{\mathbf{v}}_2\in \ker (\mathbb{T})$. Thus $\ker (\mathbb{T})$ is a subspace of $\mathbb{V}$.

Definition 4.51

The range or image of a linear transformation $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ denoted by $\mathrm{R}(\mathbb{T})$ or $\mathrm{im}(\mathbb{T})$ is defined as

$$\mathrm{R}(\mathbb{T})=\mathrm{im}(\mathbb{T})\triangleq \{\mathbb{T}(\mathbf{x})\mathrm{\forall }\mathbf{x}\in \mathbb{V}\}.$$

We note that $\mathrm{im}(\mathbb{T})\subseteq \mathbb{W}$.

Theorem 4.24

The image of a linear transformation $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ is a subspace of $\mathbb{W}$.

Proof. Let ${\mathbf{w}}_1,{\mathbf{w}}_2\in \mathrm{im}(\mathbb{T})$. Then there exist ${\mathbf{v}}_1,{\mathbf{v}}_2\in \mathbb{V}$ such that

$${\mathbf{w}}_1=\mathbb{T}({\mathbf{v}}_1);{\mathbf{w}}_2=\mathbb{T}({\mathbf{v}}_2).$$

Thus

$$\alpha {\mathbf{w}}_1+{\mathbf{w}}_2=\alpha \mathbb{T}({\mathbf{v}}_1)+\mathbb{T}({\mathbf{v}}_2)=\mathbb{T}(\alpha {\mathbf{v}}_1+{\mathbf{v}}_2).$$

Thus $\alpha {\mathbf{w}}_1+{\mathbf{w}}_2\in \mathrm{im}(\mathbb{T})$. Hence $\mathrm{im}(\mathbb{T})$ is a subspace of $\mathbb{W}$.

Theorem 4.25

Let $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ be a linear transformation. Assume $\mathbb{V}$ to be finite dimensional. Let $\mathcal{B}=\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_n\}$ be some basis of $\mathbb{V}$. Then

$$\mathrm{im}(\mathbb{T})=\mathrm{span}\mathbb{T}(\mathcal{B})=\mathrm{span}\{\mathbb{T}({\mathbf{v}}_1),\mathbb{T}({\mathbf{v}}_2),\dots ,\mathbb{T}({\mathbf{v}}_n)\}.$$

i.e., the image of a basis of $\mathbb{V}$ under a linear transformation $\mathbb{T}$ spans the range of the transformation.

Proof. Let $\mathbf{w}$ be some arbitrary vector in $\mathrm{im}(\mathbb{T})$. Then there exists $\mathbf{v}\in \mathbb{V}$ such that $\mathbf{w}=\mathbb{T}(\mathbf{v})$. Now

$$\mathbf{v}=\sum _{i=1}^n{c}_i{\mathbf{v}}_i$$

since $\mathcal{B}$ forms a basis for $\mathbb{V}$. Thus,

$$\mathbf{w}=\mathbb{T}(\mathbf{v})=\mathbb{T}(\sum _{i=1}^n{c}_i{\mathbf{v}}_i)=\sum _{i=1}^n{c}_i(\mathbb{T}({\mathbf{v}}_i)).$$

This means that $\mathbf{w}\in \mathrm{span}\mathbb{T}(\mathcal{B})$.

Definition 4.52 (Nullity)

For vector spaces $\mathbb{V}$ and $\mathbb{W}$ and linear transformation $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ if $\ker \mathbb{T}$ is finite dimensional then nullity of $\mathbb{T}$ is defined as

$$\mathrm{nullity}\mathbb{T}\triangleq \dim \ker \mathbb{T};$$

i.e., the dimension of the null space or kernel of $\mathbb{T}$.

Definition 4.53

For vector spaces $\mathbb{V}$ and $\mathbb{W}$ and linear $\mathbb{T}:\mathbb{V}\to \mathbb{W}$, if $\mathrm{range}\mathbb{T}$ is finite dimensional then rank of $\mathbb{T}$ is defined as

$$\mathrm{rank}\mathbb{T}\triangleq \dim \mathrm{range}\mathbb{T};$$

i.e., the dimension of the range or image of $\mathbb{T}$.

Theorem 4.26 (Dimension theorem)

For vector spaces $\mathbb{V}$ and $\mathbb{W}$ and linear $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ if $\mathbb{V}$ is finite dimensional, then

$$\dim \mathbb{V}=\mathrm{nullity}\mathbb{T}+\mathrm{rank}\mathbb{T}.$$

This is known as dimension theorem.

Theorem 4.27

For vector spaces $\mathbb{V}$ and $\mathbb{W}$ and linear $\mathbb{T}:\mathbb{V}\to \mathbb{W}$, $\mathbb{T}$ is injective if and only if $\ker (\mathbb{T})=\{\mathbf{0}\}$.

Proof. If $\mathbb{T}$ is injective, then

$${\mathbf{v}}_1\ne {\mathbf{v}}_2⟹T({\mathbf{v}}_1)\ne T({\mathbf{v}}_2)$$

Let $\mathbf{v}\ne \mathbf{0}$. Now $\mathbb{T}(\mathbf{0})=\mathbf{0}⟹\mathbb{T}(v)\ne \mathbf{0}$ since $\mathbb{T}$ is one-one. Thus $\ker \mathbb{T}=\{\mathbf{0}\}$.

For the converse, let us assume that $\ker \mathbb{T}=\{\mathbf{0}\}$. Let ${\mathbf{v}}_1,{\mathbf{v}}_2\in \mathbb{V}$ be two vectors such that they have the same image. Then,

$$\begin{array}{rl}& \mathbb{T}({\mathbf{v}}_1)=\mathbb{T}({\mathbf{v}}_2)\\ ⟹& \mathbb{T}({\mathbf{v}}_1-{\mathbf{v}}_2)=\mathbf{0}\\ ⟹& {\mathbf{v}}_1-{\mathbf{v}}_2\in \ker \mathbb{T}\\ ⟹& {\mathbf{v}}_1-{\mathbf{v}}_2=\mathbf{0}\\ ⟹& {\mathbf{v}}_1={\mathbf{v}}_2.\end{array}$$

Thus $\mathbb{T}$ is injective.

Theorem 4.28 (Bijective transformation characterization)

For vector spaces $\mathbb{V}$ and $\mathbb{W}$ of equal finite dimensions and linear $\mathbb{T}:\mathbb{V}\to \mathbb{W}$, the following are equivalent.

1. $\mathbb{T}$ is injective.

2. $\mathbb{T}$ is surjective.

3. $\mathrm{rank}\mathbb{T}=\dim \mathbb{V}$.

Proof. From (1) to (2)

Let $\mathcal{B}=\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots {\mathbf{v}}_n\}$ be some basis of $\mathbb{V}$ with $\dim \mathbb{V}=n$.

Let us assume that $\mathbb{T}(\mathcal{B})$ are linearly dependent. Thus, there exists a linear relationship

$$\sum _{i=1}^n{\alpha }_i\mathbb{T}({\mathbf{v}}_i)=\mathbf{0}$$

where ${\alpha }_i$ are not all 0. Now

$$\begin{array}{rl}& \sum _{i=1}^n{\alpha }_i\mathbb{T}({\mathbf{v}}_i)=\mathbf{0}\\ ⟹& \mathbb{T}\left(\sum _{i=1}^n{\alpha }_i{\mathbf{v}}_i\right)=\mathbf{0}\\ ⟹& \sum _{i=1}^n{\alpha }_i{\mathbf{v}}_i\in \ker \mathbb{T}\\ ⟹& \sum _{i=1}^n{\alpha }_i{\mathbf{v}}_i=\mathbf{0}\end{array}$$

since $\mathbb{T}$ is injective (see Theorem 4.27). This means that ${\mathbf{v}}_i$ are linearly dependent. This contradicts our assumption that $\mathcal{B}$ is a basis for $\mathbb{V}$.

Thus $\mathbb{T}(\mathcal{B})$ are linearly independent.

Since $\mathbb{T}$ is injective, hence all vectors in $\mathbb{T}(\mathcal{B})$ are distinct, hence

$$|\mathbb{T}(\mathcal{B})|=n.$$

Since $\mathbb{T}(\mathcal{B})$ span $\mathrm{im}\mathbb{T}$ and are linearly independent, hence they form a basis of $\mathrm{im}\mathbb{T}$.

But

$$\dim \mathbb{V}=\dim \mathbb{W}=n$$

and $\mathbb{T}(\mathcal{B})$ are a set of $n$ linearly independent vectors in $\mathbb{W}$.

Hence, $\mathbb{T}(\mathcal{B})$ form a basis of $\mathbb{W}$. Thus

$$\mathrm{im}\mathbb{T}=\mathrm{span}\mathbb{T}(\mathcal{B})=\mathbb{W}.$$

Thus $\mathbb{T}$ is surjective.

From (2) to (3)

$\mathbb{T}$ is surjective means $\mathrm{im}\mathbb{T}=\mathbb{W}$. Thus

$$\mathrm{rank}\mathbb{T}=\dim \mathbb{W}=\dim \mathbb{V}.$$

From (3) to (1)

We know that

$$\dim \mathbb{V}=\mathrm{rank}\mathbb{T}+\mathrm{nullity}\mathbb{T}.$$

But, it is given that $\mathrm{rank}\mathbb{T}=\dim \mathbb{V}$. Thus

$$\mathrm{nullity}\mathbb{T}=0.$$

Thus $\mathbb{T}$ is injective (due to Theorem 4.27).

4.3.5. Bracket Operator

Recall the definition of coordinate vector from Definition 4.19. Conversion of a given vector to its coordinate vector representation can be shown to be a linear transformation.

Definition 4.54 (Bracket operator)

Let $\mathbb{V}$ be a finite dimensional vector space over a field $\mathbb{F}$ where $\dim \mathbb{V}=n$. Let $\mathcal{B}=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ be an ordered basis in $\mathbb{V}$. We define a bracket operator from $\mathbb{V}$ to ${\mathbb{F}}^n$ as

$$\begin{array}{r}\begin{array}{rl}{\left[\right]}_{\mathcal{B}}:& \mathbb{V}\to {\mathbb{F}}^n\\ & \mathbf{x}\to [\mathbf{x}{]}_{\mathcal{B}}\\ & \triangleq \left[\begin{array}{c}{\alpha }_1\\ ⋮\\ {\alpha }_n\end{array}\right]\end{array}\end{array}$$

where

$$\mathbf{x}=\sum _{i=1}^n{\alpha }_i{\mathbf{v}}_i$$

is the unique representation of $\mathbf{x}$ in $\mathcal{B}$.

In other words, the bracket operator takes a vector $\mathbf{x}$ from a finite dimensional space $\mathbb{V}$ to its representation in ${\mathbb{F}}^n$ for a given basis $\mathcal{B}$.

We now show that the bracket operator is linear.

Theorem 4.29 (Bracket operator is linear and bijective)

Let $\mathbb{V}$ be a finite dimensional vector space over a field $\mathbb{F}$ where $\dim \mathbb{V}=n$. Let $\mathcal{B}=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ be an ordered basis in $\mathbb{V}$. The bracket operator ${\left[\right]}_{\mathcal{B}}:\mathbb{V}\to {\mathbb{F}}^n$ as defined in Definition 4.54 is a linear operator.

Moreover ${\left[\right]}_{\mathcal{B}}$ is a bijective mapping.

Proof. Let $\mathbf{x},\mathbf{y}\in \mathbb{V}$ such that

$$\mathbf{x}=\sum _{i=1}^n{\alpha }_i{\mathbf{v}}_i$$

and

$$\mathbf{y}=\sum _{i=1}^n{\beta }_i{\mathbf{v}}_i.$$

Then

$$c\mathbf{x}+\mathbf{y}=c\sum _{i=1}^n{\alpha }_i{\mathbf{v}}_i+\sum _{i=1}^n{\beta }_i{\mathbf{v}}_i=\sum _{i=1}^n(c{\alpha }_i+{\beta }_i){\mathbf{v}}_i.$$

Thus,

$$\begin{array}{r}[c\mathbf{x}+\mathbf{y}{]}_{\mathcal{B}}=\left[\begin{array}{c}c{\alpha }_1+{\beta }_1\\ ⋮\\ c{\alpha }_n+{\beta }_n\end{array}\right]=c\left[\begin{array}{c}{\alpha }_1\\ ⋮\\ {\alpha }_n\end{array}\right]+\left[\begin{array}{c}{\beta }_1\\ ⋮\\ {\beta }_n\end{array}\right]=c[\mathbf{x}{]}_{\mathcal{B}}+[\mathbf{y}{]}_{\mathcal{B}}.\end{array}$$

Thus ${\left[\right]}_{\mathcal{B}}$ is linear.

We can see that by definition ${\left[\right]}_{\mathcal{B}}$ is injective. Now since $\dim \mathbb{V}=n=\dim {\mathbb{F}}^n$ hence ${\left[\right]}_{\mathcal{B}}$ is surjective due to Theorem 4.28.

4.3.6. Matrix Representations

It is much easier to work with a matrix representation of a linear transformation. In this section we describe how matrix representations of a linear transformation are developed.

In order to develop a representation for the map $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ we first need to choose a representation for vectors in $\mathbb{V}$ and $\mathbb{W}$. This can be easily done by choosing a basis in $\mathbb{V}$ and another in $\mathbb{W}$. Once the bases are chosen, then we can represent vectors as coordinate vectors.

Definition 4.55 (Matrix representation of a linear transformation)

Let $\mathbb{V}$ and $\mathbb{W}$ be finite dimensional vector spaces with ordered bases $\mathcal{B}=\{{\mathbf{v}}_1,\dots ,{\mathbf{v}}_n\}$ and $\mathrm{\Gamma }=\{{\mathbf{w}}_1,\dots ,{\mathbf{w}}_m\}$ respectively. Let $\mathbb{T}:\mathbb{V}\to \mathbb{W}$ be a linear transformation. For each ${\mathbf{v}}_j\in \mathcal{B}$ we can find a unique representation for $\mathbb{T}({\mathbf{v}}_j)$ in $\mathrm{\Gamma }$ given by

$$\mathbb{T}({\mathbf{v}}_j)=\sum _{i=1}^m{a}_{ij}{\mathbf{w}}_i\mathrm{\forall }1\le j\le n.$$

The $m\times n$ matrix $A$ defined by $A_{ij}=a_{ij}$ is the matrix representation of $\mathbb{T}$ in the ordered bases $\mathcal{B}$ and $\mathrm{\Gamma }$, denoted as

$$A=[\mathbb{T}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}.$$

If $\mathbb{V}=\mathbb{W}$ and $\mathcal{B}=\mathrm{\Gamma }$ then we write

$$A=[\mathbb{T}{]}_{\mathcal{B}}.$$

The $j$-th column of $A$ is the representation of $\mathbb{T}(v_j)$ in $\mathrm{\Gamma }$.

In order to justify the matrix representation of $\mathbb{T}$ we need to show that application of $\mathbb{T}$ is same as multiplication by $A$. This is stated formally below.

Theorem 4.30 (Justification of matrix representation)

$$[\mathbb{T}(\mathbf{v}){]}_{\mathrm{\Gamma }}=[\mathbb{T}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}[\mathbf{v}{]}_{\mathcal{B}}\mathrm{\forall }\mathbf{v}\in \mathbb{V}.$$

Proof. Let

$$\mathbf{v}=\sum _{j=1}^n{c}_j{\mathbf{v}}_j.$$

Then

$$\begin{array}{r}[\mathbf{v}{]}_{\mathcal{B}}=\left[\begin{array}{c}{c}_1\\ ⋮\\ c_n\end{array}\right]\end{array}$$

Now

$$\begin{array}{rl}\mathbb{T}(\mathbf{v})& =\mathbb{T}\left(\sum _{j=1}^n{c}_j{\mathbf{v}}_j\right)\\ & =\sum _{j=1}^n{c}_j\mathbb{T}({\mathbf{v}}_j)\\ & =\sum _{j=1}^n{c}_j\sum _{i=1}^m{a}_{ij}{\mathbf{w}}_i\\ & =\sum _{i=1}^m\left(\sum _{j=1}^n{a}_{ij}{c}_j\right){\mathbf{w}}_i\end{array}$$

Thus

$$\begin{array}{r}[\mathbb{T}(\mathbf{v}){]}_{\mathrm{\Gamma }}=\left[\begin{array}{c}\sum _{j=1}^n{a}_{1j}{c}_j\\ ⋮\\ \sum _{j=1}^n{a}_{mj}{c}_j\end{array}\right]=A\left[\begin{array}{c}{c}_1\\ ⋮\\ c_n\end{array}\right]=[\mathbb{T}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}[\mathbf{v}{]}_{\mathcal{B}}.\end{array}$$

4.3.7. Vector Space of Linear Transformations

If we consider the set of linear transformations from $\mathbb{V}$ to $\mathbb{W}$ we can impose some structure on it and take its advantages.

First of all we will define basic operations like addition and scalar multiplication on the general set of mappings from a vector space $\mathbb{V}$ to another vector space $\mathbb{W}$.

Definition 4.56 (Addition and scalar multiplication on mappings)

Let $\mathbb{T}$ and $\mathbb{U}$ be arbitrary mappings from vector space $\mathbb{V}$ to vector space $\mathbb{W}$ over the field $\mathbb{F}$. Then addition of mappings is defined as

$$(\mathbb{T}+\mathbb{U})(\mathbf{v})=\mathbb{T}(\mathbf{v})+\mathbb{U}(\mathbf{v})\mathrm{\forall }\mathbf{v}\in \mathbb{V}.$$

Scalar multiplication on a mapping is defined as

$$(\alpha \mathbb{T})(\mathbf{v})=\alpha (\mathbb{T}(\mathbf{v}))\mathrm{\forall }\alpha \in \mathbb{F},\mathbf{v}\in \mathbb{V}.$$

With these definitions we have

$$(\alpha \mathbb{T}+\mathbb{U})(\mathbf{v})=(\alpha \mathbb{T})(\mathbf{v})+\mathbb{U}(\mathbf{v})=\alpha (\mathbb{T}(\mathbf{v}))+\mathbb{U}(\mathbf{v}).$$

We are now ready to show that with the addition and scalar multiplication as defined above, the set of linear transformations from $\mathbb{V}$ to $\mathbb{W}$ actually forms a vector space.

Theorem 4.31 (Linear transformations form a vector space)

Let $\mathbb{V}$ and $\mathbb{W}$ be vector spaces over field $\mathbb{F}$. Let $\mathbb{T}$ and $\mathbb{U}$ be some linear transformations from $\mathbb{V}$ to $\mathbb{W}$. Let addition and scalar multiplication of linear transformations be defined as in Definition 4.56. Then $\alpha \mathbb{T}+\mathbb{U}$ where $\alpha \in \mathbb{F}$ is a linear transformation.

Moreover, the set of linear transformations from $\mathbb{V}$ to $\mathbb{W}$ forms a vector space.

Proof. We first show that $\alpha \mathbb{T}+\mathbb{U}$ is linear.

Let $\mathbf{x},\mathbf{y}\in \mathbb{V}$ and $\beta \in \mathbb{F}$. Then we need to show that

$$\begin{array}{r}(\alpha \mathbb{T}+\mathbb{U})(\mathbf{x}+\mathbf{y})=(\alpha \mathbb{T}+\mathbb{U})(\mathbf{x})+(\alpha \mathbb{T}+\mathbb{U})(\mathbf{y})\\ (\alpha \mathbb{T}+\mathbb{U})(\beta \mathbf{x})=\beta ((\alpha \mathbb{T}+\mathbb{U})(\mathbf{x})).\end{array}$$

Starting with the first one:

$$\begin{array}{rl}(\alpha \mathbb{T}+\mathbb{U})(\mathbf{x}+\mathbf{y})& =(\alpha \mathbb{T})(\mathbf{x}+\mathbf{y})+\mathbb{U}(\mathbf{x}+\mathbf{y})\\ & =\alpha (\mathbb{T}(\mathbf{x}+\mathbf{y}))+\mathbb{U}(\mathbf{x})+\mathbb{U}(\mathbf{y})\\ & =\alpha \mathbb{T}(\mathbf{x})+\alpha \mathbb{T}(\mathbf{y})+\mathbb{U}(\mathbf{x})+\mathbb{U}(\mathbf{y})\\ & =(\alpha \mathbb{T})(\mathbf{x})+\mathbb{U}(\mathbf{x})+(\alpha \mathbb{T})(\mathbf{y})+\mathbb{U}(\mathbf{y})\\ & =(\alpha \mathbb{T}+\mathbb{U})(\mathbf{x})+(\alpha \mathbb{T}+\mathbb{U})(\mathbf{y}).\end{array}$$

Now the next one

$$\begin{array}{rl}(\alpha \mathbb{T}+\mathbb{U})(\beta \mathbf{x})& =(\alpha \mathbb{T})(\beta \mathbf{x})+\mathbb{U}(\beta \mathbf{x})\\ & =\alpha (\mathbb{T}(\beta \mathbf{x}))+\beta (\mathbb{U}(\mathbf{x}))\\ & =\alpha (\beta (\mathbb{T}(\mathbf{x})))+\beta (\mathbb{U}(\mathbf{x}))\\ & =\beta (\alpha (\mathbb{T}(\mathbf{x})))+\beta (\mathbb{U}(\mathbf{x}))\\ & =\beta ((\alpha \mathbb{T})(\mathbf{x})+\mathbb{U}(\mathbf{x}))\\ & =\beta ((\alpha \mathbb{T}+\mathbb{U})(\mathbf{x})).\end{array}$$

We can now easily verify that the set of linear transformations from $\mathbb{V}$ to $\mathbb{W}$ satisfies all the requirements of a vector space. Hence it is a vector space (of linear transformations from $\mathbb{V}$ to $\mathbb{W}$).

Definition 4.57 (The vector space of linear transformations)

Let $\mathbb{V}$ and $\mathbb{W}$ be vector spaces over field $\mathbb{F}$. Then the vector space of linear transformations from $\mathbb{V}$ to $\mathbb{W}$ is denoted by $\mathcal{L}(\mathbb{V},\mathbb{W})$.

When $\mathbb{V}=\mathbb{W}$ then it is simply denoted by $\mathcal{L}(\mathbb{V})$.

The addition and scalar multiplication as defined in Definition 4.56 carries forward to matrix representations of linear transformations also.

Theorem 4.32

Let $\mathbb{V}$ and $\mathbb{W}$ be finite dimensional vector spaces over field $\mathbb{F}$ with $\mathcal{B}$ and $\mathrm{\Gamma }$ being their respective bases. Let $\mathbb{T}$ and $\mathbb{U}$ be some linear transformations from $\mathbb{V}$ to $\mathbb{W}$.

Then, the following hold

1. $[\mathbb{T}+\mathbb{U}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}=[\mathbb{T}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}+[\mathbb{U}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}$.

2. $[\alpha \mathbb{T}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}=\alpha [\mathbb{T}{]}_{\mathcal{B}}^{\mathrm{\Gamma }}\mathrm{\forall }\alpha \in \mathbb{F}$.

4.3.8. Projections

Definition 4.58

A projection is a linear transformation $P$ from a vector space $\mathbb{V}$ to itself such that $P^2=P$; i.e., if $P\mathbf{v}=\mathbf{x}$, then $P\mathbf{x}=\mathbf{x}$.

Remark 4.4

Whenever $P$ is applied twice (or more) to any vector, it gives the same result as if it was applied once.

Thus, $P$ is an idempotent operator.

Example 4.14 (Projection operators)

Consider the operator $P:{\mathbb{R}}^3\to {\mathbb{R}}^3$ defined as

$$\begin{array}{r}P=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right].\end{array}$$

Then application of $P$ on any arbitrary vector is given by

$$\begin{array}{r}P\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=\left[\begin{array}{c}x\\ y\\ 0\end{array}\right]\end{array}$$

A second application doesn’t change it

$$\begin{array}{r}P\left[\begin{array}{c}x\\ y\\ 0\end{array}\right]=\left[\begin{array}{c}x\\ y\\ 0\end{array}\right]\end{array}$$

Thus $P$ is a projection operator.

Often, we can directly verify the property by computing $P^2$ as

$$\begin{array}{r}{P}^2=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right]=P.\end{array}$$