# Linear Transformations

## Contents

# 4.3. Linear Transformations#

In this section, we will be using symbols \(\VV\) and \(\WW\) to represent arbitrary vector spaces over a field \(\FF\). Unless otherwise specified, the two vector spaces won’t be related in any way. Following results can be restated for more general situations where \(\VV\) and \(\WW\) are defined over different fields, but we will assume that they are defined over the same field \(\FF\) for simplicity of discourse.

## 4.3.1. Operators#

Operators are mappings from one vector space to another space. Normally, they are total functions.

In this section, we introduce different types of operators between vector spaces. Some operators are relevant only for real vector spaces.

(Homogeneous operator)

Let \(\VV\) and \(\WW\) be vector spaces (over some field \(\FF\)).
An operator \(T : \VV \to \WW\) is called *homogeneous* if
for every \(\bx \in \VV\) and for every \(\lambda \in \FF\)

(Positively homogeneous operator)

Let \(\VV\) and \(\WW\) be *real* vector spaces (on field \(\RR\)).
An operator \(T : \VV \to \WW\) is called *positively homogeneous* if
for every \(\bx \in \VV\) and for every \(\lambda \in \RR_{++}\)

(Additive operator)

Let \(\VV\) and \(\WW\) be vector spaces.
An operator \(T : \VV \to \WW\) is called *additive* if
for every \(\bx,\by \in \VV\)

## 4.3.2. Linear Transformations#

A linear operator is additive and homogeneous.

(Linear transformation)

We call a map \(\TT : \VV \to \WW\) a *linear transformation*
from \(\VV\) to \(\WW\)
if for all \(\bx, \by \in \VV\) and \(\alpha \in \FF\), we have

\(\TT(\bx + \by) = \TT(\bx) + \TT(\by)\) and

\(\TT(\alpha \bx) = \alpha \TT(\bx)\)

A linear transformation is also known as a *linear map*
or a *linear operator*.

## 4.3.3. Properties#

(Zero maps to zero)

If \(\TT\) is linear then \(\TT(\bzero) = \bzero\).

This is straightforward since

\(\TT\) is linear \(\iff \TT(\alpha \bx + \by) = \alpha \TT(\bx) + \TT(\by) \Forall \bx, \by \in \VV, \alpha \in \FF\)

Proof. Assuming \(\TT\) to be linear we have

Now for the converse, assume

Choosing both \(\bx\) and \(\by\) to be \bzero and \(\alpha=1\) we get

Choosing \(\by=\bzero\) we get

Choosing \(\alpha = 1\) we get

Thus, \(\TT\) is a linear transformation.

If \(\TT\) is linear then \(\TT(\bx - \by) = \TT(\bx) - \TT(\by)\).

(Linear transformation preserves linear combinations)

\(\TT\) is linear \(\iff\) for \(\bx_1, \dots, \bx_n \in \VV\) and \(\alpha_1, \dots, \alpha_n \in \FF\),

We can use mathematical induction to prove this.

Some special linear transformations need mention.

(Identity transformation)

The *identity transformation* \(\mathrm{I}_{\VV} : \VV \to \VV\) is defined as

The *zero transformation* \(\ZERO : \VV \to \WW\) is defined as

Note that \(\bzero\) on the R.H.S. is the zero vector or \(\WW\).

In this definition \(0\) is taking up multiple meanings: a linear transformation from \(\VV\) to \(\WW\) which maps every vector in \(\VV\) to the \(\bzero\) vector in \(\WW\).

From the context usually it should be obvious whether we are talking about \(0 \in \FF\) or \(\bzero \in \VV\) or \(\bzero \in \WW\) or \(\ZERO\) as a linear transformation from \(\VV\) to \(\WW\).

## 4.3.4. Null Space and Range#

(Null space / Kernel)

The *null space* or *kernel* of a linear transformation \(\TT : \VV \to \WW\)
denoted by \(\NullSpace(\TT)\) or \(\Kernel(\TT)\) is defined as

The null space of a linear transformation \(\TT : \VV \to \WW\) is a subspace of \(\VV\).

Proof. Let \(\bv_1, \bv_2 \in \Kernel(\TT)\). Then

Thus \(\alpha \bv_1 + \bv_2 \in \Kernel(\TT)\). Thus \(\Kernel(\TT)\) is a subspace of \(\VV\).

The *range* or *image* of a linear transformation \(\TT : \VV \to \WW\)
denoted by \(\Range(\TT)\) or \(\Image(\TT)\) is defined as

We note that \(\Image(\TT) \subseteq \WW\).

The image of a linear transformation \(\TT : \VV \to \WW\) is a subspace of \(\WW\).

Proof. Let \(\bw_1, \bw_2 \in \Image(\TT)\). Then there exist \(\bv_1, \bv_2 \in \VV\) such that

Thus

Thus \(\alpha \bw_1 + \bw_2 \in \Image(\TT)\). Hence \(\Image(\TT)\) is a subspace of \(\WW\).

Let \(\TT : \VV \to \WW\) be a linear transformation. Assume \(\VV\) to be finite dimensional. Let \(\BBB = \{\bv_1, \bv_2, \dots, \bv_n\}\) be some basis of \(\VV\). Then

i.e., the image of a basis of \(\VV\) under a linear transformation \(\TT\) spans the range of the transformation.

Proof. Let \(\bw\) be some arbitrary vector in \(\Image(\TT)\). Then there exists \(\bv \in \VV\) such that \(\bw = \TT(\bv)\). Now

since \(\BBB\) forms a basis for \(\VV\). Thus,

This means that \(\bw \in \span \TT(\BBB)\).

(Nullity)

For vector spaces \(\VV\) and \(\WW\) and linear transformation \(\TT : \VV \to \WW\)
if \(\kernel \TT\) is finite dimensional then
*nullity* of \(\TT\) is defined as

i.e., the dimension of the null space or kernel of \(\TT\).

For vector spaces \(\VV\) and \(\WW\) and linear \(\TT : \VV \to \WW\),
if \(\range \TT\) is finite dimensional then *rank* of \(\TT\) is
defined as

i.e., the dimension of the range or image of \(\TT\).

(Dimension theorem)

For vector spaces \(\VV\) and \(\WW\) and linear \(\TT : \VV \to \WW\) if \(\VV\) is finite dimensional, then

This is known as *dimension theorem*.

For vector spaces \(\VV\) and \(\WW\) and linear \(\TT : \VV \to \WW\), \(\TT\) is injective if and only if \(\Kernel(\TT) = \{ \bzero\}\).

Proof. If \(\TT\) is injective, then

Let \(\bv \neq \bzero\). Now \(\TT(\bzero) = \bzero \implies \TT(v) \neq \bzero \) since \(\TT\) is one-one. Thus \(\kernel \TT = \{ \bzero\}\).

For the converse, let us assume that \(\kernel \TT = \{ \bzero\}\). Let \(\bv_1, \bv_2 \in \VV\) be two vectors such that they have the same image. Then,

Thus \(\TT\) is injective.

(Bijective transformation characterization)

For vector spaces \(\VV\) and \(\WW\) of equal finite dimensions and linear \(\TT : \VV \to \WW\), the following are equivalent.

\(\TT\) is injective.

\(\TT\) is surjective.

\(\rank \TT = \dim \VV\).

Proof. From (1) to (2)

Let \(\BBB = \{\bv_1, \bv_2, \dots \bv_n \}\) be some basis of \(\VV\) with \(\dim \VV = n\).

Let us assume that \(\TT(\BBB)\) are linearly dependent. Thus, there exists a linear relationship

where \(\alpha_i\) are not all 0. Now

since \(\TT\) is injective (see Theorem 4.27). This means that \(\bv_i\) are linearly dependent. This contradicts our assumption that \(\BBB\) is a basis for \(\VV\).

Thus \(\TT(\BBB)\) are linearly independent.

Since \(\TT\) is injective, hence all vectors in \(\TT(\BBB)\) are distinct, hence

Since \(\TT(\BBB)\) span \(\image \TT\) and are linearly independent, hence they form a basis of \(\image \TT\).

But

and \(\TT(\BBB)\) are a set of \(n\) linearly independent vectors in \(\WW\).

Hence, \(\TT(\BBB)\) form a basis of \(\WW\). Thus

Thus \(\TT\) is surjective.

From (2) to (3)

\(\TT\) is surjective means \(\image \TT = \WW\). Thus

From (3) to (1)

We know that

But, it is given that \(\rank \TT = \dim \VV\). Thus

Thus \(\TT\) is injective (due to Theorem 4.27).

## 4.3.5. Bracket Operator#

Recall the definition of coordinate vector from Definition 4.19. Conversion of a given vector to its coordinate vector representation can be shown to be a linear transformation.

(Bracket operator)

Let \(\VV\) be a finite dimensional vector space over a field \(\FF\) where \(\dim \VV = n\). Let \(\BBB = \{ \bv_1, \dots, \bv_n\}\) be an ordered basis in \(\VV\). We define a bracket operator from \(\VV\) to \(\FF^n\) as

where

is the unique representation of \(\bx\) in \(\BBB\).

In other words, the bracket operator takes a vector \(\bx\) from a finite dimensional space \(\VV\) to its representation in \(\FF^n\) for a given basis \(\BBB\).

We now show that the bracket operator is linear.

(Bracket operator is linear and bijective)

Let \(\VV\) be a finite dimensional vector space over a field \(\FF\) where \(\dim \VV = n\). Let \(\BBB = \{ \bv_1, \dots, \bv_n\}\) be an ordered basis in \(\VV\). The bracket operator \(\Bracket_{\BBB} : \VV \to \FF^n\) as defined in Definition 4.54 is a linear operator.

Moreover \(\Bracket_{\BBB}\) is a bijective mapping.

Proof. Let \(\bx, \by \in \VV\) such that

and

Then

Thus,

Thus \(\Bracket_{\BBB}\) is linear.

We can see that by definition \(\Bracket_{\BBB}\) is injective. Now since \(\dim \VV = n = \dim \FF^n\) hence \(\Bracket_{\BBB}\) is surjective due to Theorem 4.28.

## 4.3.6. Matrix Representations#

It is much easier to work with a matrix representation of a linear transformation. In this section we describe how matrix representations of a linear transformation are developed.

In order to develop a representation for the map \(\TT : \VV \to \WW\) we first need to choose a representation for vectors in \(\VV\) and \(\WW\). This can be easily done by choosing a basis in \(\VV\) and another in \(\WW\). Once the bases are chosen, then we can represent vectors as coordinate vectors.

(Matrix representation of a linear transformation)

Let \(\VV\) and \(\WW\) be finite dimensional vector spaces with ordered bases \(\BBB = \{\bv_1, \dots, \bv_n\}\) and \(\Gamma = \{\bw_1, \dots,\bw_m\}\) respectively. Let \(\TT : \VV \to \WW\) be a linear transformation. For each \(\bv_j \in \BBB\) we can find a unique representation for \(\TT(\bv_j)\) in \(\Gamma\) given by

The \(m\times n\) matrix \(A\) defined by \(A_{ij} = a_{ij}\) is the
*matrix representation* of \(\TT\) in the ordered bases
\(\BBB\) and \(\Gamma\), denoted as

If \(\VV = \WW\) and \(\BBB = \Gamma\) then we write

The \(j\)-th column of \(A\) is the representation of \(\TT(v_j)\) in \(\Gamma\).

In order to justify the matrix representation of \(\TT\) we need to show that application of \(\TT\) is same as multiplication by \(A\). This is stated formally below.

(Justification of matrix representation)

Proof. Let

Then

Now

Thus

## 4.3.7. Vector Space of Linear Transformations#

If we consider the set of linear transformations from \(\VV\) to \(\WW\) we can impose some structure on it and take its advantages.

First of all we will define basic operations like addition and scalar multiplication on the general set of mappings from a vector space \(\VV\) to another vector space \(\WW\).

(Addition and scalar multiplication on mappings)

Let \(\TT\) and \(\UU\) be arbitrary mappings from vector space \(\VV\)
to vector space \(\WW\) over the field \(\FF\).
Then *addition* of mappings is defined as

*Scalar multiplication* on a mapping is defined as

With these definitions we have

We are now ready to show that with the addition and scalar multiplication as defined above, the set of linear transformations from \(\VV\) to \(\WW\) actually forms a vector space.

(Linear transformations form a vector space)

Let \(\VV\) and \(\WW\) be vector spaces over field \(\FF\). Let \(\TT\) and \(\UU\) be some linear transformations from \(\VV\) to \(\WW\). Let addition and scalar multiplication of linear transformations be defined as in Definition 4.56. Then \(\alpha \TT + \UU\) where \(\alpha \in \FF\) is a linear transformation.

Moreover, the set of linear transformations from \(\VV\) to \(\WW\) forms a vector space.

Proof. We first show that \(\alpha \TT + \UU\) is linear.

Let \(\bx,\by \in \VV\) and \(\beta \in \FF\). Then we need to show that

Starting with the first one:

Now the next one

We can now easily verify that the set of linear transformations from \(\VV\) to \(\WW\) satisfies all the requirements of a vector space. Hence it is a vector space (of linear transformations from \(\VV\) to \(\WW\)).

(The vector space of linear transformations)

Let \(\VV\) and \(\WW\) be vector spaces over field \(\FF\). Then
the *vector space of linear transformations* from \(\VV\) to \(\WW\)
is denoted by \(\LinTSpace(\VV, \WW)\).

When \(\VV = \WW\) then it is simply denoted by \(\LinTSpace(\VV)\).

The addition and scalar multiplication as defined in Definition 4.56 carries forward to matrix representations of linear transformations also.

Let \(\VV\) and \(\WW\) be finite dimensional vector spaces over field \(\FF\) with \(\BBB\) and \(\Gamma\) being their respective bases. Let \(\TT\) and \(\UU\) be some linear transformations from \(\VV\) to \(\WW\).

Then, the following hold

\([\TT + \UU]_{\BBB}^{\Gamma} = [\TT]_{\BBB}^{\Gamma} + [\UU]_{\BBB}^{\Gamma}\).

\([\alpha \TT]_{\BBB}^{\Gamma} = \alpha [\TT]_{\BBB}^{\Gamma} \Forall \alpha \in \FF\).

## 4.3.8. Projections#

A *projection* is a linear transformation \(P\) from a
vector space \(\VV\) to itself such that \(P^2=P\);
i.e., if \( P \bv = \bx\), then \(P \bx = \bx\).

Whenever \(P\) is applied twice (or more) to any vector, it gives the same result as if it was applied once.

Thus, \(P\) is an *idempotent* operator.

(Projection operators)

Consider the operator \(P : \RR^3 \to \RR^3\) defined as

Then application of \(P\) on any arbitrary vector is given by

A second application doesn’t change it

Thus \(P\) is a projection operator.

Often, we can directly verify the property by computing \(P^2\) as