Normed Linear Spaces
Contents
4.4. Normed Linear Spaces#
A norm is a real number attached to every vector. Norm is a generalization of the notion of length. Adding a norm to a vector space makes it a normed linear space with rich topological properties as a norm induces a distance function (a metric) on the vector space converting it into a metric space with an algebraic structure. We can identify open sets, closed sets, bounded sets, compact sets, etc.. We can introduce the notion of continuity on the functions defined from one normed linear space to another metric space (which could be a normed linear space or just the real line).
The algebraic structure allows us to consider operations like translations, scaling, rotation, and general affine transformations on sets of vectors which can be thought of as geometrical objects.
There are many ways to define norms on a vector space. A question arises as to which norms are equivalent to each other in the sense that they determine the same topology on the underlying vector space. It turns out that in finite dimensional vector spaces, all norms are equivalent.
Linear transformations play a central role in linear algebra. Linear transformations are generally unbounded by the usual notion of boundedness of a function (where we say that the range of the function is bounded). However, there is another way to examine the boundedness of linear transformations by examine the ratio of norms of the input to the transformation and the output generated by it. For bounded linear transformations, the ratio is bounded. For matrices, this is the largest singular value. Not all linear transformations are continuous, but linear transformations on finite dimensional spaces are continuous. In fact, for linear transformations, boundedness, continuity, uniform continuity and Lipschitz continuity turn out to be the same thing.
A normed linear space that is complete is called a Banach space.
We restrict our attention to real vector spaces and complex vector spaces. Thus, the field \(\FF\) can be either \(\RR\) or \(\CC\).
4.4.1. Norm#
Definition 4.59 (Norm)
A norm over a \(\FF\)-vector space \(\VV\) is any real valued function \(\| \cdot \| : \VV \to \RR\) mapping \( \bv \mapsto \| \bv\|\) satisfying following properties:
[Positive definiteness]
\[ \| \bv\| \geq 0 \quad \forall \bv \in \VV \text{ and } \| \bv\| = 0 \iff \bv = \bzero. \][Positive homogeneity]
\[ \| \alpha \bv \| = | \alpha | \| \bv \| \quad \forall \alpha \in \FF; \forall \bv \in \VV. \][Triangle inequality]
\[ \| \bv_1 + \bv_2 \| \leq \| \bv_1 \| + \| \bv_2 \| \quad \forall \bv_1, \bv_2 \in \VV. \]
The norm doesn’t change on negation.
Remark 4.5 (Negation and norm)
There is a different form of triangle inequality which is quite useful in the following analysis. It can be derived from the properties of a norm.
Theorem 4.33 (Triangle inequality II)
Proof. Putting \(\bv_1 = \bx\) and \(\bv_2 = \by - \bx\) in the triangle inequality, we get:
Interchanging \(\bx\) and \(\by\) in previous inequality, we get:
Combining the two inequalities, we get:
Theorem 4.34 (Triangle inequality for vector differences)
The triangle inequality is equivalent to the following property:
Proof. Start with
Put \(\bv_1 = \bx - \bz \) and \(\bv_2 = \bz - \by\). We get:
For the converse, start with:
Put \(\bv_1 = \bx - \bz\) and \(\bv_2 = \bz - \by\). Then, \(\bv_1 + \bv_2 = \bx - \by\). We get:
4.4.2. Normed Linear Space#
Definition 4.60 (Normed linear space)
An \(\FF\)-vector space \(\VV\) equipped with a norm \(\| \| : \VV \to \RR\) is known as a normed linear space. Other common terms are normed vector space or simply normed space.
Example 4.15 (Norm for the trivial vector space)
The trivial vector space contains a single vector which is the zero vector; i.e., \(\VV = \{ \bzero \}\).
A function \(f : \VV \to \RR\) given by \(f(\bzero) = 0\) satisfies all the properties of a norm. It is easy to see that this is the only possible norm for the trivial vector space.
Remark 4.6
We will assume that the vector space is non-trivial; i.e., different from \(\{ \bzero \}\) in this section. Wherever necessary, we will provide details about how a particular result is valid for the trivial vector space also.
Example 4.16 (Real line \(\RR\))
Recall from Example 4.3 that \(\RR\) is a vector space by itself.
The function \( f : \RR \to \RR\) given by
satisfies all the requirements of a norm. Thus, \((\RR, | \cdot |)\) is a normed linear space.
Example 4.17 (Euclidean space \(\RR^n\))
The Euclidean space \(\RR^n\) can be equipped with a variety of norms. They are discussed in detail in The Euclidean Space.
\(\ell_p\) norms
When we propose a norm for a vector space, we justify it by showing that it satisfies all the properties of a norm.
Theorem 4.111 justifies that all \(\ell_p\) norms on \(\RR^n\) are indeed norms.
4.4.3. Normed Space as a Metric Space#
Definition 4.61 (Metric induced by a norm)
Every norm \(\| \cdot \| : \VV \to \RR\) induces a metric defined as:
Theorem 4.35 (Norm induced metric justification)
The metric \(d : \VV \times \VV \to \RR\) induced by a norm \(\| \cdot \| : \VV \to \RR\) is indeed a metric satisfying all the properties of a metric( distance function).
Proof. We proceed as follows:
Non-negativity: \(d(\bx, \by) \geq 0\) since \(\| \cdot \|\) is positive definite.
Identity of indiscernibles.
Assume \(d(\bx, \by) = 0\).
Then, \(\| \bx - \by \| = 0\).
Thus, \(\bx = \by\) since \(\| \cdot \|\) is positive definite.
Now, assume \(\bx = \by\).
Then, \(d(\bx, \by) = \| \bx - \by \| = \| \bzero \| = 0\) since \(\| \cdot \|\) is positive definite.
Symmetry: \(d(\bx, \by) = \| \bx - \by \| = \| (-1)(\by - \bx) \| = |-1| \| \by - \bx \| = d(\by, \bx)\) using the positive homogeneity property of \(\| \cdot \|\).
Triangle inequality: See Theorem 4.34 above.
Remark 4.7
We will use the notation \(\| \cdot \|\) to denote both the norm and the metric induced by the norm.
Definition 4.62 (Metric space from a norm)
The normed space \(\VV\) equipped with the metric induced by the norm as defined in Definition 4.61 becomes a metric space \((\VV, \| \cdot \|)\).
If the norm and the induced metric are clear from the context, then we shall simply write it as \(\VV\).
Theorem 4.36 (Translation invariance)
The metric induced by a norm is translation invariant.
For any \(\bu, \bv, \bw \in \VV\):
Proof. Expanding from definition:
The topology of a general metric space is discussed in detail in Metric Topology and sections thereafter. In this section, we discuss results which are specific to normed linear spaces as they take advantage of the additional structure provided by the vector space.
There is a special zero vector \(\bzero \in \VV\). It provides a reference point to define unit balls.
Vectors in \(\VV\) can be added, subtracted and scaled. Thus, general balls can be described in terms of unit balls.
It is possible to introduce the notion of translation of sets. Recall that the metric induced by the norm is translation invariant.
Sets of vectors in a vector space can be added/subtracted/scaled since the underlying vectors can be. This produces a number of interesting phenomena.
4.4.4. Balls#
An open ball in a normed space is defined analogously as:
A closed ball in a normed space is defined analogously as:
We sometimes use the notation \(B_{\| \cdot \|}(\ba,r)\) and \(B_{\| \cdot \|}[\ba,r]\) to identify the specific norm being used to describe the open and closed balls.
Definition 4.63 (Unit ball)
A ball centered at origin \(\bzero \in \VV\) is called a unit ball if its radius is \(1\). \(B[\bzero, 1]\) denotes a closed unit ball and \(B(\bzero, 1)\) denotes an open unit ball. The open unit ball is often written simply as \(B\).
Observation 4.1 (Ball arithmetic)
Following the notation in Definition 4.25, a closed ball can be expressed in terms of closed unit ball as:
Similarly, any open ball can be expressed in terms of the open unit ball as:
Theorem 4.37 (Ball addition and subtraction)
Let \(\VV\) be a normed linear space and \(B\) be the unit open ball. Then,
Proof. Due to Theorem 4.16, \(2 B \subseteq B + B\).
We now show that \(B + B \subseteq 2 B\).
Let \(\ba \in B + B\).
Then, \(\ba = \bb + \bc\) such that \(\bb \in B\) and \(\bc \in B\).
Now,
\( \| \ba \| = \| \bb + \bc \| \leq \| \bb \| + \| \bc \| < 1 + 1 = 2. \)
Thus, \(\ba \in 2 B\).
Thus, \(B + B = 2B\).
Next, note that \(B\) is symmetric; i.e. \(B = -B\).
Thus,
Theorem 4.38 (Moving a point into a ball)
Let \(\VV\) be a normed linear space. Let \(B(\bx, r)\) be an open ball in \(\VV\).
Then, every point in \(\VV\) can be translated and scaled into a point which is inside the ball \(B(\bx, r)\).
Proof. Let \(\bv \in \VV\).
If \(\bv = \bzero\), then we can simply translate it by \(\bx\) to put it inside \(B(\bx, r)\).
Now, assume that \(\bv \neq \bzero\). Then, \(\| \bv \| \neq 0\).
Let \(\by = \bx + \frac{r}{2 \| \bv \|} \bv\).
Then,
\[ \| \by - \bx \| = \left \| \frac{r}{2 \| \bv \|} \bv \right \| = \frac{r}{2} < r. \]Thus, \(\by \in B(\bx, r)\).
In the following, \(B\) means the open unit ball \(B(\bzero, 1)\).
4.4.5. Interior#
Theorem 4.39 (Interior points in a normed space)
Let \(A\) be a subset of a normed linear space \(\VV\). Then, \(\bx \in \interior A\) if and only if there exists \(r > 0\) such that
Proof. From Definition 3.12, \(\bx\) is an interior point of \(A\) if there exists an \(r > 0\) such that the open ball \(B(\bx, r) \subseteq A\). But, as per algebraic notation \(B(\bx, r) = \bx + r B\).
Theorem 4.40 (Interior in a normed space)
Let \(A\) be a subset of a normed linear space \(\VV\). The interior of \(A\) is given by
Proof. This follows from the fact that the interior is a collection of all the interior points of \(A\).
Theorem 4.41 (Subspaces and interior)
Let \(\VV\) be a normed vector space. The only subspace of \(\VV\) that has a nonempty interior is \(\VV\) itself.
Proof. Let \(S\) be a subspace of \(\VV\).
Assume that \(S\) has a nonempty interior.
Then, there is an interior point \(\bx \in S\).
Thus, there is an open ball \(B(\bx, r) \subseteq S\).
By Theorem 4.38, any nonzero point \(\bv \in \VV\) can be moved into the ball \(B(\bx, r)\) as
\[ \by = \bx + \frac{r}{2 \| \bv \|} \bv. \]We have \(\by \in S\).
But a subspace is closed under vector addition and scalar multiplication.
Thus,
\[ \bv = \frac{2 \| \bv \|}{r} (\by - \bx) \in S. \]Thus, \(\bv \in \VV\) implies \(\bv \in S\).
Thus, \(\VV = S\).
Thus, the only subspace which has a nonempty interior is \(\VV\) itself.
Corollary 4.10 (Interior of proper subspaces)
Every proper subspace of a normed linear space \(\VV\) has an empty interior.
4.4.6. Closure#
Theorem 4.42 (Closure points in a normed space)
Let \(A\) be a subset of a normed linear space \(\VV\). Then, \(\bx \in \closure A\) if and only if
Proof. Recall that
Thus, \(\bx \in A + r B\) if and only if there exists \(\ba \in A\) such that \(\bx \in \ba + r B\).
Assume that \(\bx \in A + r B \Forall r > 0\).
Thus, for every \(r > 0\), there exists \(\ba \in A\) (depending on \(r\)) such that \(\bx \in \ba + r B = B(\ba, r)\).
Thus, for every \(r > 0\), there exists \(\ba \in A\) (depending on \(r\)) such that \(d(\bx, \ba) < r\).
Thus, for every \(r > 0\), there exists \(\ba \in A\) (depending on \(r\)) such that \(\ba \in B(\bx, r)\).
Thus, \(A \cap B(\bx, r) \neq \EmptySet\) for every \(r > 0\).
Thus, \(\bx\) is a closure point of \(A\).
For the converse, assume that \(\bx\) is a closure point of \(A\).
Then, \(A \cap B(\bx, r) \neq \EmptySet\) for every \(r > 0\).
Thus, for every \(r > 0\), there exists \(\ba \in A\) (depending on \(r\)) such that \(\ba \in B(\bx, r)\).
Thus, for every \(r > 0\), there exists \(\ba \in A\) (depending on \(r\)) such that \(d(\bx, \ba) < r\).
Thus, for every \(r > 0\), there exists \(\ba \in A\) (depending on \(r\)) such that \(\bx \in B(\ba, r) = \ba + r B\).
Thus, for every \(r > 0\), \(\bx \in A + r B\).
Theorem 4.43 (Closure in a normed space)
Let \(A\) be a subset of a normed linear space \(\VV\). Then, the closure of \(A\) is given by
Proof. From previous result, a point \(\bx\) is a closure point of \(A\) if and only if
The result follows from the fact that the closure of \(A\) is the collection of all its closure points.
4.4.7. Continuity#
Definition 4.64 (Continuity in normed linear spaces)
Let \((\VV, \| \cdot \|_v)\) and \((\WW, \| \cdot \|_w)\) be normed linear spaces.
A function \(f: \VV \to \WW\) between the two normed spaces is said to be continuous at a point \(\ba \in \dom f\) if for every \(\epsilon > 0\), there exists \(\delta > 0\) (depending on \(\epsilon\) and \(\ba\)) such that for all \(\bx \in \dom f\)
holds true.
\(f\) is said to be continuous on \(A \subseteq \dom f\) if \(f\) is continuous at every point of \(A\).
The function \(f\) is said to be uniformly continuous on \(A \subseteq \dom f\) if for every \(\epsilon > 0\), there exists some \(\delta > 0\) (depending on \(\epsilon\)) such that
These definitions are just adapted from the corresponding definitions for metric spaces for convenience.
4.4.7.1. Norms#
Theorem 4.44 (Norms are uniformly continuous )
A function \(\| \cdot \| : \VV \to \RR\) satisfying all the properties of a norm is uniformly continuous in the metric space induced by the norm.
Proof. Let \(\bx, \by \in \VV\). Assume \(d(\bx, \by) < \delta\). Choose \(\epsilon = \delta\). Now, due to Theorem 4.33:
Thus,
Thus, \(\| \cdot \|\) is uniformly continuous.
4.4.7.2. Translations#
Theorem 4.45 (Translations are uniformly continuous )
Let \((\VV, \| \cdot \|)\) be a normed linear space. Let \(\ba \in \VV\) be some fixed vector. Let a translation map \(g_a : \VV \to \VV\) be defined as
Then, \(g_a\) is uniformly continuous.
Proof. Let \(\epsilon > 0\) and \(\delta = \epsilon\). Let \(\bx, \by \in \VV\). Note that
Thus, \(\| \bx - \by \| < \delta\) implies \(\| g_a (\bx) - g_a(\by) \| < \delta = \epsilon\).
Thus, \(g_a\) is uniformly continuous.
Theorem 4.46 (Translations preserve topology)
Translations preserve the topology.
Let \((\VV, \| \cdot \|)\) be a normed linear space. Let \(\ba \in \VV\) be some fixed vector. Let a translation map \(g_a : \VV \to \VV\) be defined as
Then,
\(A\) is open if and only if \(g_a(A)\) is open.
\(A\) is closed if and only if \(g_a(A)\) is closed.
\(A\) is compact if and only if \(g_a(A)\) is compact.
\(A\) has an empty interior if and only if \(g_a(A)\) has an empty interior.
Proof. We note that the inverse of \(g_a\) is given by \(g_{-a}\) which is also a translation. Consequently, both \(g_a\) and \(g_{-a}\) are uniformly continuous.
Preservation of open and closed sets are simple applications of Theorem 3.42 (characterization of continuous functions).
Preservation of compactness is an application of Theorem 3.78 (continuous images of compact sets are compact).
Assume that \(A\) has an interior point \(\bx \in A\).
Then, there exists a ball \(B(\bx, r) \subseteq A\).
Then, the ball \(B(\bx, r) + \ba = B(\bx + \ba, r) \subseteq A + \ba\).
Thus, \(\bx + ba\) is an interior point of \(A + \ba = g_a(A)\).
Thus, if \(g_a(A)\) has an empty interior then \(A\) must have an empty interior. The converse can be proved by using the inverse translation \(g_{-a}\).
4.4.7.3. Scalar Multiplication#
Theorem 4.47 (Scalar multiplication is uniformly continuous )
Let \((\VV, \| \cdot \|)\) be a normed linear space. Let \(t \in \FF\). Let a scalar multiplication map \(g_t : \VV \to \VV\) be defined as
Then, \(g_t\) is uniformly continuous.
Proof. If \(t=0\), then \(g_t(\bx) = \bzero\) for all \(\bx \in \VV\) so it is trivially uniformly continuous.
Assume \(t \neq 0\).
Let \(\epsilon > 0\).
Choose \( \delta = \frac{\epsilon}{|t|}\).
Now, for any \(\bx, \by \in \VV\)
\[ \| g_t(\bx) - g_t(\by)\| = \| t \bx - t \by \| = \| t (\bx - \by)\| = |t | \| \bx - \by \|. \]Thus, \(\| \bx - \by \| < \delta\) implies
\[ \| g_t(\bx) - g_t(\by)\| < | t | \delta = \epsilon. \]Thus, \(g_t\) is uniformly continuous.
Theorem 4.48 (Scalar multiplication preserves topology )
Let \((\VV, \| \cdot \|)\) be a normed linear space. Let \(t \in \FF\) be nonzero (i.e., \(t \neq 0\)). Let a scalar multiplication map \(g_t : \VV \to \VV\) be defined as
Then, \(g_t\) is a homeomorphism. Consequently,
\(A\) is open if and only if \(g_t(A)\) is open.
\(A\) is closed if and only if \(g_t(A)\) is closed.
\(A\) is compact if and only if \(g_t(A)\) is compact.
\(A\) has an empty interior if and only if \(g_t(A)\) has an empty interior.
Proof. We first show that \(g_t\) is a homeomorphism. Since \(t \neq 0\), let \(r = \frac{1}{t}\).
\(g_t\) is uniformly continuous by Theorem 4.47.
Then, \(g_r\) is also uniformly continuous by Theorem 4.47.
Note that
\[ g_r(g_t)(\bx) = g_t(g_r(\bx)) = \bx \Forall \bx \in \VV. \]Thus, \(g_t\) is bijective and \(g_r\) is its inverse.
Since \(g_t\) is bijective, \(g_t\) is continuous, and its inverse \(g_r\) is continuous, hence \(g_t\) is a homeomorphism.
Morever, \(g_r\) is also a homeomorphism.
By Theorem 3.49, \(g_t\) and \(g_r\) are both a closed mapping (mapping closed sets to closed sets) and an open mapping (mapping open sets to open sets). Consequently,
\(A\) is open if and only if \(g_t(A)\) is open.
\(A\) is closed if and only if \(g_t(A)\) is closed.
Due to Theorem 3.80 (homeomorphisms preserve compact sets) \(A\) is compact if and only if \(g_t(A)\) is compact.
Due to Theorem 3.52, homeomorphisms preserve interiors. Hence, \(A\) has an empty interior if and only if \(g_t(A)\) has an empty interior.
4.4.8. Open Sets#
Theorem 4.49 (Sum of an open set with any set is open)
Let \(\VV\) be a normed linear space. Let \(A, B \subseteq \VV\). If \(A\) is open, then the sum \(A+B\) is open.
Proof. We proceed as follows:
Let \(x \in B\).
Then the set \(x + A\) is open since \(A\) is open and translations preserve open sets (Theorem 4.46).
Then,
\[ A + B = \bigcup_{x \in B} x + A \]is a union of open sets. Hence, it is open.
4.4.9. Closed Sets#
Theorem 4.50
If \(A\) is closed and \(B\) is compact, then their sum \(A+B\) is closed.
Proof. Let \(\{z_n\}\) with \(z_n = a_n + b_n \in A + B\) be a convergent sequence of \(A+B\) where \(a_n \in A\) and \(b_n \in B\).
\(\{a_n \}\) is a sequence of \(A\).
\(\{b_n \}\) is a sequence of \(B\).
Let \(\lim z_n = z\).
Since \(B\) is compact, \(\{b_n\}\) has a convergent subsequence, say \(\{b_{k_n}\}\) (Theorem 3.75).
Let \(\lim b_{k_n} = l \in B\).
Now consider the sequence \(\{a_{k_n}\}\) given by \(a_{k_n} = z_{k_n} - b_{k_n}\).
\(\{z_{k_n}\}\) is convergent since it is a subsequence of a convergent sequence (Theorem 3.35).
Since \(\{z_{k_n}\}\) and \(\{b_{k_n}\}\) are both convergent, hence \(\{a_{k_n}\}\) is also convergent.
Let \(\lim a_{k_n} = m\).
Since \(A\) is closed. Hence \(m \in A\) (Theorem 3.33).
Now, \(\lim a_{k_n} = \lim z_{k_n} - \lim b_{k_n}\) gives us \(m = z - l\).
Thus, \(z = m + l\).
But \(m \in A\) and \(l \in B\).
Hence, \(z \in A + B\).
Thus, every convergent sequence of \(A + B\) converges in \(A+B\).
Thus, \(A+B\) is closed. (Theorem 3.33).
4.4.10. Boundedness#
By definition \(\| \bx \| \geq 0\). Thus, for any subset \(A \subseteq \VV\), \(0\) is a lower bound for the set \(\{ \| \bx \| \ST \bx \in A \}\). If there is an upper bound also for the set of norms, then the set \(A\) is called bounded.
Definition 4.65 (Bounded set)
Let \(\VV\) be a normed linear space. A subset \(A\) of \(\VV\) is called norm bounded or simply bounded if there exist \(M > 0\) such that:
Compare the definition with the definition of bounded sets in metric spaces.
We can characterize the case where \(0\) is indeed the infimum or greatest lower bound for the set \(\{ \| \bx \| \ST \bx \in A \}\).
Theorem 4.51 (Zero as the greatest lower bound)
Let \(\VV\) be a normed linear space. Let \(A \subseteq \VV\). Then, \(\inf \{ \| \bx \| \ST \bx \in A \} = 0\) if and only if \(\bzero \in \closure A\).
Proof. Note that \(B(\bzero, r)\) is an open ball of radius \(r\) around \(\bzero\) given by:
Suppose \(\inf \{ \| \bx \| \ST \bx \in A \} = 0\).
Then, for every \(r > 0\), there exists \(\bx \in A\) such that \(\| \bx \| < r\).
Thus, for every \(r > 0\), \(\bx \in B(\bzero, r) \cap A\).
Thus, for every \(r > 0\), \(B(\bzero, r) \cap A \neq \EmptySet\).
Thus, \(\bzero \in \closure A\).
Now suppose that \(\bzero \in \closure A\).
Then, for every \(r> 0\), \(B(\bzero, r) \cap A \neq \EmptySet\).
Thus, for every \(r > 0\), there exists \(\bx \in A\) such that \(\| \bx \| < r\).
Thus, \(\inf \{ \| \bx \| \ST \bx \in A \} = 0\).
4.4.11. Sequences#
Definition 4.66 (Convergence in norm)
A sequence \(\{ \bx_n \}\) of a normed space \(\VV\) is said to converge in norm to \(\bx \in \VV\) if
i.e., if \(\{\bx_n \}\) converges to \(\bx\) with respect to the metric induced by the norm. We write this as:
Following is an alternative proof for Theorem 4.51 in terms of convergent sequences.
Proof. Assume that \(\bzero \in \closure A\).
Then, there exists a sequence \(\{ \bx_n \}\) such that \(\lim \| \bx_n - \bzero \| = 0\).
Thus, \(\lim \| \bx_n \| = 0\).
Thus, for every \(r > 0\), there exists \(n_0 \in \Nat\) such that \(\| \bx_n \| < r\) for all \(n > n_0\).
Thus, for every \(r > 0\), there exists \(\bx \in A\) such that \(\| \bx \| < r\).
Thus, \(\inf \{ \| \bx \| \ST \bx \in A \} = 0\).
Assume that \(\inf \{ \| \bx \| \ST \bx \in A \} = 0\).
Thus, for every \(r > 0\), there exists \(\bx \in A\) such that \(\| \bx \| < r\).
For every \(n \in \Nat\), pick a \(\bx_n \in A\) such that \(\| \bx_n \| < \frac{1}{n}\).
Form the sequence \(\{ \bx_n \}\).
Then, for every \(r > 0\), there exists \(n_0 \in \Nat\) such that for all \(n > n_0\), \(\| \bx_n \| = \| \bx_n - \bzero \| < r\).
Thus, \(\lim \bx_n = \bzero\).
Thus, \(\bzero \in \closure A\).
4.4.12. The Calculus of Limits#
Let \(\{ \bx_n \}\) and \(\{ \by_n \}\) be convergent sequences of \(\VV\). Our concern here is to understand what happens to the limits if the sequences are combined.
Our presentation here is similar to the presentation for sequences of real numbers in The Calculus of Limits.
Let \(\lim \{\bx_n\} = \bx\) and \(\lim \{\by_n\} = \by\). Then:
Theorem 4.52 (Scaling a sequence)
Proof. If \(\alpha = 0\), then we have a constant sequence and the result is trivial. So assume that \(\alpha \neq 0\). Then:
Let \(\epsilon > 0\) and choose \(n_0 \in \Nat\) such that \(\| \bx - \bx_n \| < \frac{\epsilon}{ | \alpha | }\) for all \(n > n_0\). Then
Corollary 4.11 (Negating a sequence)
We get this result by choosing \(\alpha = -1\).
Theorem 4.53 (Addition of sequences)
Proof. From triangle inequality we get:
For any \(\epsilon > 0\), choose \(n_1\) such that \(\| \bx_n - \bx \| < \frac{\epsilon}{2} \Forall n > n_1\).
Similarly, choose \(n_2\) such that \(\| \by_n - \by \| < \frac{\epsilon}{2} \Forall n > n_2\).
Now choose \(n_0 = \max (n_1, n_2)\). Then:
Corollary 4.12 (Subtraction of sequences)
Negate \(\{ \by_n \}\) and add to \(\{\bx_n \}\).
4.4.13. Cauchy Sequences#
Theorem 4.54 (Cauchy sequence is bounded)
Every Cauchy sequence in a normed space is bounded.
The proof is very similar to the proof for the boundedness of Cauchy sequences in real line (Theorem 2.18).
Proof. Let \(\{ \bx_n \}\) be a sequence of a normed space \(\VV\).
Choose \(\epsilon = 1\).
Then there exists \(n_0 \in \Nat\) such that \(\| \bx_n - \bx_m \| < 1\) whenever \(m, n \geq n_0\).
In particular, the statement is valid when \(m = n_0\). i.e. \(\| \bx_n - \bx_{n_0} \| < 1\).
But,
\[\begin{split} \begin{aligned} & \| \bx_n - \bx_{n_0} \| < 1\\ & \implies | \| \bx_n \| - \| \bx_{n_0 } \| | < 1\\ & \implies \|\bx_n \| < 1 + \| \bx_{n_0 } \| \Forall n \geq n_0. \end{aligned} \end{split}\]Choosing \(M = \max(\|x_1\|, \dots, \|\bx_{n_0-1}\|, \|\bx_{n_0}\| + 1)\), it is clear that \(\| x_n \| \leq M\)
Hence \(\{ \bx_n \}\) is bounded.
4.4.14. Linear Transformations#
4.4.14.1. Boundedness#
Definition 4.67 (Bounded linear transformation)
Let \((\VV, \| \cdot \|_v)\) and \((\WW, \| \cdot \|_w)\) be normed linear spaces. Let \(T : \VV \to \WW\) be a linear transformation. We say that \(T\) is bounded (in the sense of linear transformations) if the set
is bounded.
Note that the set \(S\) in (4.1) is trivially bounded from below by \(0\). Thus, by bounded we mean, bounded from above.
The notion of boundedness for linear transformations is different from the notion of boundedness for bounded functions in general metric spaces as defined in Definition 3.37. There, we posit that the range of the function is bounded. For bounded linear transformations, the range (in norm) may be unbounded, yet the ratio of the norms of output and input is bounded.
Since linear transformations commute with scalar multiplication as in \(T(t \bx) = t T(\bx)\), hence the norm of the output is unbounded unless \(T\) is an identically \(\bzero\) linear transformation.
If there exists \(\bv \in \VV\) such that \(T(\bv) \neq \bzero\), then
Thus, the idea of a bounded range is not very useful for linear transformations.
Remark 4.8 (Bounded linear transformation in terms of unit norm vectors)
For \(T, \bx\) as in (4.1), and any nonzero \(t \in \FF\), we have
In particular, if we choose \(t = \frac{1}{\|\bx \|_v}\), then \(\| t \bx \|_v = 1\).
Hence, every element in set \(S\) in (4.1) equals \(\| T (\bx) \|_w\) for some unit vector \(\bx\). It follows that
Some authors prefer this description of \(S\) as the definition of bounded linear transformations.
Example 4.18 (Unbounded linear transformation)
It is possible to construct unbounded linear transformations in infinite dimensional normed linear spaces.
We shall consider a sequence space as described below.
Let \(V\) be the set of all real sequences with finitely many nonzero terms.
It is easy to verify that \(V\) is a linear subspace of the space of all sequences \(\RR^{\infty}\).
For any \(\bx \in V\) given by \(\bx = \{ x_n \}\), define the supremum norm as:
It is easy to verify that it is indeed a norm.
Define a transformation \(T : V \to V\) as
It is easy to verify that \(T\) is linear.
For any sequence \(\be_n = (0, 0, \dots, 1, 0, 0, \dots)\) with \(1\) in n-th position and all other entries 0, \(\| \be_n \| = 1\) and \( \| T (\be_n) \| = n\). Since we can construct \(\be_n\) for every \(n \in \Nat\), hence the set \(S\) as defined in (4.2) is unbounded.
Consequently, \(T\) is an unbounded linear transformation.
4.4.14.2. Continuity#
Theorem 4.55 (Characterization of continuity for linear transformations)
Let \((\VV, \| \cdot \|_v)\) and \((\WW, \| \cdot \|_w)\) be normed linear spaces. Let \(T : \VV \to \WW\) be a linear transformation. Then, the following statements are equivalent.
\(T\) is continuous at one point in \(\VV\).
\(T\) is continuous at \(\bzero \in \VV\).
\(T\) is bounded.
\(T\) is Lipschitz continuous at \(\bzero \in \VV\).
\(T\) is Lipschitz continuous.
\(T\) is uniformly continuous.
\(T\) is continuous.
Proof. (1) \(\implies\) (2)
Assume \(T\) is continuous at \(\bx_0 \in \VV\).
Let \(\epsilon > 0\) and let \(\delta > 0\) such that if \(\| \bx - \bx_0 \|_v < \delta\) then \( \| T(\bx) - T(\bx_0) \|_w < \epsilon\).
Then, for all \(\by \in B_v(\bzero, \delta)\), we have
\[ \| (\by + \bx_0) - \bx_0 \|_v = \| \by \|_v < \delta. \]Hence, \(\| T(\by + \bx_0) - T(\bx_0) \|_w < \epsilon\).
But \(T\) is linear. Hence
\[ T(\by + \bx_0) - T(\bx_0) = T(\by) = T(\by) - \bzero = T(\by) - T(\bzero). \]Thus, \(\| T(\by) - T(\bzero) \|_w < \epsilon\).
Thus, for any \(\epsilon > 0\), there exists \(\delta > 0\) such that \(\| \by - \bzero \|_v < \delta\) implies \(\| T (\by) - T(\bzero) \|_w < \epsilon\).
Thus, \(T\) is continuous at \(\bzero \in \VV\).
(2) \(\implies\) (3)
Assume that \(T\) is continuous at \(\bzero \in \VV\).
Then, for any \(\by \in \VV\),
\[ \| T(\by) - T(\bzero) \|_w = \| T (\by) - \bzero \|_w = \| T (\by) \|_w. \]Let \(\epsilon = 1\) and choose \(\delta > 0\) such that if \(\| \by \|_v < \delta\) then \(\| T (\by) \|_w < 1\). We can choose such \(\delta > 0\) since \(T\) is continuous at \(\bzero\).
Let \(\bx \in \VV\) be any nonzero vector.
Let \(\by = \frac{\delta}{2 \| \bx \|_v } \bx\).
Then, \(\| \by \|_v = \frac{\delta}{2} < \delta\).
And \(\bx = \frac{2 \| \bx \|_v}{\delta} \by\).
Then,
\[\begin{split} \| T(\bx)\|_w &= \left \| T \left ( \frac{2 \| \bx \|_v}{\delta} \by \right ) \right \|_w\\ &= \left \| \frac{2 \| \bx \|_v}{\delta} T ( \by ) \right \|_w\\ &= \frac{2 \| \bx \|_v}{\delta} \| T (\by) \|_w \\ &< \frac{2 \| \bx \|_v}{\delta} \times 1 \\ &= \frac{2 }{\delta} \| \bx \|_v. \end{split}\]Thus,
\[ \frac{\| T(\bx)\|_w }{\| \bx \|_v} < \frac{2 }{\delta} \]holds true for every nonzero \(\bx \in \VV\).
Thus, the set in (4.1) is bounded from above by \(\frac{2 }{\delta}\).
Thus, \(T\) is bounded.
(3) \(\implies\) (4)
Assume that \(T\) is bounded. Let the set \(S\) in (4.1) have an upper bound \(M > 0\). Let \(\bx \in \VV\) be a nonzero vector. Then,
Thus, there exists \(M > 0\) such that for every nonzero \(\bv \in \VV\),
Thus, \(T\) is Lipschitz continuous at \(\bzero \in \VV\).
(4) \(\implies\) (5)
We assume that \(T\) is Lipschitz continuous at \(\bzero \in \VV\).
Then, let \(K, \delta > 0\) such that for every \(\by \in \VV\) with \(\| \by \|_v < \delta\), we have
Let \(\bx \in \VV\) be a nonzero vector.
Let \(\by = \frac{\delta}{2} \frac{\bx}{\| \bx \|_v}\).
Then, \(\| \by \|_v = \frac{\delta}{2} < \delta\).
Hence, \(\| T (\by) \|_w \leq K \| \by \|_v\).
Now, using linearity of \(T\):
\[\begin{split} \| T(\bx)\|_w &= \left \| T \left ( \frac{2 \| \bx \|_v}{\delta} \by \right ) \right \|_w\\ &= \left \| \frac{2 \| \bx \|_v}{\delta} T ( \by ) \right \|_w\\ &= \frac{2 \| \bx \|_v}{\delta} \| T (\by) \|_w \\ &\leq \frac{2 \| \bx \|_v}{\delta} K \| \by \|_v \\ &= K \| \bx \|_v. \end{split}\]Thus, we have \(\| T(\bx)\|_w \leq K \| \bx \|_v\) for every nonzero \(\bx \in \VV\).
Now, let \(\bx, \bz \in \VV\) be distinct.
Using linearity of \(T\)
\[\begin{split} \| T (\bx) - T(\bz) \|_w &= \| T (\bx - \bz) \|_w\\ \leq K \| \bx - \bz \|_v. \end{split}\]Hence, \(T\) is Lipschitz continuous.
(5) \(\implies\) (6)
Assume that \(T\) is Lipschitz continuous.
Let \(K >0\) such that for every \(\bx, \by \in \VV\),
Let \(\epsilon > 0\).
Choose \(\delta = \frac{\epsilon}{K}\).
Then, for every \(\bx, \by \in \VV\) with \(\| \bx - \by \|_v < \delta\), we have
\[ \| T(\bx) - T(\by)\|_w \leq K \| \bx - \by \|_v < K \delta = K \frac{\epsilon}{K} = \epsilon. \]Hence, \(T\) is uniformly continuous.
(6) \(\implies\) (7)
A uniformly continuous function is trivially continuous.
(7) \(\implies\) (1)
A continuous function is trivially continuous at a point.
4.4.15. Equivalent Norms#
Definition 4.68 (Equivalent norms)
Let \(\VV\) be a vector space. Let \(\| \cdot \|_a : \VV \to \RR\) and \(\| \cdot \|_b : \VV \to \RR\) be two different norms defined on \(\VV\).
We say that the two norms are equivalent if there exist constants \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\), we have
Note that the zero-dimensional vector space \(\{\bzero \}\) has only one norm which is identically zero. Thus, the concept of equivalent norms is not interesting for it. Thus, we shall concern ourselves only with vector spaces of dimension greater than zero for the purposes of norm equivalence.
There are several ways to express the equivalent norm inequalities.
Theorem 4.56 (Characterization of equivalent norms)
Let \(\VV\) be a vector space. Let \(\| \cdot \|_a : \VV \to \RR\) and \(\| \cdot \|_b : \VV \to \RR\) be two different norms defined on \(\VV\).
The following statements are equivalent.
The norms \(\| \cdot \|_a\) and \(\| \cdot \|_b\) are equivalent.
There exist constants \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\)
\[ c_1 \| \bv \|_b \leq \| \bv \|_a \leq c_2 \| \bv \|_b. \]There exists \(c > 0\) such that for all \(\bv \in \VV\)
\[ \frac{1}{c} \| \bv \|_b \leq \| \bv \|_a \leq c \| \bv \|_b. \]
Proof. (1) \(\implies\) (2)
By hypothesis, the norms \(\| \cdot \|_a\) and \(\| \cdot \|_b\) are equivalent.
Thus, there exist \(d_1, d_2 > 0\) such that for every \(\bv \in \VV\)
\[ \| \bv \|_a \leq d_1 \| \bv \|_b \text{ and } \| \bv \|_b \leq d_2 \| \bv \|_a. \]Let \(c_1 = \frac{1}{d_2} > 0\) and \(c_2 = d_1 > 0\).
The inequalities become:
\[ \| \bv \|_a \leq c_2 \| \bv \|_b \text{ and } c_1 \| \bv \|_b \leq \| \bv \|_a. \]Combining, we get:
\[ c_1 \| \bv \|_b \leq \| \bv \|_a \leq c_2 \| \bv \|_b. \]
(2) \(\implies\) (3)
By hypothesis, there exist constants \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\)
\[ c_1 \| \bv \|_b \leq \| \bv \|_a \leq c_2 \| \bv \|_b. \]Let \(c = \max \left \{ \frac{1}{c_1}, c_2 \right \}\). Note that \(c > 0\) since \(c_1, c_2 > 0\).
Then, \(c \geq c_2\).
Thus, \(\| \bv \|_a \leq c_2 \| \bv \|_b \leq c \| \bv \|_b\) for every \(\bv \in \VV\).
Also, \(c \geq \frac{1}{c_1} \implies \frac{1}{c} \leq c_1\).
Thus, \(\frac{1}{c} \| \bv \|_b \leq c_1 \| \bv \|_b \leq \| \bv \|_a\) for every \(\bv \in \VV\).
Combining, we get:
\[ \frac{1}{c} \| \bv \|_b \leq \| \bv \|_a \leq c \| \bv \|_b. \]
(3) \(\implies\) (1)
By hypothesis, there exists \(c > 0\) such that for all \(\bv \in \VV\)
\[ \frac{1}{c} \| \bv \|_b \leq \| \bv \|_a \leq c \| \bv \|_b. \]Let \(c_1 = c\) and \(c_2 = c\).
Thus, \(\| \bv \|_a \leq c_1 \| \bv \|_b\) for all \(\bv \in \VV\).
And \(\frac{1}{c_2} \| \bv \|_b \leq \| \bv \|_a\) implies \(\| \bv \|_b \leq c_2 \| \bv \|_a\) for all \(\bv \in \VV\).
Thus, there exist constants \(c_1, c_2 > 0\) given by \(c_1 = c_2 = c\) such that for every \(\bv \in \VV\), we have
\[ \| \bv \|_a \leq c_1 \| \bv \|_b \text{ and } \| \bv \|_b \leq c_2 \| \bv \|_a. \]Thus, the two norms are equivalent.
Theorem 4.57 (Norm equivalence = Identical bounded sets)
Let \(\VV\) be a vector space. Let \(\| \cdot \|_a : \VV \to \RR\) and \(\| \cdot \|_b : \VV \to \RR\) be two different norms defined on \(\VV\).
The following statements are equivalent.
The norms \(\| \cdot \|_a\) and \(\| \cdot \|_b\) are equivalent.
Every set \(A \subseteq \VV\) is bounded in \((\VV, \| \cdot \|_a)\) if and only if it is bounded in \((\VV, \| \cdot \|_b)\).
Proof. (1) \(\implies\) (2)
By hypothesis, the norms \(\| \cdot \|_a\) and \(\| \cdot \|_b\) are equivalent.
Thus, there exist \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\)
\[ \| \bv \|_a \leq c_1 \| \bv \|_b \text{ and } \| \bv \|_b \leq c_2 \| \bv \|_a. \]Let \(A\) be bounded in \((\VV, \| \cdot \|_a)\).
Then, there exists a constant \(M > 0\) such that
\[ \| \bx \|_a \leq M \Forall \bx \in A. \]But then,
\[ \| \bx \|_b \leq c_2 \| \bx \|_a \leq M c_2 \Forall \bx \in A. \]Thus, \(A\) is bounded in \((\VV, \| \cdot \|_b)\).
A similar reasoning shows that if \(A\) is bounded in \((\VV, \| \cdot \|_b)\), then \(A\) must be bounded in \((\VV, \| \cdot \|_a)\) also.
(2) \(\implies\) (1)
By hypothesis, every set \(A \subseteq \VV\) is bounded in \((\VV, \| \cdot \|_a)\) if and only if it is bounded in \((\VV, \| \cdot \|_b)\).
For contradiction, assume that the norms are not equivalent and there is no constant \(c_1 > 0\) such that \(\| \bv \|_a \leq c_1 \| \bv \|_b\) for every \(\bv \in \VV\).
Then, for each \(n \in \Nat\), there exists \(\bx_n \in \VV\) such that \(\| \bx_n \|_a > n \| \bx_n \|_b\).
In particular, \(\bx_n \neq \bzero\).
The set \(S = \{ \frac{\bx_n}{\| \bx_n \|_b} | n \in \Nat \}\) is bounded in \((\VV, \| \cdot \|_b)\) since each point is unit norm.
Then, \(S\) is bounded in \((\VV, \| \cdot \|_a)\) also by hypothesis.
Thus, \(\left \| \frac{\bx_n}{\| \bx_n \|_b} \right \|_a \leq C\) for some \(C > 0\) and every \(n \in \Nat\).
It implies \(\| \bx_n \|_a \leq C \| \bx_n \|_b\) for every \(n \in \Nat\).
But, this contradictions with our choice above as \(\| \bx_n \|_a > n \| \bx_n \|_b\).
Thus, there must exist a constant \(c_1 > 0\) such that \(\| \bv \|_a \leq c_1 \| \bv \|_b\) for every \(\bv \in \VV\).
A similar reasoning shows that there must exist a constant \(c_2 > 0\) such that \(\| \bv \|_b \leq c_2 \| \bv \|_a\) for every \(\bv \in \VV\).
Thus, the two norms must be equivalent.
Theorem 4.58
Equivalence of norms is an equivalence relation on the set of norms for a given vector space.
Proof. Let \(\VV\) be a given vector space. Let \(\| \cdot \|_a : \VV \to \RR\), \(\| \cdot \|_b : \VV \to \RR\), \(\| \cdot \|_c : \VV \to \RR\), be norms defined on \(\VV\).
We shall say that \(\| \cdot \|_a \sim \| \cdot \|_b\) if the norms \(\| \cdot \|_a\) and \(\| \cdot \|_b\) are equivalent; i.e., there exist constants \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\), we have
[Reflexivity]
Choose \(c_1 = c_2 = 1\). Then,
\[ \| \bv \|_a \leq \| \bv \|_a \]for every \(\bv \in \VV\).
Hence, \(\| \cdot \|_a \sim \| \cdot \|_a\).
[Symmetry]
Let \(\| \cdot \|_a \sim \| \cdot \|_b\).
Then, there exist constants \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\), we have
\[ \| \bv \|_a \leq c_1 \| \bv \|_b \text{ and } \| \bv \|_b \leq c_2 \| \bv \|_a. \]Choose \(d_1 = c_2\) and \(d_2 = c_1\).
Then, for every \(\bv \in \VV\), we have
\[ \| \bv \|_b \leq d_1 \| \bv \|_a \text{ and } \| \bv \|_a \leq d_2 \| \bv \|_b. \]Thus, \(\| \cdot \|_b \sim \| \cdot \|_a\).
[Transitivity]
Let \(\| \cdot \|_a \sim \| \cdot \|_b\) and \(\| \cdot \|_b \sim \| \cdot \|_c\).
Then, there exist constants \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\), we have
\[ \| \bv \|_a \leq c_1 \| \bv \|_b \text{ and } \| \bv \|_b \leq c_2 \| \bv \|_a. \]And, there exist constants \(d_1, d_2 > 0\) such that for every \(\bv \in \VV\), we have
\[ \| \bv \|_b \leq d_1 \| \bv \|_c \text{ and } \| \bv \|_c \leq d_2 \| \bv \|_b. \]Let \(e_1 = c_1 d_1 > 0\) and \(e_2 = c_2 d_2 > 0\).
Then, for every \(\bv \in \VV\):
\[ \| \bv \|_a \leq c_1 \| \bv \|_b \leq c_1 (d_1 \| \bv \|_c) = (c_1 d_1)\| \bv \|_c = e_1 \| \bv \|_c. \]Similarly, for every \(\bv \in \VV\):
\[ \| \bv \|_c \leq d_2 \| \bv \|_b \leq d_2 (c_2 \| \bv \|_a) = (d_2 c_2) \| \bv \|_a = e_2 \| \bv \|_a. \]Thus, \(\| \cdot \|_a \sim \| \cdot \|_c\).
Theorem 4.59 (Norm equivance \(\implies\) metric strong equivalence)
If two norms are equivalent, then there associated metrics are strongly equivalent.
Proof. Let \(\VV\) be a given vector space. Let \(\| \cdot \|_a : \VV \to \RR\), and \(\| \cdot \|_b : \VV \to \RR\) be norms defined on \(\VV\) which are equivalent.
Then, there exist constants \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\), we have
The associated metrics are given by
Clearly, for every \(\bx, \by \in \VV\):
Similarly,
Thus, there exist \(c_1, c_2 > 0\) such that
Thus, the two associated metrics are strongly equivalent.
4.4.15.1. Norms on Finite Dimensional Spaces#
Theorem 4.60 (Equivalence of norms on a finite dimensional vector space)
Let \(\VV\) be a finite dimensional vector space over the scalar field \(\FF\) where \(\FF\) is \(\RR\) or \(\CC\). Then, all norms on \(\VV\) are equivalent.
Reaching this conclusion requires significant amount of work on the norms on Euclidean spaces which are discussed in detail in The Euclidean Space. Readers are advised to read the material on norms on \(\RR^n\) and the fact that all norms on \(\RR^n\) are equivalent before proceeding further.
Proof. If \(0 = \dim \VV\), then there is only one norm and there is nothing to prove. So assume that \(\dim \VV > 0\). Then, \(\VV\) is isomorphic to \(\RR^n\) for some \(n\).
Let \(L : \RR^n \to \VV\) be an isomorphism.
Now, if \(\| \cdot \|\) is a norm on \(\VV\), then the function \(\| \cdot \|_L : \RR^n \to \RR\) defined by
\[ \| \bx \|_L = \| L (\bx)\| \]is a norm on \(\RR^n\).
Let \(\| \cdot \|_a\) and \(\| \cdot \|_b\) be two different norms on \(\VV\).
Let \(\| \cdot \|_{L,a}\) and \(\| \cdot \|_{L,b}\) be the corresponding induced norms on \(\RR^n\).
By Theorem 4.117, all norms on \(\RR^n\) are equivalent.
Hence, \(\| \cdot \|_{L,a}\) and \(\| \cdot \|_{L,b}\) are equivalent.
Hence, there exist constants \(c_1, c_2 > 0\) such that
\[ \| \bx \|_{L, a} \leq c_1 \| \bx \|_{L, b} \text{ and } \| \bx \|_{L, b} \leq c_2 \| \bx \|_{L, a} \]holds true for every \(\bx \in \RR^n\).
Then, for every \(\bv \in \VV\)
\[\begin{split} \| \bv \|_a &= \| L (L^{-1} (\bv)) \|_a\\ &= \| L^{-1} (\bv) \|_{L, a} \\ &\leq c_1 \| L^{-1} (\bv) \|_{L, b}\\ &= c_1 \| L (L^{-1} (\bv)) \|_b \\ &= c_1 \| \bv \|_b. \end{split}\]Similarly, for every \(\bv \in \VV\)
\[ \| \bv \|_b \leq c_2 \| \bv \|_a \]holds true.
Thus, the two norms are equivalent.
Since, the two norms chosen were arbitrary, hence all norms on \(\VV\) are equivalent.
Definition 4.69 (Norm topology)
Let \(\VV\) be a finite dimensional space. Since all norms on \(\VV\) are equivalent, hence, they induce the same topology (family of open sets, closed sets, compact sets). The topology induced by a norm (the collection of open sets) on a finite dimensional space is called its norm topology.
4.4.15.2. Continuity#
Continuity of a function doesn’t depend on the choice of a norm within the class of all equivalent norms. In other words, if a function \(f\) is continuous for a given norm, then it is continuous for all norms equivalent to it.
Theorem 4.61 (Continuity with equivalent norms)
Let \(\VV\) be a given vector space. Let \(\| \cdot \|_a : \VV \to \RR\), and \(\| \cdot \|_b : \VV \to \RR\) be norms defined on \(\VV\) which are equivalent. Let \(d_a\) and \(d_b\) be corresponding metrics. Let \((X, d)\) be a metric space.
Let \(f : X \to \VV\) be a (total) function.
Then, \(f : (X, d) \to (\VV, d_a)\) is continuous if and only if \(f : (X, d) \to (\VV, d_b)\) is continuous.
Similarly, let \(g : \VV \to X\) be a (total) function.
Then, \(g : (\VV, d_a) \to (X, d)\) is continuous if and only if \(g : (\VV, d_b) \to (X, d)\) is continuous.
Proof. By Theorem 4.59, the metrics \(d_a\) and \(d_b\) are strongly equivalent. By Theorem 3.28, they are equivalent. Thus, they determine the same topology on \(\VV\). Thus, a subset \(A \subseteq \VV\) is open in \((\VV, d_a)\) if and only if it is open in \((\VV, d_b)\).
Now, assume \(f : (X, d) \to (\VV, d_a)\) to be continuous.
Let \(A\) be an open set of \((\VV, d_a)\).
Then, \(f^{-1}(A)\) is an open set of \((X, d)\) since \(f : (X, d) \to (\VV, d_a)\) is continuous (Theorem 3.42).
But \(A\) is also an open set of \((\VV, d_b)\) since both metrics determine same open sets.
Thus, whenever \(A\) is an open set of \((\VV, d_b)\), \(f^{-1}(A)\) is an open set of \((X,d)\).
Thus, \(f : (X, d) \to (\VV, d_b)\) is continuous due to Theorem 3.42.
A similar reasoning shows that if \(f : (X, d) \to (\VV, d_b)\) is continuous then \(f : (X, d) \to (\VV, d_a)\) must be continuous too.
Combining, \(f : (X, d) \to (\VV, d_a)\) is continuous if and only if \(f : (X, d) \to (\VV, d_b)\) is continuous.
Now, assume \(g : (\VV, d_a) \to (X, d)\) to be continuous.
Let \(A\) be an open set of \((X, d)\).
Then, \(g^{-1}(A)\) is an open set of \((\VV, d_a)\) since \(g : (\VV, d_a) \to (X, d)\) is continuous (Theorem 3.42).
But \(g^{-1}(A)\) is also an open set of \((\VV, d_b)\) since both metrics determine same open sets.
Thus, whenever \(A\) is an open set of \((X, d)\), \(g^{-1}(A)\) is an open set of \((\VV, d_b)\).
Thus, \(g : (\VV, d_b) \to (X, d)\) is continuous due to Theorem 3.42.
A similar reasoning shows that if \(g : (\VV, d_b) \to (X, d)\) is continuous then \(g : (\VV, d_a) \to (X, d)\) must be continuous too.
Combining, \(g : (\VV, d_a) \to (X, d)\) is continuous if and only if \(g : (\VV, d_b) \to (X, d)\) is continuous.
Remark 4.9 (Continuity in finite dimensional spaces)
In the special case where \(\VV\) is finite dimensional, the continuity of \(f\) or \(g\) is independent of the choice of norm chosen on \(\VV\) since all norms are equivalent.
4.4.16. Linear Transformations in Finite Dimensional Spaces#
Theorem 4.62 (Linear transformations in finite dimensional spaces are bounded)
Let \((\VV, \| \cdot \|_v)\) and \((\WW, \| \cdot \|_w)\) be normed linear spaces. Let \(T : \VV \to \WW\) be a linear transformation.
If \(\VV\) is finite dimensional, then \(T\) is bounded.
Proof. If \(\dim \VV = 0\), then \(\VV = \{ \bzero \}\) and any linear transformation is bounded. Hence, let \(\dim \VV > 0\).
Since \(\VV\) is finite dimensional, we can choose a basis \(\BBB = \{\be_1, \dots, \be_n \}\) for \(\VV\).
Let \(c_i = \| T (\be_i) \|_w\) for \(i=1,\dots,n\) and \(c = \max \{ c_1, \dots, c_n \}\).
Let \(\bx \in \VV\).
It can be uniquely written as
\[ \bx = t_1 \be_1 + \dots + t_n \be_n. \]Then,
\[\begin{split} \| T (\bx) \|_w &= \| T (t_1 \be_1 + \dots + t_n \be_n) \|\\ &= \| t_1 T(\be_1) + \dots + t_n T(\be_n) \| \\ &\leq |t_1| \| T (\be_1) \|_w + \dots + |t_n | \| T (\be_n) \|_w \\ &\leq c (|t_1 | + \dots + |t_n |). \end{split}\]Note that the function \( \| \cdot \|_* : \VV \to \RR\) given by
\[ \| \bx\|_* = |t_1 | + \dots + |t_n | \]is a norm on \(\VV\).
By Theorem 4.60, all norms on \(\VV\) are equivalent since \(\VV\) is finite dimensional.
Thus, there exists \(c_1 > 0\) such that
\[ \| \bx\|_* \leq c_1 \| \bx\|_v \]for every \(\bx \in \VV\).
Thus, \(\| T (\bx) \|_w \leq c c_1 \| \bx\|_v\) for every \(\bx \in \VV\).
Thus, for every nonzero \(\bx \in \VV\),
\[ \frac{\| T (\bx) \|_w}{\| \bx\|_v} \leq c c_1. \]Thus, the set \(S\) in (4.1) is bounded.
Thus, \(T\) is bounded.
Theorem 4.63 (Linear transformations in finite dimensional spaces are continuous)
Let \((\VV, \| \cdot \|_v)\) and \((\WW, \| \cdot \|_w)\) be normed linear spaces. Let \(T : \VV \to \WW\) be a linear transformation.
If \(\VV\) is finite dimensional, then \(T\) is continuous as well as uniformly continuous as well as Lipschitz continuous.
Proof. By Theorem 4.62, \(T\) is bounded. The rest follows from the characterization of continuous linear transformations on vector spaces in Theorem 4.55.
4.4.17. Linear Subspaces#
Theorem 4.64 (Linear subspaces are closed)
Every subspace of a finite dimensional normed linear space \(\VV\) is a closed set.
Proof. Let \(\VV\) be a finite dimensional subspace equipped with a norm \(\| \cdot \|\).
The trivial subspace \(\{\bzero \}\) is closed since it is a singleton. The space \(\VV\) is closed by definition.
Let \(W\) be a proper nontrivial subspace of \(\VV\). Then, there exists a linear transformation \(T\) such that
i.e., \(W\) is the kernel of \(T\).
Then,
By Theorem 4.63, \(T\) is continuous.
And the set \(\{ \bzero \}\) is a singleton, hence closed.
Then, by Theorem 3.42 (5), \(W = T^{-1}(\{ \bzero \})\) is a closed set.
4.4.18. Completeness and Banach Spaces#
Definition 4.70 (Banach space)
A normed space \(\VV\) that is complete with respect to the metric induced by its norm is called a Banach space.
In other words, \(\VV\) is a Banach space if every Cauchy sequence of \(\VV\) converges in \(\VV\).
Example 4.19 (Examples of Banach spaces)
The spaces in examples below have been described in detail elsewhere. Follow the links.
The Euclidean space \(\RR^n\) is complete with respect to the Euclidean norm. Thus, it is a Banach space.
The space of bounded real valued functions \(B(X)\) over a nonempty set \(X\) equipped with sup norm is complete.
Theorem 4.65 (\(n\)-dim normed linear spaces are complete)
Let \(\VV\) be an \(n\)-dimensional vector space over \(\FF\) where \(\FF\) is either \(\RR\) or \(\CC\). Let \(\VV\) be equipped with a norm \(\| \cdot \| : \VV \to \RR\) making it a normed linear space. Then \(\VV\) is complete.
In other words, every finite dimensional normed linear space is complete.
Proof. \(\VV\) is a normed linear space with \(n = \dim \VV\).
Let \(\BBB = \{\be_1, \dots, \be_n \}\) be a basis for \(\VV\).
For any \(\bv \in \VV\), we can represent it in the basis \(\BBB\) as
\[ \bv = v_1 \be_1 + \dots + v_n \be_n. \]We can define the \(\| \cdot \|_1\) norm as
\[ \| \bv \|_1 = |v_1| + \dots + | v_n |. \]By Theorem 4.60, all norms on \(\VV\) are equivalent.
Thus, \(\| \cdot \|_1\) and \(\| \cdot \|\) are equivalent.
Thus, there are \(c_1, c_2 > 0\) such that for every \(\bv \in \VV\),
\[ c_1 \| \bv \|_1 \leq \| \bv \| \leq c_2 \| \bv \|_1. \]Let \(\{ \bv_k \}\) be a Cauchy sequence of \(\VV\).
Let \(\epsilon > 0\).
Since \(\{ \bv_k \}\) is a Cauchy sequence of \(\VV\), hence there exists \(N\) such that for every \(k, l > N\),
\[ \| \bv_k - \bv_l \| < \epsilon. \]Then,
\[\begin{split} \epsilon &> \| \bv_k - \bv_l \| \\ &\geq c_1 \| \bv_k - \bv_l \|_1 \\ &= c_1 \sum_{i=1}^n | v_{k i} - v_{l i} |\\ &\geq c_1 | v_{k i} - v_{l i} | \end{split}\]for every \(i \in 1,\dots,n\).
Hence \(\{ v_{k i} \}\) is a Cauchy sequence of \(\FF\) for every \(i \in 1,\dots,n\).
Since both \(\RR\) and \(\CC\) are complete, hence \(\{ v_{k i} \}\) is a convergent sequence of \(\FF\) for every \(i \in 1,\dots,n\).
Thus, there exists \(u_i = \lim_{n \to \infty} v_{k i}\) for every \(i \in 1,\dots,n\).
Now, let \(\bu = (u_1, \dots, u_n)\).
Then,
\[\begin{split} \| \bv_k - \bu \| &\leq c_2 \| \bv_k - \bu \|_1 \\ &= c_2 \sum_{i=1}^n | v_{k i} - u_i |. \end{split}\]Thus,
\[\begin{split} \lim_{k \to \infty} \| \bv_k - \bu \| &= \lim_{k \to \infty} c_2 \sum_{i=1}^n | v_{k i} - u_i | \\ &= c_2 \sum_{i=1}^n \lim_{k \to \infty} | v_{k i} - u_i | = 0. \end{split}\]Thus, \(\lim_{k \to \infty} \bv_k = \bu\).
Thus, \(\{ \bv_k \}\) is convergent.
Thus, every Cauchy sequence of \(\VV\) is convergent.
Thus, \(\VV\) is complete.
4.4.19. Compact Sets#
In this subsection, we pay special attention to the compact subsets of \(n\)-dimensional real normed linear spaces. We show that in such spaces, closed and bounded sets are compact. We provides some results related to set arithmetic for compact sets. We further establish the Bolzano Weierstrass theorems for such spaces. Recall from Definition 3.57 that a set has Bolzano-Weierstrass property if every sequence of the set has a convergent subsequence that converges to a point in the set.
Theorem 4.66 (Compact = Closed and Bounded in \(n\)-dim)
Let \(\VV\) be a real \(n\)-dimensional normed linear space. Let \(S\) be a subset of \(\VV\).
Then, \(S\) is compact if and only if \(S\) is closed and bounded.
Proof. By Theorem 3.76, every compact set is closed and bounded. Thus, if \(S\) is compact then \(S\) is closed and bounded.
For the converse, assume that \(S\) is closed and bounded.
By Theorem 4.65, both \(\RR^n\) and \(\VV\) are complete.
Let \(T: \RR^n \to \VV\) be an isomorphism.
Then, for any \(\bv \in \RR^n\), a function \(\| \cdot \|_T : \RR^n \to \RR\) given by
\[ \| \bv \|_T = \| T (\bv)\| \]defines a norm on \(\RR^n\).
The norm \(\| \cdot \|_T\) is equivalent to the Euclidean norm on \(\RR^n\) due to Theorem 4.60.
Also, \(T : (\RR^n, \| \cdot \|_T) \to (\VV, \| \cdot \|)\) is an isometry
Accordingly \(T\) is uniformly continuous by Theorem 3.54.
Since \(T\) is an isomorphism, it is bijective.
Thus, \(T\) is a homeomorphism by Theorem 3.55.
Then, \(T^{-1}\) is also an isometry.
Then, \(T^{-1}(S)\) is bounded in \((\RR^n, \| \cdot \|_T)\) since \(S\) is bounded in \((\VV, \| \cdot \|)\) and \(T^{-1}\) is an isometry.
By Theorem 4.57, \(T^{-1}(S)\) is also bounded in \((\RR^n, \| \cdot \|_2)\).
Also, \(T^{-1}(S)\) is closed in \((\RR^n, \| \cdot \|_T)\). since \(S\) is closed in \((\VV, \| \cdot \|)\) and \(T\) is continuous (see Theorem 3.42).
Since equivalent norms (metrics) determine same topology, hence \(T^{-1}(S)\) is closed in \((\RR^n, \| \cdot \|_2)\) also.
Thus, \(T^{-1}(S)\) is closed and bounded in \((\RR^n, \| \cdot \|_2)\).
By Heine-Borel theorem, \(T^{-1}(S)\) is a compact set in \((\RR^n, \| \cdot \|_2)\).
By Theorem 3.90, \(T^{-1}(S)\) is a compact set in \((\RR^n, \| \cdot \|_T)\) also.
Then, \(S = T (T^{-1}(S))\) is also compact in \((\VV, \| \cdot \|)\) since a homeomorphism preserves compactness (see Theorem 3.80).
Theorem 4.67 (Sum of compact sets is compact)
Let \(\VV\) be a real \(n\)-dimensional normed linear space. Let \(A, B \subseteq \VV\) be compact subsets of \(\VV\). Then, their sum \(A+B\) is compact.
Proof. We proceed as follows
Both \(A\) and \(B\) are compact. Hence, they are closed and bounded.
By Theorem 4.50, \(A+B\) is closed.
Let \(\| \ba \| \leq M_a\) for every \(\ba \in A\).
Let \(\| \bb \| \leq M_b\) for every \(\bb \in B\).
Let \(\bx \in A + B\).
Then, there exists \(\ba \in A\) and \(\bb \in B\) such that \(\bx = \ba + \bb\).
Then,
\[ \| \bx \| = \| \ba + \bb \| \leq \| \ba \| + \| \bb \| \leq M_a + M_b. \]Thus, \(\| \bx \| \leq M_a + M_b\) for every \(\bx \in A + B\).
Thus, \(A+B\) is bounded.
Since \(A+B\) is closed and bounded, hence \(A+B\) is compact due to Theorem 4.66.
Theorem 4.68 (Bolzano Weierstrass theorem for bounded subsets)
Let \(\VV\) be a real \(n\)-dimensional normed linear space. Let \(A\) be a bounded subset of \(\VV\). Then, every sequence of \(A\) has a convergent subsequence.
If \(A\) is closed, then the subsequence converges in \(A\) itself. Otherwise, the subsequence converges in \(\closure A\).
Proof. We are given that \(A\) is bounded.
Then, there exist \(M > 0\) such that:
\[ \| \bx \| \leq M \Forall \bx \in A. \]Thus, \(A \subseteq B[\bzero, M]\).
\(B[\bzero, M]\) is a closed and bounded subset of \(\VV\).
\(\closure A \subseteq B[\bzero, M]\) since \(\closure A\) is the smallest closed set containing \(A\).
Thus, \(\closure A\) is closed and bounded.
By Theorem 4.66, \(\closure A\) is compact.
Let \(\{ \bx_k \}\) be a sequence of \(A\). Then, it is also a sequence of \(\closure A\).
By Theorem 3.75, \(\{ \bx_k \}\) has a subsequence that converges in \(\closure A\).
If \(A\) is closed, then \(\closure A = A\) and we are done.
Theorem 4.69 (Bolzano Weierstrass theorem for bounded sequences)
Let \(\VV\) be a real \(n\)-dimensional normed linear space. Every bounded sequence of \(\VV\) has a convergent subsequence.
Proof. Let \(\{ \bx_k \}\) be a bounded sequence of \(\VV\).
Then there exists a closed ball \(B[\bzero, M]\) such that \(\{ \bx_k \} \subset B[\bzero, M]\).
\(B[\bzero, M]\) is closed and bounded.
By Theorem 4.66, \(B[\bzero, M]\) is compact.
By Theorem 3.75, \(\{ \bx_k \}\) has a subsequence that converges in \(B[\bzero, M]\).