4.11. Important Vector Spaces#

In this section, we will list some important vector spaces which occur frequently in analysis and optimization.

4.11.1. The Vector Space of Symmetric Matrices#

Recall from Definition 1.111 that the set of real symmetric matrices is given by

\[ \SS^n = \{\bX \in \RR^{n \times n} | \bX = \bX^T\}. \]

Theorem 4.139 (The vector space of symmetric matrices)

The set \(\SS^n\) is a vector space with dimension \(\frac{n(n+1)}{2}\).

Proof. It suffices to show that any linear combination of symmetric matrices is also symmetric. The dimension of this vector space comes from the number of entries in a symmetric matrix which can be independently chosen.

Definition 4.134 (Matrix inner product)

An inner-product on the vector space of \(n \times n\) real matrices can be defined as

\[ \langle \bA, \bB \rangle \triangleq \sum_i \sum_j A_{i,j} B_{i, j} = \Trace (\bA^T \bB) = \Trace (\bB^T \bA). \]

This is known as the Frobenius inner product.

Remark 4.28

Equipped with this inner product as defined in Definition 4.134, \(\SS^n\) is a finite dimensional real inner product space.

4.11.2. The Vector Space of Real Valued Functions#

Definition 4.135 (The vector space of (total) real valued functions)

Let \(X\) be a non-empty set. Let \(\FFF (X, \RR)\) be the set of real valued total functions on \(X\). The set \(\FFF (X, \RR)\) is a vector space over the scalar field of \(\RR\) with the definitions following Definition 2.47:

Vector addition: If \(f,g \in \FFF (X, \RR)\), then \(h = f + g\) is defined as:

\[ h(\bx) \triangleq f(\bx) + g(\bx) \Forall \bx \in X. \]

Scalar multiplication: if \(\alpha \in \RR\) and \(f \in \FFF (X, \RR)\), then \(h = \alpha f\) is defined as:

\[ h (\bx) \triangleq \alpha f(\bx) \Forall X. \]

Additive identity: There exists a function \(\bzero \in \FFF (X, \RR)\) given by:

\[ \bzero(\bx) = 0 \Forall \bx \in X. \]

4.11.3. The Vector Space of Bounded Functions#

It was discussed earlier in Example 3.20.

Recall from Definition 2.50 that a real valued (total) function \(f: X \to \RR\) is called bounded if there exists a number \(M \geq 0\) (depending on \(f\)) such that

\[ |f(x)| \leq M \Forall x \in X. \]

Definition 4.136 (The vector space of bounded functions)

Let \(X\) be a non-empty set. Let \(B(X)\) be the set of bounded functions on \(X\). The set \(B(X)\) is a vector space of bounded functions over the scalar field of \(\RR\) with the following operations:

Vector addition: If \(f,g \in B(X)\), then \(h = f + g\) is defined as:

\[ h(x) \triangleq f(x) + g(x) \Forall x \in X. \]

Scalar multiplication: if \(\alpha \in \RR\), then \(h = \alpha f\) is defined as:

\[ h (x) \triangleq \alpha f(x) \Forall x \in X. \]

Definition 4.137 (Sup norm for the space of bounded functions)

The standard norm for \(B(X)\) is defined for any \(f \in B(X)\) as:

\[ \| f \| \triangleq \sup \{ |f(x) | \Forall x \in X\}. \]

This norm is known as sup norm and often written as \(\| f \|_{\infty}\).

Definition 4.138 (Metric induced by the norm)

The standard metric induced by the standard norm for \(B(X)\) is defined for any \(f,g \in B(X)\) as:

\[ d(f,g) \triangleq \| f - g \| = \sup \{ | f(x) - g(x) | \Forall x \in X \}. \]

Theorem 4.140 (\(B(X)\) is complete)

The normed vector space \(B(X)\) is complete. Thus, \(B(X)\) is a Banach space.

Proof. See Example 3.20 for the detailed proof.