# 4.11. Important Vector Spaces#

In this section, we will list some important vector spaces which occur frequently in analysis and optimization.

## 4.11.1. The Vector Space of Symmetric Matrices#

Recall from Definition 1.111 that the set of real symmetric matrices is given by

$\SS^n = \{\bX \in \RR^{n \times n} | \bX = \bX^T\}.$

Theorem 4.139 (The vector space of symmetric matrices)

The set $$\SS^n$$ is a vector space with dimension $$\frac{n(n+1)}{2}$$.

Proof. It suffices to show that any linear combination of symmetric matrices is also symmetric. The dimension of this vector space comes from the number of entries in a symmetric matrix which can be independently chosen.

Definition 4.134 (Matrix inner product)

An inner-product on the vector space of $$n \times n$$ real matrices can be defined as

$\langle \bA, \bB \rangle \triangleq \sum_i \sum_j A_{i,j} B_{i, j} = \Trace (\bA^T \bB) = \Trace (\bB^T \bA).$

This is known as the Frobenius inner product.

Remark 4.28

Equipped with this inner product as defined in Definition 4.134, $$\SS^n$$ is a finite dimensional real inner product space.

## 4.11.2. The Vector Space of Real Valued Functions#

Definition 4.135 (The vector space of (total) real valued functions)

Let $$X$$ be a non-empty set. Let $$\FFF (X, \RR)$$ be the set of real valued total functions on $$X$$. The set $$\FFF (X, \RR)$$ is a vector space over the scalar field of $$\RR$$ with the definitions following Definition 2.47:

Vector addition: If $$f,g \in \FFF (X, \RR)$$, then $$h = f + g$$ is defined as:

$h(\bx) \triangleq f(\bx) + g(\bx) \Forall \bx \in X.$

Scalar multiplication: if $$\alpha \in \RR$$ and $$f \in \FFF (X, \RR)$$, then $$h = \alpha f$$ is defined as:

$h (\bx) \triangleq \alpha f(\bx) \Forall X.$

Additive identity: There exists a function $$\bzero \in \FFF (X, \RR)$$ given by:

$\bzero(\bx) = 0 \Forall \bx \in X.$

## 4.11.3. The Vector Space of Bounded Functions#

It was discussed earlier in Example 3.20.

Recall from Definition 2.50 that a real valued (total) function $$f: X \to \RR$$ is called bounded if there exists a number $$M \geq 0$$ (depending on $$f$$) such that

$|f(x)| \leq M \Forall x \in X.$

Definition 4.136 (The vector space of bounded functions)

Let $$X$$ be a non-empty set. Let $$B(X)$$ be the set of bounded functions on $$X$$. The set $$B(X)$$ is a vector space of bounded functions over the scalar field of $$\RR$$ with the following operations:

Vector addition: If $$f,g \in B(X)$$, then $$h = f + g$$ is defined as:

$h(x) \triangleq f(x) + g(x) \Forall x \in X.$

Scalar multiplication: if $$\alpha \in \RR$$, then $$h = \alpha f$$ is defined as:

$h (x) \triangleq \alpha f(x) \Forall x \in X.$

Definition 4.137 (Sup norm for the space of bounded functions)

The standard norm for $$B(X)$$ is defined for any $$f \in B(X)$$ as:

$\| f \| \triangleq \sup \{ |f(x) | \Forall x \in X\}.$

This norm is known as sup norm and often written as $$\| f \|_{\infty}$$.

Definition 4.138 (Metric induced by the norm)

The standard metric induced by the standard norm for $$B(X)$$ is defined for any $$f,g \in B(X)$$ as:

$d(f,g) \triangleq \| f - g \| = \sup \{ | f(x) - g(x) | \Forall x \in X \}.$

Theorem 4.140 ($$B(X)$$ is complete)

The normed vector space $$B(X)$$ is complete. Thus, $$B(X)$$ is a Banach space.

Proof. See Example 3.20 for the detailed proof.