4.11. Important Vector Spaces

In this section, we will list some important vector spaces which occur frequently in analysis and optimization.

4.11.1. The Vector Space of Symmetric Matrices

Recall from Definition 1.111 that the set of real symmetric matrices is given by

$${\mathbb{S}}^n=\{\mathbf{X}\in {\mathbb{R}}^{n\times n}|\mathbf{X}={\mathbf{X}}^T\}.$$

Theorem 4.139 (The vector space of symmetric matrices)

The set ${\mathbb{S}}^n$ is a vector space with dimension $\frac{n(n+1)}{2}$.

Proof. It suffices to show that any linear combination of symmetric matrices is also symmetric. The dimension of this vector space comes from the number of entries in a symmetric matrix which can be independently chosen.

Definition 4.134 (Matrix inner product)

An inner-product on the vector space of $n\times n$ real matrices can be defined as

$$⟨\mathbf{A},\mathbf{B}⟩\triangleq \sum _i\sum _j{A}_{i,j}{B}_{i,j}=\mathrm{tr}({\mathbf{A}}^T\mathbf{B})=\mathrm{tr}({\mathbf{B}}^T\mathbf{A}).$$

This is known as the Frobenius inner product.

Remark 4.28

Equipped with this inner product as defined in Definition 4.134, ${\mathbb{S}}^n$ is a finite dimensional real inner product space.

4.11.2. The Vector Space of Real Valued Functions

Definition 4.135 (The vector space of (total) real valued functions)

Let $X$ be a non-empty set. Let $\mathcal{F}(X,\mathbb{R})$ be the set of real valued total functions on $X$. The set $\mathcal{F}(X,\mathbb{R})$ is a vector space over the scalar field of $\mathbb{R}$ with the definitions following Definition 2.47:

Vector addition: If $f,g\in \mathcal{F}(X,\mathbb{R})$, then $h=f+g$ is defined as:

$$h(\mathbf{x})\triangleq f(\mathbf{x})+g(\mathbf{x})\mathrm{\forall }\mathbf{x}\in X.$$

Scalar multiplication: if $\alpha \in \mathbb{R}$ and $f\in \mathcal{F}(X,\mathbb{R})$, then $h=\alpha f$ is defined as:

$$h(\mathbf{x})\triangleq \alpha f(\mathbf{x})\mathrm{\forall }X.$$

Additive identity: There exists a function $\mathbf{0}\in \mathcal{F}(X,\mathbb{R})$ given by:

$$\mathbf{0}(\mathbf{x})=0\mathrm{\forall }\mathbf{x}\in X.$$

4.11.3. The Vector Space of Bounded Functions

It was discussed earlier in Example 3.20.

Recall from Definition 2.50 that a real valued (total) function $f:X\to \mathbb{R}$ is called bounded if there exists a number $M\ge 0$ (depending on $f$) such that

$$|f(x)|\le M\mathrm{\forall }x\in X.$$

Definition 4.136 (The vector space of bounded functions)

Let $X$ be a non-empty set. Let $B(X)$ be the set of bounded functions on $X$. The set $B(X)$ is a vector space of bounded functions over the scalar field of $\mathbb{R}$ with the following operations:

Vector addition: If $f,g\in B(X)$, then $h=f+g$ is defined as:

$$h(x)\triangleq f(x)+g(x)\mathrm{\forall }x\in X.$$

Scalar multiplication: if $\alpha \in \mathbb{R}$, then $h=\alpha f$ is defined as:

$$h(x)\triangleq \alpha f(x)\mathrm{\forall }x\in X.$$

Definition 4.137 (Sup norm for the space of bounded functions)

The standard norm for $B(X)$ is defined for any $f\in B(X)$ as:

$$\Vert f\Vert \triangleq sup\{|f(x)|\mathrm{\forall }x\in X\}.$$

This norm is known as sup norm and often written as $\Vert f{\Vert }_{\mathrm{\infty }}$.

Definition 4.138 (Metric induced by the norm)

The standard metric induced by the standard norm for $B(X)$ is defined for any $f,g\in B(X)$ as:

$$d(f,g)\triangleq \Vert f-g\Vert =sup\{|f(x)-g(x)|\mathrm{\forall }x\in X\}.$$

Theorem 4.140 ($B(X)$ is complete)

The normed vector space $B(X)$ is complete. Thus, $B(X)$ is a Banach space.

Proof. See Example 3.20 for the detailed proof.