Continuity
Contents
9.13. Continuity#
This section focuses on topological properties of convex functions in normed linear spaces. In particular, we discuss closure of convex functions, continuity of convex functions at interior points.
Main references for this section are [6, 17, 67].
Throughout this section, we assume that \(\VV\) is a finite dimensional real normed linear space equipped with a norm \(\| \cdot \| : \VV \to \RR\). It is also equipped with a metric \(d(\bx, \by) = \| \bx - \by \|\). Wherever necessary, it is also equppied with an real inner product \(\langle \cdot, \cdot \rangle : \VV \times \VV \to \RR\).
We recall that a function \(f : \VV \to \RR\) is \(L\)-Lipschitz if
The concept of semicontinuity, inferior and superior limits and closedness of real valued functions in metric spaces is discussed in detail in Real Valued Functions.
We recall some results.
Let \(f : \VV \to \RR\) with \(S = \dom f\) be a function.
Let \(\ba\) be an accumulation point of \(S\). The limit superior of \(f\) at \(\ba\) is defined by
The limit inferior of \(f\) at \(\ba\) is defined by
\(f\) is lower-semicontinuous at \(\ba \in S\) if for every \(\epsilon > 0\), there exists \(\delta > 0\) such that
\(f\) is upper-semicontinuous at \(\ba \in S\) if for every \(\epsilon > 0\), there exists \(\delta > 0\) such that
We say that \(f\) is lower semicontinuous (l.s.c.) if \(f\) is l.s.c. at every point of \(S\). Similarly, we say that \(f\) is upper semicontinuous (u.s.c.) if \(f\) is u.s.c. at every point of \(S\).
\(f\) is continuous at \(\ba \in S\) if and only if \(f\) is l.s.c. as well as u.s.c. at \(\ba\).
Let \(\ba \in S\) be an accumulation point of \(S\). Then, \(f\) is lower semicontinuous at \(\ba\) if and only if
Similarly, \(f\) is upper semicontinuous at \(\ba\) if and only if
Let \(\ba \in S\). Then, \(f\) is l.s.c. at \(\ba\) if and only if for every sequence \(\{\bx_n \}\) of \(S\) that converges to \(\ba\),
Similarly, \(f\) is upper semicontinuous at \(\ba\) if and only if every sequence \(\{\bx_k \}\) of \(S\) that converges to \(\ba\),
The following conditions are equivalent.
\(f\) is l.s.c.
\(f\) is closed; i.e., every sublevel set of \(f\) is closed with respect to the subspace topology \((S, \| \cdot \|)\).
\(\epi f\) is closed in \(\VV \oplus \RR\).
9.13.1. Closure#
Definition 9.63 (Closure of a convex function)
Let \(f : \VV \to \RR\) be a convex function. Then, its closure is defined to be the lower semicontinuous hull of \(f\).
If \(g\) is the closure of \(f\), then
If \(f: \VV \to \RERL\) is a proper convex function, then also, its closure is defined to be the lower semicontinuous hull.
If \(f: \VV \to \ERL\) is an improper convex function which attains a value \(f(\bx) = -\infty\) at some \(\bx \in \VV\), then its closure is defined to be the constant function \(g(\bx) = -\infty\) for all \(\bx \in \VV\).
The closure of a convex function is denoted by \(\closure f\).
Theorem 9.172 (Closure of a convex function is convex)
The closure of a convex function is convex.
Proof. Let \(f\) be a convex function. Then, \(\epi f\) is convex. Let \(g\) be its closure.
If \(f\) is improper, then \(g\) is a constant function, hence convex.
Otherwise, \(\epi g = \closure \epi f\).
Since \(f\) is convex, hence \(\epi f\) is convex.
Due to Theorem 9.123, the closure of a convex set is convex.
Hence, \(\epi g\) is convex.
Hence, \(g\) is convex.
Definition 9.64 (Closed convex function)
A convex function \(f: \VV \to \ERL\) is called closed if
For a proper convex function, closedness is same as lower semicontinuity.
The only closed improper convex functions are
\(f(\bx) = \infty \Forall \bx \in \VV\). Here \(\epi f = \EmptySet\).
\(f(\bx) = -\infty \Forall \bx \in \VV\). Here \(\epi f = \VV \times \RR\).
Example 9.62 (Closed convex function with open domain)
Let \(f: \RR \to \RERL\) be given as
Then, \(\dom f = (0, \infty)\).
\(\dom f\) is an open interval in \(\RR\).
\(f\) is continuous at every \(x > 0\).
Thus, \(f\) is l.s.c. at every \(x > 0\).
Thus, \(f\) is l.s.c.
Let the sublevel set for \(r \in \RR\) be given by
\[ T_r = \{ x \in (0, \infty) \ST f(x) \leq r \}. \]We can see that \(T_r = \EmptySet\) for \(r \leq 0\).
For \(r > 0\),
\[ f(x) = \frac{1}{x} \leq r \iff x \geq \frac{1}{r}. \]Thus, \(T_r = [\frac{1}{r}, \infty)\).
\(T_r\) is indeed closed in the topology \((\dom f, | \cdot |)\).
Since \(f\) is l.s.c., hence \(\epi f\) is closed.
Thus, \(\closure f = f\).
Thus, \(f\) is a closed convex function.
Remark 9.11 (Closure of proper convex functions and epigraph)
Let \(f: \VV \to \RERL\) be a proper convex function. Then,
This follows from the definition, since \(g = \closure f\) is defined by the fact that
9.13.2. Continuity#
Recall from Definition 3.38 that a function \(f: \VV \to \RERL\) is continuous at a point \(\ba \in \dom f\) if for every \(\epsilon > 0\), there exists \(\delta > 0\) (depending on \(\epsilon\) and \(\ba\)) such that for every \(\bx \in \dom f\)
holds true. In other words,
Convex functions are not necessarily continuous on non-open sets.
Example 9.63 (A convex function which is not continuous)
Let \(f : \RR \to \RR\) be given by
We can see that \(\dom f = [0, 1]\). \(f\) is continuous on \((0,1)\) but \(f\) is not continuous (from the right) at \(x=0\). It is continuous (from the left) at \(x=1\).
Convex functions are continuous at points in the interior of their domain.
9.13.2.1. Continuity of Univariate Closed Convex Functions#
Theorem 9.173 (Continuity of closed convex univariate functions)
Let \(f: \RR \to \RERL\) be a proper closed and convex function. Then, \(f\) is continuous over \(\dom f\).
Proof. Since \(f\) is convex, hence its domain is convex. Hence \(\dom f\) must be an interval \(I\).
If \(\interior I = \EmptySet\), then \(I\) must be a singleton.
In that case \(f\) is continuous obviously.
Now consider the case where \(\interior I \neq \EmptySet\).
Then, due to Theorem 9.174, \(f\) is continuous at every \(x \in \interior I\).
If \(I\) is open (i.e., it has no endpoints), then there is nothing more to prove.
We are left with showing the (one sided) continuity of \(f\) at one of the endpoints of \(I\) if it has any.
Since, the argument will be identical for either of the endpoints, without loss of generality, let us assume that \(I\) has a left endpoint \(a\) and we show the continuity from the right at \(a\); i.e. \(\lim_{x \to a^+} f(x) = f(a)\).
Pick any \(c \in I\) such that \(c > a\).
Define a function
\[ g(t) = \frac{f(c -t) - f(c)}{t}. \]Clearly, \(g\) is defined over \((0, c-a]\).
We shall show that \(g\) is nondecreasing and upper bounded over \((0, c-a]\).
Pick any \(t,s\) satisfying \(0 < t \leq s \leq c-a\).
Then,
\[ c - t = \left (1 - \frac{t}{s} \right ) c + \frac{t}{s} (c - s). \]\(\frac{t}{s}\) is well defined and \(\frac{t}{s} \in (0, 1]\).
Thus, \(c-t\) is a convex combination of \(c\) and \(c-s\).
Since \(f\) is convex, hence
\[\begin{split} & f(c - t) \leq \left (1 - \frac{t}{s} \right ) f(c) + \frac{t}{s} f(c - s)\\ & \iff f(c - t) - f(c) \leq \frac{t}{s} (f(c -s) - f(c)) \\ &\iff \frac{f(c - t) - f(c)}{t} \leq \frac{f(c -s) - f(c)}{s}. \end{split}\]Thus,
\[ g(t) \leq g(s) \Forall 0 < t \leq s \leq c-a. \]Thus, \(g\) is nondecreasing over \((0, c-a]\).
Finally \(g(c-a) = \frac{f(a) - f(c)}{c - a}\) is finite since both \(c,a \in \dom f\).
Since \(g\) is nondecreasing, hence
\[ g(t) \leq g(c -a) \Forall t \in (0, c-a]. \]Thus, \(g\) is upper bounded.
Since \(g\) is nondecreasing and upper bounded, hence due to Theorem 2.38, the left hand limit of \(g(t)\) at \(c-a\) exists and is equal to some real number, say,
\[ \lim_{t \to (c-a)^-} g(t) = \ell. \]Note that we haven’t said that \(g\) is continuous from the left at \(c-a\).
Recall from the definition of \(g\) that
\[ f(c-t) = f(c) + t g(t). \]Hence
\[ \lim_{t \to (c-a)^-} f(c -t) = f(c) + (c-a) \ell. \]Replacing \(c-t\) by \(r\), we get
\[ \lim_{r \to a^+} f(r) = f(c) + (c - a) \ell. \]We have shown so far that the limit from the right at \(a\) exists for \(f\) and is equal to \(f(c) + (c - a) \ell\).
Using the upper bound on \(g\), we can say that
\[\begin{split} f(c -t) &= f(c) + t g(t) \\ &\leq f(c) + (c-a) g(c-a) \\ &= f(c) + f(a) - f(c) = f(a) \end{split}\]holds true for every \(t \in (0, c-a]\).
Thus,
\[ \lim_{r \to a^+} f(r) = \lim_{t \to (c-a)^-} f(c -t) \leq f(a). \]On the other hand, since \(f\) is closed, hence it is also lower semicontinuous. This means that
\[ \lim_{r \to a^+} f(r) \geq f(a). \]Combining these two inequalities, we get
\[ \lim_{r \to a^+} f(r) = f(a). \]Thus, \(f\) is indeed continuous from the right at \(a\).
Similarly, if \(I\) has a right endpoint \(b\), then \(f\) is continuous from the left at \(b\).
Thus, \(f\) is continuous at every point in its domain.
9.13.2.2. Local Lipschitz Continuity#
Theorem 9.174 (Local Lipschitz continuity of convex functions)
Let \(\VV\) be an \(n\)-dimensional real normed linear space. Let \(f: \VV \to \RR\) be a convex function with \(S = \dom f\). Let \(\ba \in \interior S\). Then, there exists \(r > 0\) and \(L > 0\) such that \(B(\ba, r) \subseteq S\) and
for every \(\bx \in B[\ba, r]\).
In other words, \(f\) is locally Lipschitz continuous at every interior point of its domain.
We recall from Theorem 9.125 that if \(\dim \affine S < n\) then \(S\) has an empty interior. Thus, if \(\interior S\) is nonempty, then, \(\affine S = \VV\).
Proof. We shall structure the proof as follows. For any \(\ba \in \interior S\):
We show that \(f\) is bounded on a closed ball \(B[\ba, r] \subseteq S\).
Then, we show that \(f\) satisfies the Lipschitz inequality \(|f (\bx) - f(\ba)| \leq L \| \bx - \ba \|\) on the closed ball \(B[\ba, r]\) for a specific choice of \(L\) depending on \(\ba\) and \(r\).
We first introduce \(\| \cdot \|_{\infty}\) norm on \(\VV\) and describe its implications.
Choose a basis \(\BBB = \{\be_1, \dots, \be_n \}\) for \(\VV\).
For every \(\bx \in \VV\), we have a unique representation
\[ \bx = \sum_{i=1}^n x_i \be_i. \]Let \(T : \VV \to \RR^n\) be a coordinate mapping which maps every vector \(\bx \in \VV\) to its coordinate vector \((x_1, \dots, x_n) \in \RR^n\).
\(T\) is an isomorphism.
Define \(\| \cdot \|_{\infty} : \VV \to \RR\) as
\[ \| \bx \|_{\infty} = \| T(\bx) \|_{\infty} = \max_{i=1,\dots,n}|x_i|. \]It is easy to show that \(\| \cdot \|_{\infty}\) is a norm on \(\VV\).
By Theorem 4.60, all norms are equivalent.
Thus, \(\| \cdot \|\) and \(\| \cdot \|_{\infty}\) are equivalent norms for \(\VV\).
By Definition 4.69, the norm topology is identical for all norms in a finite dimensional space.
Thus, a point is an interior point of \(S\) irrespective of the norm chosen.
We introduce the closed and open balls in \((\VV, \| \cdot \|_{\infty})\) as
\[ B_{\infty}[\ba, \delta] = \{\bx \in \VV \ST \| \bx - \ba \|_{\infty} \leq \delta \} \text{ and } B_{\infty}(\ba, \delta) = \{\bx \in \VV \ST \| \bx - \ba \|_{\infty} < \delta \}. \]Let \(\ba \in \interior S\).
Then, there exists \(r_1 > 0\) such that \(B_{\infty}[\ba, r_1] \subseteq S\). due to Remark 3.2.
Then, \(B_{\infty}(\ba, r_1) \subseteq B_{\infty}[\ba, r_1]\).
By Theorem 3.25, there is an \(r_2 > 0\) such that \(B(\ba, r_2) \subseteq B_{\infty}(\ba, r_1)\).
By Theorem 3.2, we can pick an \(0 < r < r_2\) such that
\[ B[\ba, r] \subseteq B(\ba, r_2) \subseteq B_{\infty}(\ba, r_1) \subseteq B_{\infty}[\ba, r_1] \subseteq S. \]
We now show that \(f\) is bounded on \(B[\ba, r]\).
\(B_{\infty}[\ba, r_1]\) is closed and bounded.
Hence \(B_{\infty}[\ba, r_1]\) is compact due to Theorem 4.66.
By Krein Milman theorem, a compact convex set is convex hull of its extreme points. Thus,
\[ B_{\infty}[\ba, r_1] = \convex \extreme B_{\infty}[\ba, r_1]. \]Let \(\bv_1, \dots, \bv_N\) be the \(N=2^n\) extreme points of \(B_{\infty}[\ba, r_1]\).
These extreme points are given by
\[ \bv_i = \ba + r_1 \bw_i \]where \(\bw_i\) are the vectors with coordinates \(\{-1, 1\}^n\).
In other words,
\[ \bw_i = \sum_{j=1}^n w^i_j \be_j \]where \(w^i_j \in \{ -1, 1 \}\).
Note that
\[ \| \bv_i - \ba \|_{\infty} = r \| \bw_i \|_{\infty} = r_1 \max \{ |w^i_j| \} = r_1. \]Thus, \(\bv_i \in \boundary B_{\infty}[\ba, r_1]\).
Readers can verify that these are indeed the extreme points of \(B_{\infty}[\ba, r_1]\) and there are no other extreme points.
Then, by Krein Milman theorem,
\[ B_{\infty}[\ba, r_1] = \convex \{\bv_1, \dots, \bv_N \}. \]Thus, every \(\bx \in B_{\infty}[\ba, r_1]\) is a convex combination of the extreme points. Specially, there exists \(t \in \Delta_N\) (unit simplex of \(\RR^N\)) such that
\[ \bx = \sum_{i=1}^N t_i \bv_i. \]Now, by Jensen's inequality,
\[ f(\bx) \leq \sum_{i=1}^N t_i f(\bv_i). \]Let \(M = \max \{f(\bv_1), \dots, f(\bv_N) \}\).
Then,
\[ f(\bx) \leq \sum_{i=1}^N t_i f(\bv_i) \leq \sum_{i=1}^N t_i M = M \sum_{i=1}^N t_i = M. \]Since \(B[\ba, r] \subseteq B_{\infty}[\ba, r_1]\), hence \(f(\bx) \leq M\) for every \(\bx \in B[\ba, r]\).
We have shown that \(f(\bx) \leq M\) for every \(\bx \in B[\ba, r]\). We next find an \(L\) such that
for every \(\bx \in B[\ba, r]\).
Let \(\bx \in B_d[\ba, r]\) (the deleted neighborhood).
Then, \(\| \bx - \ba \| \leq r\) and \(f(\bx) \leq M\).
Let \(\alpha = \frac{1}{r} \| \bx - \ba \|\). Note that by definition \(\alpha \leq 1\).
Define
\[ \by = \ba + \frac{1}{\alpha}(\bx - \ba). \]Note that
\[ \| \by - \ba \| = r \frac{\| \bx - \ba \| }{\| \bx - \ba \|} = r. \]Thus, \(\by \in B[\ba, r]\).
Hence \(f(\by)\leq M\).
We can rewrite the above equation (defining \(\by\)) as
\[ \bx = \alpha \by + (1- \alpha) \ba. \]Thus, \(\bx\) is a convex combination of \(\by, \ba\).
Then, by convexity,
\[\begin{split} f(\bx) &\leq \alpha f(\by) + (1- \alpha) f(\ba)\\ &= f(\ba) + \alpha (f(\bx) - f(\ba))\\ &\leq f(\ba) + \alpha (M - f(\ba))\\ &= f(\ba) + \frac{1}{r} \| \bx - \ba \| (M - f(\ba)). \end{split}\]Consequently,
\[ f(\bx) - f(\ba) \leq \frac{M - f(\ba)}{r} \| \bx - \ba \|. \]Let \(L = \frac{M - f(\ba)}{r}\).
Then, for every \(\bx \in B[\ba, r]\), we have
\[ f(\bx) - f(\ba) \leq L \| \bx - \ba \|. \]
We next show that for this choice of \(L\)
for every \(\bx \in B[\ba, r]\).
Define
\[ \bz = \ba + \frac{1}{\alpha} (\ba - \bx). \]It is easy to see that \(\| \ba - \bz \| = r\).
Hence, \(\bz \in B[\ba, r]\) and \(f(\bz) \leq M\).
Rearranging, we have
\[ \bx = \ba + \alpha (\ba - \bz). \]Now, note that:
\[ \ba = \frac{1}{1 + \alpha}(\ba + \alpha(\ba - \bz)) + \frac{\alpha}{1 + \alpha} \bz. \]Thus, \(\ba\) is a convex combination of \(\bx = \ba + \alpha (\ba - \bz)\) and \(\bz\).
Also, both \(\bx, \bz \in B[\ba, r]\).
Applying convexity,
\[ f(\ba) \leq \frac{1}{1 + \alpha} f(\bx) + \frac{\alpha}{1 + \alpha} f(\bz). \]Thus,
\[ (1 + \alpha)f(\ba) \leq f(\bx) + \alpha f(\bz). \]Thus,
\[ f(\bx) \geq (1 + \alpha)f(\ba) - \alpha f(\bz) = f(\ba) + \alpha (f(\ba) - f(\bz)). \]Continuing from here
\[\begin{split} f(\bx) &\geq f(\ba) + \alpha (f(\ba) - f(\bz))\\ &\geq f(\ba) - \alpha (M - f(\ba))\\ &= f(\ba) - \frac{M - f(\ba)}{r} \| \bx - \ba \|\\ = f(\ba) - L \| \bx - \ba \|. \end{split}\]Thus, \(f(\ba) - f(\bx) \leq L \| \bx - \ba \|\).
Combining, we see that with \(L = \frac{M - f(\ba)}{r}\),
for every \(\bx \in B[\ba, r]\).
Thus, \(f\) is locally Lipschitz at every interior point of \(S = \dom f\).
9.13.2.3. Continuity of Proper Convex Functions#
Theorem 9.175 (Continuity of proper convex functions)
Let \(\VV\) be an \(n\)-dim real normed linear space. Let \(f: \VV \to \RR\) be a real valued convex function. Then it is continuous. More generally, if \(f : \VV \to \RERL\) is a proper convex function, then \(f\), restricted to \(\dom f\), is continuous over the relative interior of \(\dom f\).
Proof. We will restrict our attention to the affine hull of the domain of \(f\). We shall assume that origin belongs to \(\relint \dom f\). We shall use a transformation argument as needed.
Let \(S = \dom f\).
Let \(A = \affine S\).
Let \(L\) the linear subspace parallel to \(A\).
Let \(\ba \in \relint S\). If \(\bzero \in \relint S\), we can pick \(\ba = \bzero\).
Define the function
\[ g(\bx) = f(\bx + \ba). \]Clearly \(g(\bzero) = f(\ba)\).
We can see that \(\dom g = S - \ba\) and \(\affine \dom g = L\).
Consider the restriction of \(g\) to \(L\) defined as \(h : L \to \RERL\) given by \(h(\bx) = g(\bx)\).
\(\dom h = S - \ba\).
We note that relative interior of \(S - \ba\) w.r.t. \(\VV\) is the same as the interior of \(S - \ba\) w.r.t. \(L\).
By Theorem 9.174, \(h\) is locally Lipschitz continuous at every interior point of \(S - \ba\) (relative to \(L\)).
Hence \(g\) is continuous over the relative interior of \(S - \ba\).
Hence \(f\) is continuous over the relative interior of \(S\).
Corollary 9.12 (Closedness of real valued convex functions)
Let \(\VV\) be an \(n\)-dim real normed linear space. A real valued convex function \(f: \VV \to \RR\) with \(\VV = \dom f\) is closed.
Proof. By Theorem 9.175, \(f\) is continuous over \(\VV\).
For real valued convex functions, \(\VV = \dom f\).
\(\VV\) is a closed set in the topology of \((\VV, \| \cdot \|)\).
By Theorem 3.96, continuity and closed domain imply that \(f\) is closed.