2.7. Differentiable Functions

We continue our discussion on real functions and focus on a special class of functions which are differentiable.

Definition 2.78 (Differentiable function)

A real function \(f: \RR \to \RR\) is differentiable at an interior point \(x=a\) of its domain \(\dom f\) if the difference quotient

\[ \frac{f(x) - f(a)}{x - a}, \quad x \neq a \]

approaches a limit as \(x\) approaches \(a\).

If \(f\) is differentiable at \(x=a\), the limit is called the derivative of \(f\) at \(x=a\) and is denoted by \(f'(a)\); thus,

\[ f'(a) = \lim_{x \to a}\frac{f(x) - f(a)}{x - a}. \]

An alternative way is to write \(x\) as \(x = a + h\) and define \(f'(a)\) as:

\[ f'(a) = \lim_{h \to 0}\frac{f(a + h) - f(a)}{h}. \]

Notes

• The difference quotient is not defined at \(x=a\). This is okay as computing the limit \(\lim_{x \to a} g(x)\) doesn’t require \(g\) to be defined at \(x=a\).

• The derivative is not defined for the non-interior points of \(\dom f\). Only one sided limits may be computed at the non-interior points on the difference quotient.

• We can treat \(f'\) as a function from \(\RR\) to \(\RR\) where \(f'\) is defined only on points at which \(f\) is differentiable.

• The type signature for \(f'\) is \(f' : \RR \to \RR\).

• The domain of \(f'\) denoted by \(\dom f'\) is the set of points at which \(f\) is differentiable.

Remark 2.17 (Domain of the derivative function)

The domain of the derivative of a function \(f\); i.e., the set of points at which the derivative exists (or is defined) is a subset of the interior of the domain of the function \(f\) itself.

\[ \dom f' \subseteq \interior \dom f. \]

Definition 2.79 (Differentiable function)

Let \(f\) be defined on an open set \(A\). We say that \(f\) is differentiable on \(A\) if \(f\) is differentiable at every point in \(A\).

If \(f\) is differentiable on (open) \(A\), then \(f'\) is defined on \(A\). In other words: \(\dom f' = A\).

Definition 2.80 (Continuously differentiable function)

We say that \(f\) is continuously differentiable on an open set \(A\) if \(f\) is differentiable on \(A\) and \(f'\) is continuous on \(A\).

Definition 2.81 (Second and \(n\)-th derivatives)

If \(f\) is differentiable on a neighborhood of \(x=a\) and \(f'\) is differentiable at \(x=a\), we denote the derivative of \(f'\) at \(x=a\) by \(f''(a)\) and call it the second derivative of \(f\) at \(x=a\). Another notation for the second derivative is \(f^{(2)}(a)\).

Inductively, if \(f^{(n-1)}\) is defined on a neighborhood of \(x=a\) and \(f^{(n-1)}\) is differentiable at \(x=a\), then the n-th derivative of \(f\) at \(x=a\), denoted by \(f^{(n)}(a)\), is the derivative of \(f^{(n-1)}\) at \(x=a\).

The zeroth derivative of \(f\) is defined to be \(f\) itself.

\[ f^{(0)} = f. \]

Another common notation is:

\[ \frac{d f}{d x} = f' \text{ and } \frac{d^n f}{d x^n} = f^{(n)}. \]

Definition 2.82 (Tangent line)

If \(f\) is differentiable at \(x=a\), then the tangent to \(f\) at \(x=a\) can be given by:

\[ T(x) = f(a) + f'(a)(x -a). \]

It is useful to remove the contribution of the tangent in \(f\) and study the remaining part of \(f\).

Lemma 2.1 (Removal of tangent line from function)

If \(f\) is differentiable at \(x=a\), then we can write \(f\) as:

(2.1)\[f(x) = f(a) + [f'(a) + E(x)](x -a)\]

where \(E\) is defined in the neighborhood of \(x=a\) and

\[ \lim_{x \to a} E(x) = E(a) = 0. \]

In other words, \(E\) is continuous at \(x=a\).

Proof. We define \(E\) as:

\[\begin{split} E(x) = \begin{cases} \frac{f(x) - f(a)}{x - a} - f'(a), & x \in \dom f \text{ and } x \neq a\\ 0, & x = a \end{cases}. \end{split}\]

This \(E\) meets the requirements of (2.1). Note that \(E\) is continuous at \(x=a\) as \(\lim_{x \to a} E(x) = E(a)= 0\).

We note that:

\[ f(x) - T(x) = E(x) (x - a). \]

Alternatively

\[ f(x) = T(x) + E(x) (x - a). \]

Remark 2.18 (Difference quotient and derivative)

At \(x \neq a\), (2.1) can also be written as:

\[ \frac{f(x) - f(a)}{x -a} = f'(a) + E(x). \]

In other words, the difference quotient \(\frac{f(x) - f(a)}{x -a}\) is the sum of the derivative \(f'(a)\) and \(E(x)\).

Theorem 2.48 (Differentiability implies continuity)

If \(f\) is differentiable at \(x=a\), then \(f\) is continuous at \(x=a\).

Proof. Using Definition 2.82 and Lemma 2.1, we have:

\[ f(x) = T(x) + E(x) (x - a). \]

It is easy to see that \(E(x)\) and \(T(x)\) are both continuous at \(x=a\). Thus, \(f\) is continuous at \(x=a\).

Notes:

1. If \(f\) is not continuous at \(x=a\) then \(f\) is not differentiable at \(x=a\).

2. Continuity is a necessary condition but not sufficient condition for differentiability.

Remark 2.19 (Derivative sign and monotonicity in the neighborhood)

If \(f\) is differentiable at \(x=a\), and \(f'(a) \neq 0\), then there is \(\delta > 0\) such that if \(f'(a) > 0\), then

\[ \sgn (f(x) - f(a)) = \sgn (x - a) \Forall |x - a| < \delta \]

and if \(f'(a) < 0\), then

\[ \sgn (f(x) - f(a)) = \sgn (a - x) \Forall |x - a| < \delta. \]

Proof. We have, from Lemma 2.1, for \(x \neq a\):

\[ \frac{f(x) - f(a)}{x -a} = f'(a) + E(x). \]

Assume \(f'(a) \neq 0\). Then \(|f'(a)| > 0\). Since \(E\) is continuous at \(a\), with \(\epsilon = |f'(a)| > 0\), there exists \(\delta > 0\) such that

\[ |E(x) - E(a)| = |E(x)| < \epsilon = |f'(a)| \text{ whenever } |x - a| < \delta. \]

Thus,

\[ \sgn(f'(a) + E(x)) = \sgn (f'(a)) \Forall |x - a| < \delta. \]

Thus,

\[ \sgn \left ( \frac{f(x) - f(a)}{x -a} \right ) = \sgn (f'(a)) \Forall |x - a| < \delta. \]

Now, if \(f'(a) > 0\), then

\[ \sgn (f(x) - f(a)) = \sgn (x - a) \Forall |x - a| < \delta. \]

If \(f'(a) < 0\), then

\[ \sgn (f(x) - f(a)) = - \sgn (x - a) = \sgn (a - x) \Forall |x - a| < \delta. \]

2.7.1. Arithmetic

Theorem 2.49 (Differentiation and arithmetic )

If \(f\) and \(g\) are differentiable at \(x=a\), then so are \(f+g\), \(f-g\), \(fg\). \(\frac{f}{g}\) is differentiable at \(x=a\) if \(g'(a) \neq 0\). The derivatives are:

1. \((f + g)'(a) = f'(a) + g'(a)\).

2. \((f - g)'(a) = f'(a) - g'(a)\).

3. \((f g)'(a) = f'(a) g(a) + f(a) g'(a)\).

4. \(\left(\frac{f}{g} \right)'(a) = \frac{f'(a) g(a) - f(a) g'(a)}{[g'(a)]^2}\) provided \(g'(a) \neq 0\).

2.7.2. The Chain Rule

Theorem 2.50 (The chain rule)

Let \(f\) be differentiable at \(x=a\). Assume that \(g\) is differentiable at \(f(a)\). Then the composite function given by \(h = g \circ f\) is differentiable at \(f(a)\) with

\[ h'(a) = g'(f(a))f'(a). \]

Proof. Let \(b = f(a)\). Since \(g\) is differentiable at \(b\), we can write \(g\) as (Lemma 2.1):

\[ g(t) = g(b) + [g'(b) + E(t)] (t - b) \]

where \(E\) is continuous in the neighborhood of \(t=b\) and \(\lim_{t \to b} E(t) = E(b) = 0\). Thus,

\[ g(t) - g(b) = [g'(b) + E(t)] (t - b). \]

Putting \(t=f(x)\), we get:

\[ g(f(x)) - g(f(a)) = [g'(f(a)) + E(f(x))][f(x) - f(a)]. \]

Since \(h(x) = g(f(x))\), we get:

\[ h(x) - h(a) = [g'(f(a)) + E(f(x))][f(x) - f(a)]. \]

Dividing both sides by \((x-a)\), we get:

\[ \frac{h(x) - h(a)}{x - a} = [g'(f(a)) + E(f(x))] \frac{f(x) - f(a)}{x-a}. \]

Since \(f\) is continuous at \(x=a\), \(E\) is continuous at \(t=b=f(a)\), and \(b\) is an interior point of \(\dom E\), hence \(E\circ f\) is continuous at \(x=a\) due to Theorem 2.41. Thus,

\[ \lim_{x \to a} E(f(x)) = E(f(a)) = E(b) = 0. \]

Therefore,

\[\begin{split} \begin{aligned} h'(a) &= \lim_{x \to a} \frac{h(x) - h(a)}{x - a} \\ &= \lim_{x \to a} [g'(f(a)) + E(f(x))] \lim_{x \to a} \frac{f(x) - f(a)}{x-a}\\ &= [g'(f(a)) + \lim_{x \to a}E(f(x)) ] f'(a) = g'(f(a)) f'(a). \end{aligned} \end{split}\]

Example 2.12

Let

\[ f(x) = \frac{1}{x} \text{ and } g(x) = \sin x. \]

Then, the composition \(h = g \circ f\) is given by

\[ h(x) = (g \circ f)(x) = g (f (x)) = \sin \frac{1}{x}. \]

We have:

\[ f'(x) = -\frac{1}{x^2} \text{ and } g'(x) = \cos x. \]

Thus,

\[ h'(x) = g'(f(x))f'(x) = \left (\cos \frac{1}{x} \right ) \left (-\frac{1}{x^2} \right ). \]

2.7.3. One Sided Derivatives

Definition 2.83 (One sided derivatives)

One sided limits of the difference quotient

\[ \frac{f(x) - f(a)}{x - a}, \quad x \neq a \]

are called one-sided derivatives if they exist.

If \(f\) is defined over \([a,b)\), then the right hand derivative is defined as:

\[ f'_+(a) \triangleq \lim_{x \to a^+} \frac{f(x) - f(a)}{x - a} \]

if the limit exists.

If \(f\) is defined over \((c,a]\), then the left hand derivative is defined as:

\[ f'_-(a) \triangleq \lim_{x \to a^-} \frac{f(x) - f(a)}{x - a} \]

if the limit exists.

Remark 2.20 (Differentiability and one-sided derivatives)

A function \(f\) is differentiable at \(x=a\) if and only if its left and right hand derivatives exist and are equal. In that case:

\[ f'_-(a) = f'(a) = f'_+(a). \]

This is a direct implication of Theorem 2.37.

Remark 2.21

One sided derivative is not the same thing as one sided limit of a derivative.

1. \(f'_+(a)\) need not be equal to \(f'(a^+)\).

2. \(f'_-(a)\) need not be equal to \(f'(a^-)\).

2.7.4. Closed Intervals

Definition 2.84 (Differentiability on a closed interval)

We say that \(f\) is differentiable on the closed interval \([a,b]\) if \(f\) is differentiable on the open interval \((a,b)\) and the one sided derivatives \(f'_+(a)\) and \(f'_-(b)\) both exist.

We assign \(f'(a) = f'_+(a)\) and \(f'(b) = f'_-(b)\) to complete the definition of \(f'\) over \([a,b]\).

Note

While it is possible to use the notion of one sided derivatives to define \(f'\) on a closed interval, this notion doesn’t generalize to multivariable calculus. The definition of derivative on an open interval (or an open subset of \(\dom f\)) can be easily extended to multivariable calculus.

Definition 2.85 (Continuous differentiability on a closed interval)

We say that \(f\) is continuously differentiable on the closed interval \([a,b]\) if

1. \(f\) is differentiable on the closed interval \([a,b]\)

2. \(f'\) is continuous over the open interval \((a,b)\)

3. \(f'_+(a) = f'(a^+)\).

4. \(f'_-(b) = f'(b^-)\).

2.7.5. Extreme Values

Definition 2.86 (Local extreme value)

We say that \(f(a)\) is a local extreme value of \(f\) if there exists \(\delta > 0\) such that \(f(x) - f(a)\) doesn’t change sign on

\[ (a - \delta, a + \delta) \cap \dom f. \]

More specifically,

1. \(f(a)\) is a local maximum value of \(f\) if for some \(\delta > 0\):

   \[ f(x) \geq f(a) \Forall x \in (a - \delta, a + \delta) \cap \dom f. \]

2. \(f(a)\) is a local minimum value of \(f\) if for some \(\delta > 0\):

   \[ f(x) \geq f(a) \Forall x \in (a - \delta, a + \delta) \cap \dom f. \]

The point \(x=a\) is called a local extreme point of \(f\) or more specifically, a local maximum or a local minimum point of \(f\).

Theorem 2.51

If \(f\) is differentiable at a local extreme point \(a \in \dom f\), then \(f'(a) = 0\).

In other words, if the derivative exists at a local extreme point, it vanishes there.

Proof. We show that if \(f'(a) \neq 0\) then \(a\) is not a local extreme point. Thus, if \(a\) is a local extreme point then \(f'(a)\) must be 0.

Assume \(f'(a) \neq 0\).

From Lemma 2.1, we have (at \(x \neq a\)):

\[ \frac{f(x) - f(a)}{x -a} = f'(a) + E(x) \]

where \(\lim_{x \to a} E(x) = 0\) and \(E\) is continuous at \(x=a\).

Since \(f'(a) \neq 0\), hence \(|f'(a)| > 0\), hence there exists \(\delta > 0\) such that

\[ |E(x) | < |f'(a)| \text{ whenever } |x - a | < \delta. \]

Thus, in the interval \(|x - a | < \delta\), the term \(f'(a) + E(x)\) has the same sign as \(f'(a)\).

Hence the term \(\frac{f(x) - f(a)}{x -a}\) must not change sign in \(|x - a | < \delta\).

But the term \((x -a)\) changes sign in \(|x - a | < \delta\). Hence, \(f(x) - f(a)\) must also change sign.

Moreover, \((x -a)\) changes sign in every neighborhood \(|x - a | < \delta_1\) with \(\delta_1 > 0\). Hence \(f(x) - f(a)\) must also change sign in every neighborhood.

Hence there is no neighborhood of \(a\) in which \(f(x) - f(a)\) doesn’t change sign. Hence, \(a\) is not a local extreme point of \(f\).

Definition 2.87 (Critical point)

Let \(f : \RR \to \RR\) be a real function. Let \(a \in \interior \dom f\). If \(f\) is not differentiable at \(a\) or if \(f\) is differentiable at \(x=a\) and \(f'(a) = 0\), then we say that \(a\) is a critical point of \(f\).

Definition 2.88 (Stationary point)

If \(f\) is differentiable at \(x=a\) and \(f'(a) = 0\), then we say that \(a\) is a stationary point of \(f\).

All stationary points are critical points while all critical points need not be stationary points. If the derivative doesn’t exist at some point \(a \in \interior \dom f\), it could indicate a potential maximum or minimum.

Remark 2.22

All local extreme points are critical points.

Example 2.13 (A non-extreme critical point)

A critical point need not be a local extreme point. For the function \(f(x) = x^3\), \(f'(0) = 0\). Thus, \(x=0\) is a critical point. But it is not a local extreme point since \(f\) changes sign around \(x=0\).

Theorem 2.52 (Rolle’s theorem)

Let \(f\) be continuous on the closed interval \([a,b]\). Assume \(f\) to be differentiable on the open interval \((a,b)\). Further assume that \(f(a) = f(b)\). Then, \(f'(c) = 0\) for some \(c \in (a,b)\).

Proof. Recall that if \(f\) is continuous on a closed interval, then \(f\) attains its maximum and minimum value on points in the interval (Theorem 2.43).

Assume

\[ \alpha = \inf_{a \leq x \leq b} f(x) \text{ and } \beta = \sup_{a \leq x \leq b} f(x). \]

If \(\alpha=\beta\), then \(f\) is a constant function on \([a,b]\). In that case, \(f'(c) = 0\) for all \(c \in (a,b)\).

Consider the case where \(\alpha < \beta\). In that case, either the maximum or the minimum is attained at some point \(c \in (a,b)\) since \(f(a) = f(b)\).

1. If \(\alpha = f(a) = f(b)\), then \(\beta\) must be attained at some point in \(c \in (a,b)\).

2. If \(\beta = f(a) = f(b)\), then \(\alpha\) must be attained at some point in \(c \in (a,b)\).

3. If neither of the above hold true, then both \(\alpha\) and \(\beta\) are attained at some point in \((a,b)\).

Since \(f\) is differentiable at \(c\) and \(f(c)\) is either maximum or minimum (i.e. a local extremum), hence \(f'(c) = 0\) due to Theorem 2.51.

2.7.6. Intermediate Values

Theorem 2.53 (Intermediate value theorem for derivatives)

Suppose that:

1. \(f\) is differentiable on an open interval \(I\).

2. There is a closed interval \([a,b] \subset I\).

3. \(f'(a) \neq f'(b)\).

4. \(\mu\) is in between \(f'(a)\) and \(f'(b)\).

Then \(f'(c) = \mu\) for some \(c \in (a,b)\)

Note that this result doesn’t require \(f\) to be continuously differentiable (i.e. \(f'\) to be continuous).

Proof. Since \(f\) is differentiable on \(I\), hence \(f\) is continuous on \(I\).

Assume without loss of generality:

\[ f'(a) < \mu < f'(b). \]

Define

\[ g(x) = f(x) - \mu x. \]

Then,

\[ g'(x) = f'(x) - \mu, \Forall a \leq x \leq b. \]

Since \(f'(a) < \mu\) hence \(g'(a) < 0\). Similarly, \(g'(b) > 0\).

Since \(g\) is continuous on \([a,b]\), hence \(g\) attains a minimum at some point \(c \in [a,b]\) (Theorem 2.43).

Now, \(g'(a) < 0\) implies that there exists \(\delta > 0\) such that:

\[ g(x) < g(a) \Forall a < x < a + \delta. \]

Similarly,

\[ g(x) < g(b) \Forall b - \delta < x < b. \]

Thus, minimum of \(g\) cannot be at \(a\) or \(b\). Hence, \(c \in (a, b)\). Since \(c\) is a local extreme point, and \(g\) is differentiable at \(c\), Hence \(g'(c) = 0\) due to Theorem 2.51. This in turn implies that \(f'(c) = \mu\).

The case of \(f'(a) > \mu > f'(b)\) can be handled by applying the same argument to \(-f\).

2.7.7. Mean Values

Theorem 2.54 (Generalized mean value theorem)

If \(f\) and \(g\) are continuous on the closed interval \([a,b]\) and differentiable on the open interval \((a,b)\), then

\[ [g(b) - g(a)] f'(c) = [f(b) - f(a)]g'(c) \]

holds true for some \(c \in (a,b)\).

Proof. Define the function:

\[ h(x) = [g(b) - g(a)] f(x) - [f(b) - f(a)]g(x). \]

1. Since \(f\) and \(g\) are continuous on \([a,b]\), so is \(h\).

2. Since \(f\) and \(g\) are differentiable on \((a,b)\) so is \(h\).

3. \(h(a) = h(b) = g(b)f(a) - f(b)g(a)\).

4. Therefore, by Rolle's theorem, \(h'(c) = 0\) for some \(c \in (a,b)\).

5. But \(h'(c) = [g(b) - g(a)] f'(c) - [f(b) - f(a)]g'(c)\).

6. Hence the result.

Theorem 2.55 (Mean value theorem)

If \(f\) is continuous on the closed interval \([a,b]\) and differentiable on the open interval \((a,b)\), then

\[ f'(c) = \frac{f(b) - f(a)}{b -a} \]

for some \(c \in (a,b)\).

Proof. In Theorem 2.54, let \(g(x) = x\).

Then,

\[ (b - a) f'(c) = f(b) - f(a). \]

Remark 2.23

Assume \(f\) to be differentiable on some open interval \((a,b)\). Assume \(x_1, x_2 \in (a,b)\). We haven’t specified whether \(x_1 < x_2\) or \(x_1 > x_2\).

1. \(f\) is continuous on the closed interval with endpoints \(x_1\) and \(x_2\).

2. \(f\) is differentiable on the interior of this closed interval.

3. By mean value theorem:

   (2.2)\[f(x_2) - f(x_1) = f'(c) (x_2 - x_1)\]

   for some \(c\) in the open interval between \(x_1\) and \(x_2\).

Theorem 2.56

If \(f'(x)=0\) for all \(x \in (a,b)\), then \(f\) is constant on \((a,b)\).

Proof. For any \(x_1, x_2 \in (a,b)\), using (2.2), we get:

\[ f(x_2) - f(x_1) = 0. \]

Theorem 2.57 (No change in derivative sign implies monotonicity)

If \(f'\) exists and does not change sign on \((a,b)\), then \(f\) is monotonic on \((a,b)\). In particular:

1. If \(f'(x) > 0\), then \(f\) is strictly increasing in \((a,b)\).

2. If \(f'(x) \geq 0\), then \(f\) is increasing in \((a,b)\).

3. If \(f'(x) \leq 0\), then \(f\) is decreasing in \((a,b)\).

4. If \(f'(x) < 0\), then \(f\) is strictly decreasing in \((a,b)\).

Proof. Let \(x_1, x_2 \in (a,b)\) be such that \(x_1 < x_2\). By mean value theorem, there exists \(c \in (x_1,x_2)\) such that:

\[ f(x_2) - f(x_1) = f'(c) (x_2 - x_1). \]

Now,

1. If \(f'(x) > 0 \Forall x \in (a,b)\), then \(f(x_2) - f(x_1) > 0\).

2. If \(f'(x) \geq 0 \Forall x \in (a,b)\), then \(f(x_2) - f(x_1) \geq 0\).

3. If \(f'(x) \leq 0 \Forall x \in (a,b)\), then \(f(x_2) - f(x_1) \leq 0\).

4. If \(f'(x) < 0 \Forall x \in (a,b)\), then \(f(x_2) - f(x_1) < 0\).

Theorem 2.58 (Bounded derivative implies Lipschitz continuity)

If

\[ |f'(x)| \leq M, \Forall a < x < b, \]

then:

\[ |f(x) - f(x') | \leq M | x - x'|, \Forall x, x' \in (a,b). \]

This is another direct implication of the mean value theorem.