Metric Spaces

The main concepts of real analysis on R can be carried over to a general set M once a notion of distance d(x,y) has been defined for points x,y∈M. When M=R, the distance we have been using all along is d(x,y)=|x−y|. The set R along with the distance function d(x,y)=|x−y| is an example of a metric space.
Let M be a non-empty set. A metric on M is a function d:M×M→[0,∞) satisfying the following properties:
  1. d(x,y)=0 if and only if x=y
  2. d(x,y)=d(y,x) for all x,y∈M (symmetry)
  3. d(x,y)≤d(x,z)+d(z,y) for all x,y,z∈M (triangle inequality)
A metric space is a pair (M,d) where d is a metric on M.
If the metric d is understood, then we simply refer to M as a metric space instead of formally referring to the pair (M,d).
The set M=R and function d(x,y)=|x−y| is a metric space. To see this, first of all |x−y|=0 iff x−y=0 iff x=y. Second of all, |x−y|=|−(y−x)|=|y−x|, and finally by the usual triangle inequality on R we have d(x,y)=|x−y|=|x−z+y−z|≤|x−z|+|z−y|=d(x,z)+d(z,y) for all x,y,z∈R.
Let B([a,b]) denote the set of bounded functions on the interval [a,b], that is, f∈B([a,b]) if there exists M>0 such that |f(x)|≤M for all x∈[a,b]. For f,g∈B([a,b]) let d(f,g)=supx∈[a,b]|f(x)−g(x)|. We claim that (B([a,b]),d) is a metric space. First of all, if f,g∈B([a,b]) then using the triangle inequality it follows that (f−g)∈B([a,b]). Therefore, d(f,g) is well-defined for all f,g∈B([a,b]). Next, by definition, we have that 0≤d(f,g) and it is clear that d(f,g)=d(g,f). Lastly, for f,g,h∈B([a,b]) since |f(x)−g(x)|≤|f(x)−h(x)|+|h(x)−g(x)| then d(f,g)=supx∈[a,b]|f(x)−g(x)|≤supx∈[a,b](|f(x)−h(x)|+|h(x)−g(x)|)≤supx∈[a,b](|f(x)−h(x)|)+supx∈[a,b](|h(x)−g(x)|)=d(f,h)+d(h,g). This proves that (B[a,b],d) is a metric space. It is convention to denote the metric d(f,g) as d∞(f,g), and we will follow this convention.
Let M be a non-empty set. Define d(x,y)=1 if x≠y and d(x,y)=0 if x=y. It is straightforward to show that (M,d) is a metric space. The metric d is called the discrete metric and (M,d) is a discrete space.
Let (M,d) be a metric space and let M′⊂M be a non-empty subset. Let d′ be the restriction of d onto M′, that is, d′:M′×M′→[0,∞) is defined as d′(x,y)=d(x,y) for x,y∈M′. Then (M′,d′) is a metric space. We may therefore say that M′ is a metric subspace of M.
Let C([a,b]) denote the set of continuous functions on the inteval [a,b]. Then C([a,b])⊂B([a,b]) and thus (C([a,b]),d∞) is a metric subspace of (B([a,b]),d∞).
For f,g∈C([a,b]) let d(f,g)=∫ab|f(t)−g(t)|dt. Prove that d defines a metric on C([a,b]).
Let denote the set of matrices with real entries. For define It is clear that and if and only if . For we have that Hence, is a metric space.
An important class of metric spaces are normed vector spaces.
Let be a vector space over (or ). A norm on is a function satisfying the following properties:
  1. if and only if ,
  2. for any scalar and any , and
  3. for all .
The number is called the norm of . A vector space together with a norm is called a normed vector space.
Instead of using the generic letter to denote a norm, it is convention to use instead . Hence, using to denote the norm , properties (i)-(iii) are:
  1. if and only if ,
  2. for any scalar and any , and
  3. for all .
Let be a normed vector space and define by It is a straightforward exercise (which you should do) to show that is a metric space. Hence, every normed vector space induces a metric space.
The real numbers form a vector space over under the usual operations of addition and multiplication. The absolute value function is a norm on . The induced metric is then .
The Euclidean norm on is defined as for . It can be verified that is indeed a norm on . Hence, we define the distance between as Notice that when , is the absolute value function since for . When not specified otherwise, whenever we refer to as a normed vector space we implicitly assume that the norm is and simply use the notation .
The set of bounded functions forms a vector space over with addition defined as for and scalar multiplication defined as for and . For let It is left as an (important) exercise to show that is indeed a norm on . The induced metric is The norm is called the sup-norm of . Notice that the metric in Example 9.1.3 is induced by the norm .
Two examples of norms on are and These norms are important in the analysis of Fourier series.
For let It is left as an exercise to show that defined above is a norm on .
Let be a metric space. For and , the open ball centered at of radius is by definition the set
Interpret geometrically the open balls in the normed spaces for .
Give a graphical/geometric description of the open balls in the normed space .
A subset of a metric space is called bounded if for some and .
Let be a metric space. Prove that if is bounded then there exists and such that .

Exercises

Let be the set of all real sequences such that for all . For let Prove that is a metric on . Note: Part of what you have to show is that is well-defined which means to show that converges if .

Sequences and Limits

Let be a metric space. A sequence in is a function . As with sequences of real numbers, we identify a sequence with the infinite list where for .
Let be a metric space. A sequence in is said to converge if there exists such that for any given there exists such that for all . In this case, we write or , and we call the limit of . If does not converge then we say it is divergent.
One can indeed show, just as in Theorem 3.1.12 for sequences of real numbers, that the point in Definition 9.2.1 is indeed unique.
Suppose that converges to and let . Hence, is a sequence of real numbers. If then for any there exists such that . Thus, . Conversely, if then clearly .
Prove that a sequence converges to in the normed vector space if and only if converges uniformly to on .
Several of the results for sequences of real numbers carry over to sequences on a general metric space. For example, a sequence in is said to be bounded if the set is bounded in . Then:
In a metric space, a convergent sequence is bounded.
Suppose that is the limit of . There exists such that for all . Let and we note that . Then . To see this, if then and thus . On the other hand, for we have that , and thus . This proves that is bounded.
A subsequence of a sequence is a sequence of the form where . Then (compare with Theorem 3.4.5):
Let be a metric space and let be a sequence in . If then for any subsequence of .
A sequence in is called a Cauchy sequence if for any given there exists such that for all . Then (compare with Lemma 3.6.3- 3.6.4):
Let be a metric space and let be a sequence in . The following hold:
  1. If is convergent then is a Cauchy sequence.
  2. If is a Cauchy sequence then is bounded.
  3. If is a Cauchy sequence and if has a convergent subsequence then converges.
Proofs for (i) and (ii) are left as exercises (see Lemma 3.6.3- 3.6.4). To prove (iii), let be a convergent subsequence of , say converging to . Let be arbitrary. There exists such that for all . By convergence of to , by increasing if necessary we also have that for all . Therefore, if , then since then Hence, .
The previous lemmas (that were applicable on a general metric space) show that some properties of sequences in are due entirely to the metric space structure of . There are, however, important results on , most notably the Bolzano-Weierstrass theorem and the Cauchy criterion for convergence, that do not generally carry over to a general metric space. The Bolzano-Weierstrass theorem and the Cauchy criterion rely on the completeness property of and there is no reason to believe that a general metric space comes equipped with a similar completeness property. Besides, the completeness axiom of (Axiom 2.4.6) relies on the order property of (i.e., ) and there is no reason to believe that a general metric space comes equipped with an order. We will have more to say about this in Section 9.4. For now, however, we will consider an important metric space where almost all the results for sequences in carry over (an example of a result not carrying over is the Monotone convergence theorem), namely, the normed vector space . Denoting a sequence in is notationally cumbersome. Formally, a sequence in is a function . How then should we denote as a vector in ? One way is to simply write for each and this is the notation we will adopt. It is clear that a sequence in induces sequences in , namely, for each (i.e., the component sequences). The following theorem explains why inherits almost all the results for sequences in .
Let be a sequence in the normed vector space . Then converges if and only if for each the component sequence converges. Moreover, if converges then
Suppose first that converges, say to . For any it holds that in other words, . Since then and consequently , that is, . Conversely, now suppose that converges for each . Let for each and let . By the basic limit laws of sequences in , the sequence converges to zero since and the square root function is continuous. Thus, as desired.
Every Cauchy sequence in is convergent.
Let be a Cauchy sequence in . Hence, for there exists such that for all . Thus, for any , if then Thus, is a Cauchy sequence in , and is therefore convergent by the completeness property of . By Theorem 9.2.7, this proves that is convergent.
Every bounded sequence in has a convergent subsequence.
Let be a bounded sequence in . There exists and such that for all , that is, for all . Therefore, for any we have This proves that is a bounded sequence in for each . We now proceed by induction. If then is just a (bounded) sequence in and therefore, by the Bolzano-Weierstrass theorem on , has a convergent subsequence. Assume by induction that for some , every bounded sequence in has a convergent subsequence. Let be a bounded sequence in . Let be the sequence in such that is the vector of the first components of . Then is a bounded sequence in (why>. By induction, has a convergent subsequence, say it is . Now, the real sequence is bounded and therefore by the Bolzano-Weierstrass theorem on , has a convergent subsequence which we denote by , that is, . Now, since is a subsequence of the convergent sequence , converges in . Thus, each component of the sequence in is convergent and since is a subsequence of the sequence ) the proof is complete.
Let be a metric space.
  1. A subset of is said to be open if for any there exists such that .
  2. A subset of is closed if is open.
Prove that an open ball is open. In other words, prove that for each there exists such that .
Below are some facts that are easily proved; once (a) and (b) are proved use DeMorgan's Laws to prove (c) and (d).
  1. If is a finite collection of open sets then is open.
  2. If is collection of open sets indexed by a set then is open.
  3. If is a finite collection of closed sets then is closed.
  4. If is collection of closed sets indexed by a set then is closed.
Below is a characterization of closed sets via sequences.
Let be a metric space and let . Then is closed if and only if every sequence in that converges does so to a point in , that is, if and then .
Suppose that is closed and let be a sequence in . If then there exists such that . Hence, for all and thus does not converge to . Hence, if converges then it converges to a point in . Conversely, assume that every sequence in that converges does so to a point in and let be arbitrary. Then by assumption, is not the limit point of any converging sequence in . Hence, there exists such that otherwise we can construct a sequence in converging to (how>. This proves that is open and thus is closed.
Show that is a closed subset of .
Let be an arbitrary non-empty set and let be the discrete metric, that is, if and if . Describe the converging sequences in . Prove that every subset of is both open and closed.
For , prove that is closed.

Exercises

Let be a metric space and suppose that converges in . Prove that the limit of is unique. In other words, prove that if and satisfy the convergence definition for then .
Let be a metric space.
  1. Let be fixed. Prove that if converges to then .
  2. Prove that if converges to and converges to then .
Let be a metric space. Prove that if are open then is open in .
Let be a metric space and let . A point is called a limit point (or cluster point) of if there exists a sequence in , with for all , converging to . The closure of , denoted by , is the union of and the limit points of . If then we say that is dense in . As an example, is dense in since every irrational number is the limit of a sequence of rational numbers.
  1. Prove that is dense in if and only if for every open set of .
  2. Let be the set of step functions on . Then clearly . Prove that the set of continuous function is contained in the closure of . (Hint: See Example 8.2.6)
  3. Perform an internet search and find dense subsets of (you do not need to supply proofs).
For define and . It is not hard to verify that these are norms on . Prove that:
  1. ,
  2. , and
Two metrics and on a set are equivalent if they generate the same convergent sequences, in other words, converges in if and only if converges in . Prove that are equivalent norms on .
How do we (Riemann) integrate functions from to ? Here is how. First, we equip with the standard Euclidean norm . For any function and any tagged partition of , define the Riemann sum We then say that is Riemann integrable if there exists such that for any there exists such that for any tagged partition of with it holds that We then also write that . If has component functions , prove that is Riemann integrable on if and only if the component functions are Riemann integrable on , and in this case, Hint: Recall that for any it holds that .
Let denote the set of infinite sequences in . It is not hard to see that is a -vector space with addition and scalar multiplication defined in the obvious way. Let denote the subset of sequences such that converges, that is, denotes the set of absolutely convergent series.
  1. Prove that is a subspace of , i.e., prove that is closed under addition and scalar multiplication.
  2. For let . Prove that defines a norm on .

Continuity

Using the definition of continuity for a function as a guide, it is a straightforward task to formulate a definition of continuity for a function where and are metric spaces.
Let and be metric spaces. A function is continuous at if given any there exists such that if then . We say that is continuous if it is continuous at each point of .
Using open balls, is continuous at if for any given there exists such that whenever . We note that is an open ball in while is an open ball in . Below we characterize continuity using sequences (compare with Theorem 5.1.2).
Let and be metric spaces. A function is continuous at if and only if for every sequence in converging to the sequence in converges to .
Assume that is continuous at and let be a sequence in converging to . Let be arbitrary. Then there exists such that for all . Since , there exists such that for all . Therefore, for we have that . Since is arbitrary, this proves that converges to . Suppose that is not continuous at . Then there exists such that for every there exists with . Hence, if then there exists such that . Since then . On the other hand, it is clear that does not converge to . Hence, if is not continuous at then there exists a sequence converging to such that does not converge to . This proves that if every sequence in converging to it holds that converges to then is continuous at .
A level set of a function is a set of the form for some . Prove that if is continuous then the level sets of are closed sets.
As a consequence of Theorem 9.3.2, if is continuous at and then can be written as The sequential criteria for continuity can be conveniently used to show that the composition of continuous function is a continuous function.
Let and let , where , and are metric spaces. If is continuous at and is continuous at then the composite mapping is continuous at .
If then by Theorem 9.3.2, and using the fact that is continuous at , and is continuous at :
In general, given functions on metric spaces and , there is no general way to define the functions or since does not come equipped with a vector space structure nor is it equipped with a product operation. However, when then and are real numbers which can therefore be added/subtracted/multiplied.
Let be a metric space and let be continuous functions, where is equipped with the usual metric. If and are continuous at then , , and are continuous at .
In all cases, the most economical proof is to use the sequential criterion. The details are left as an exercise.
Recall that for any function and the set is defined as
For any function prove that for any .
For a given function the following are equivalent:
  1. is continuous on .
  2. is open in for every open subset .
  3. is closed in for every closed subset .
(i) (ii): Assume that is continuous on and let be open. Let and thus . Since is open, there exists such that . By continuity of , there exists such that if then . Therefore, and this proves that is open. (ii) (i): Let and let be arbitrary. Since is open, by assumption is open. Clearly and thus there exists such that , in other words, if then . This proves that is continuous at . (ii) (iii): This follows from the fact that for any set . Thus, for instance, if is open for every open set then if is closed then is open, that is, is open, i.e., is closed.
Use Proposition 9.3.7 to prove that the level sets of a function on a metric space are closed sets.
A function is called Lipschitz on if there exists such that for all . Prove that a Lipschitz function is continuous.
For recall that . Let be the trace function on , that is, . Show that is Lipschitz and therefore continuous.
Let denote the set of all real sequences that are bounded, that is, is a bounded set. If , it is straightforward to verify that defines a norm on with addition and scalar multiplication defined in the obvious way. Let be the set of absolutely summable sequences , that is, if and only if converges. It is not too hard to verify that is a normed vector space with norm defined as . If converges then converges to zero and thus , thus .
Fix and let be defined as for . Verify that is well-defined and prove that is continuous.
Let denote the determinant function. Prove that is continuous; you may use the formula where is the set of permutations on and is the sign of the permutation .

Exercises

Let be a metric space. Fix and define the function by . Prove that is continuous.
Let be a normed vector space. Prove that defined by is continuous. Hint: for all .
Consider with norm . Define the function by . Prove that is continuous in two ways, using the definition and the sequential criterion for continuity.
Let be a metric space and let be continuous. Prove that is closed.
Consider as a normed vector space with norm for .
  1. Let be a sequence in and denote the entry of the matrix as . Prove that converges to if and only if for all the real sequence converges to .
  2. Given matrices , recall that the entries of the product matrix are Let be a sequence in converging to and let be the sequence whose th term is . Prove that converges to . Hint: By part (a), it is enough to prove that the component of converges to the component of .
  3. Deduce that if where then the sequence converges to the matrix .
  4. A polynomial matrix function is a function of the form where are constants and denotes the identity matrix. Prove that a polynomial matrix function is continuous.
According to the sequential criterion for continuity, if and are sequences in converging to the same point and is a function such that sequences and do not have the same limit (or worse one of them is divergent!) then is discontinuous at . Consider given by Show that is discontinuous at .

Completeness

Consider the space of polynomial functions on the interval . Clearly, , and thus is a metric space. The sequence of functions is a sequence in and it can be easily verified that converges in the metric , that is, converges uniformly in (see Example 8.4.8). However, the limiting function is not an element of because it can be verified that and the only polynomial equal to its derivative is the zero polynomial, however, it is clear that , i.e., is not the zero function (you may recognize, of course, that ). We do know, however, that is in because the uniform limit of a sequence of continuous functions is continuous. The set then suffers from the same ``weakness'' as do the rationals relative to , namely, there are sequences in that converge to elements not in . On the other hand, because converges it is a Cauchy sequence in and thus also in (the Cauchy condition only depends on the metric) and thus is a Cauchy sequence in that does not converge to an element of . The following discussion motivates the following definition.
A metric space is called complete if every Cauchy sequence in converges in .
This seems like a reasonable starting definition of completeness since in it can be proved that the Cauchy criterion (plus the Archimedean property) implies the Completeness property of (Theorem 3.6.8). Based on our characterization of closed sets via sequences, we have the following first theorem regarding completeness.
Let be a complete metric space and let . Then is a complete metric space if and only if is closed.
If is a Cauchy sequence in then it is also a Cauchy sequence in . Since is complete then converges. If is closed then by Theorem 9.2.13 the limit of is in . Hence, is a complete metric space. Now suppose that is a complete metric space and let be a sequence in that converges to . Then is a Cauchy sequence in and thus also Cauchy in . Since is complete then . Hence, by Theorem 9.2.13 is closed.
We now consider how to formulate the Bolzano-Weierstrass (BW) property in a general metric space. The proof in Theorem 3.6.8 can be easily modifield to prove that the BW property, namely that every bounded sequence in has a convergent subsequence, implies the completeness property of . We therefore want to develop a BW-type condition in a general metric space that implies that is complete. Our first order of business is to develop the correct notion of boundedness. We have already defined what it means for a subset to be bounded, namely, that there exists such that for some . However, this notion is not enough as the next example illustrates.
Consider with induced metric and let , in other words, is the open ball of radius centered at the zero function. Clearly, is bounded and thus any sequence in is bounded. The sequence is in , that is, for all (see Example 3.3.6). However, as already discussed, converges in but not to a point in . On the other hand, is a Cauchy sequence in and thus cannot have a converging subsequence in by part (iii) of Lemma 9.2.6. Thus, is a bounded sequence in with no converging subsequence in .
The correct notion of boundedness that is needed is the following.
Let be a metric space. A subset is called totally bounded if for any given there exists such that .
Prove that a subset of a totally bounded set is also totally bounded.
A totally bounded subset of a metric space is bounded. If then if then if then . Hence, .
The following shows that the converse in the previous example does not hold.
Consider and let where is the infinite sequence with entry equal to and all other entries zero. Then for all and therefore is bounded, in particular for any . Now, for all and thus if then no finite collection of open balls can cover . Hence, is not totally bounded.
Prove that a bounded subset of is totally bounded.
Let be a metric space. Then is complete if and only if every infinite totally bounded subset of has a limit point in .
Suppose that is a complete metric space. Let be an infinite totally bounded subset of . Let for . For there exists such that . Since is infinite, we can assume without loss of generality that contains infinitely many points of . Let then . Now, is totally bounded and thus there exists such that . Since is infinite, we can assume without loss of generality that contains infinitely many points of . Let and therefore . Since is totally bounded there exists such that . We can assume without loss of generality that contains infinitely many elements of . Let and thus . By induction, there exists a sequence in such that . Therefore, if then by the triangle inequality (and the geometric series) we have Therefore, is a Cauchy sequence and since is complete converges in . Thus has a limit point in . Conversely, assume that every infinite totally bounded subset of has a limit point in . Let be a Cauchy sequence in and let . For any given there exists such that for all . Therefore, for all and clearly for all . Thus, is totally bounded. By assumption, has a limit point, that is, there exists a subsequence of that converges in . By part (iii) of Lemma 9.2.6, converges in . Thus, is a complete metric space.
The proof in Theorem 9.4.9 that completeness implies that every infinite totally bounded subset has a limit point is reminiscent of the bisection method proof that a bounded sequence in contains a convergent subsequence. Also, the proof showed the following.
If is an infinite totally bounded subset of then contains a Cauchy sequence such that for all .
A complete normed vector space is usually referred to as a Banach space in honor of Polish mathematician Stefan Banach (1892-1945) who, in his 1920 doctorate dissertation, laid the foundations of these spaces and their applications in integral equations. An important example of a Banach space is the following. Let be a non-empty set and let be the set of bounded functions from to with sup-norm . Then convergence in is uniform convergence (Example 9.2.3). We have all the tools necessary to prove that is a Banach space.
Let be a non-empty set. The normed space is a Banach space.
First of all, it is clear that is a real vector space and thus we need only show it is complete. The proof is essentially contained in the Cauchy criterion for uniform convergence for functions on (Theorem 8.2.7). Let be a Cauchy sequence of bounded functions. Then for any given there exists such that if then . In particular, for any fixed it holds that Therefore, the sequence of real numbers is a Cauchy sequence and thus exists for each . Now, since is a Cauchy sequence in then is bounded in . Thus, there exists such that for all . Thus, for all and it holds that and by continuity of the absolute value function it holds that Thus, is a bounded function, that is, . Now, for any fixed , let be such that for all and . Therefore, for any we have that Therefore, for all . This proves that converges to in .
The space of continuous functions on the interval with sup-norm is a Banach space.
A continuous function on the interval is bounded and thus . Convergence in with sup-norm is uniform convergence. A sequence of continuous functions that converges uniformly on does so to a continuous function. Hence, Theorem 9.2.13 implies that is a closed subset of the complete metric space and then Theorem 9.4.2 finishes the proof.
Prove that and are complete and hence Banach spaces.
In a Banach space, convergence of series can be decided entirely from the convergence of real series.
Let be a Banach space and let be a sequence in . If the real series converges then the series converges in .
Suppose that converges, that is, suppose that the sequence of partial sums converges (note that is increasing). Then is a Cauchy sequence. Consider the sequence of partial sums . For we have and since is Cauchy then can be made arbitrarily small provided are sufficiently large. This proves that is a Cauchy sequence in and therefore converges since is complete.
We make two remarks. The converse of Theorem 9.4.14 is also true, that is, if every series in converges whenever converges in then is a Banach space. Notice that the proof of Theorem 9.4.14 is essentially the same as the proof of the Weierstrass -test.
Consider the set of matrices equipped with the 2-norm The norm is called the Frobenius norm or the Hilbert-Schmidt norm.
  1. Prove that is complete.
  2. Use the Cauchy-Schwarz inequality to prove that . Conclude that for all .
  3. Let be a power series converging on . Define the function as Prove that is well-defined and that if then , that is, that
  1. The norm is simply the standard Euclidean norm on with and identifying matrices as elements of . Hence, is complete.
  2. From the Cauchy-Schwarz inequality we have and therefore
  3. We first note that for any power series that converges in , the power series also converges in . The normed space is complete and thus converges whenever converges. Now by part (b), and since converges then by the comparison test for series in , the series converges. Therefore, is well-defined by Theorem 9.4.14. To prove the last inequality, we note that the norm function on a vector space is continuous and thus if then and therefore in other words, .
In view of the previous example, we can define for the following: and for instance , etc.

Exercises

Let and be metric spaces. There are several ways to define a metric on the Cartesian product . One way is to imitate what was done in . We can define as
  1. Prove that is a metric on .
  2. Prove that converges in if and only if and converge in and , respectively. (Hint: Theorem 9.2.7)
  3. Prove that is complete if and only if and are complete. (Hint: Corollary 9.2.8)
Let be a complete metric space and let be a sequence in such that for all for some fixed . Prove that converges. (See Exercise 3.6.6.)
Let be a convergent series in a normed vector space and suppose that converges. Show that Note: The -inequality can only be used on a finite sum. (See Exercise 9.3.2.)
Consider the normed space where is a non-empty set. Let be the set of constant functions on . Prove that is a Banach space. (Hint: Theorem 9.4.2)

Compactness

Important results about continuous functions, such as the Extreme Value Theorem (Theorem 5.3.7) and uniform continuity (Theorem 5.4.7), depended heavily on the domain being a closed and bounded interval. On a bounded interval, any sequence contains a Cauchy subsequence (use the bisection algorithm), and if the interval is also closed then we are guaranteed that the limit of is contained in the interval. We have already seen that a totally bounded subset of a metric space contains a Cauchy sequence (Lemma 9.4.10) and thus if is complete then Cauchy sequences converge in . This motivates the following definition.
A metric space is called compact if is totally bounded and complete.
A closed and bounded subset of is compact. Indeed, is complete because it is closed (Theorem 9.4.2) and it is easy to see how to cover with a finite number of open intervals of any given radius . Conversely, if is compact then is bounded and is closed by Theorem 9.4.2. A similar argument shows that is compact if and only if is closed and bounded. This is called the Heine-Borel theorem.
A subset is compact if and only if is closed and bounded.
The unit -sphere in is the set Explain why is compact subset of .
Consider with norm . Then is complete. A matrix is called orthogonal if , where denotes the transpose of and is the identity matrix. Prove that the set of orthogonal matrices, which we denote by , is compact.
A useful characterization of compactness is stated in the language of sequences.
A metric space is compact if and only if every sequence in has a convergent subsequence.
Assume that is compact. If is a sequence in then is totally bounded and thus has a Cauchy subsequence which converges by completeness of . Conversely, assume that every sequence in has a convergent subsequence. If is a Cauchy sequence then by assumption it has a convergent subsequence and thus converges. This proves is complete. Suppose that is not totally bounded. Then there exists such that cannot be covered by a finite number of open balls of radius . Hence, there exists such that . By induction, suppose are such that for . Then there exists such that for all . By induction then, there exists a sequence such that if . Clearly, is not a Cauchy sequence and therefore cannot have a convergent subsequence.
Let be a metric space.
  1. Prove that if is finite then is compact.
  2. Is the same true if is countable?
  3. What if is compact?
We now describe how compact sets behave under continuous functions.
Let be a continuous mapping. If is compact then is compact.
We use the sequential criterion for compactness. Let be a sequence in . Since is compact, by Theorem 9.5.5, there is a convergent subsequence of . By continuity of , the subsequence of is convergent. Hence, is compact.
We now prove a generalization of the Extreme value theorem 5.3.7.
Let be a compact metric space. If is continuous then achieves a maximum and a minimum on , that is, there exists such that for all . In particular, is bounded.
By Theorem 9.5.7, is a compact subset of and thus is closed and bounded. Let and let . By the properties of the supremum, there exists a sequence in converging to . Since is closed, then and thus for some . A similar argument shows that for some . Hence, for all .
Let be a metric space and let . A cover of is a collection of subsets of whose union contains . The index set may be countable or uncountable. The cover is called an open cover if each set is open. A subcover of a cover of is a cover of such that .
A metric space is compact if and only if every open cover of has a finite subcover.
Assume that is compact. Then by Theorem 9.5.5, every sequence in has a convergent subsequence. Let be an open cover of . We claim there exists such that for each it holds that for some . If not, then then for each there exists such that is not properly contained in a single set . By assumption, the sequence has a converging subsequence, say it is and . Hence, for each , is not properly contained in a single . Now, for some , and thus since is open there exists such that . Since , there exists sufficiently large such that and . Then which is a contradiction. This proves that such an exists. Now since is totally bounded, there exists such that , and since for some it follows that is a finite subcover of . For the converse, we prove the contrapositive. Suppose then that is not compact. Then by Theorem 9.5.5, there is a sequence in with no convergent subsequence. In particular, there is a subsequence of such that all 's are distinct and has no convergent subsequence. Then there exists such that contains only the point from the sequence , otherwise we can construct a subsequence of that converges. Hence, is an open cover of the set that has no finite subcover. The set is clearly closed since it consists entirely of isolated points of . Hence, is an open cover of with no finite subcover.
A function is uniformly continuous if for any there exists such that if then .
A function is Lipschitz if there is a constant such that . Show that a Lipschitz map is uniformly continuous.
If is uniformly continuous and is a Cauchy sequence in , prove that is a Cauchy sequence in .
If is continuous and is compact then is uniformly continuous.
Let . For each , there exists such that if then . Now is an open cover of and since is compact there exists finite such that is an open cover of , where we have set . Let . If , and say , then and thus This proves that is uniformly continuous.

Exercises

Prove that if is compact then and are elements of .
Recall that is the set of sequences in that are bounded and equipped with the norm . Show that the unit ball (which is clearly bounded) is not compact in . (see Example 9.4.7)
Let be a sequence in a metric space and suppose that converges to . Prove that is a compact subset of .
Prove that if is compact then there exists a countable subset that is dense in .
Let be a compact subset of and fix . Prove that there exists such that for all .

Fourier Series

Motivated by problems involving the conduction of heat in solids and the motion of waves, a major problem that spurred the development of modern analysis (and mathematics in general) was whether an arbitrary function can be represented by a series of the form for appropriately chosen coefficients and . A central character in this development was mathematician and physicist Joseph Fourier and for this reason such a series is now known as a Fourier series. Fourier made the bold claim (Theorie analytique de la Chaleur, 1822) that there is no function or part of a function, which cannot be expressed by a trigonometric series. Fourier's claim led B. Riemann (1854) to develop what we now call the Riemann integral. After Riemann, Cantor's (1872) interest in trigonometric series led him to the investigation of the derived set of a set (which nowadays we call the limit points of ) and he subsequently developed set theory. The general problem of convergence of a Fourier series led to the realization that by allowing arbitrary functions into the picture the theory of integration developed by Riemann would have to be extended to widened the class of integrable functions. This extension of the Riemann integral was done by Henri Lebesgue (1902) and spurred the development of the theory of measure and integration. The Lebesgue integral is widely accepted as the official integral of modern analysis. In this section, our aim is to present a brief account of Fourier series with the tools that we have already developed. To begin, suppose that is Riemann integrable and can be represented by a Fourier series, that is, for . In other words, the series on the right of converges pointwise to on . The first question we need to answer is what are the coefficients and in terms of ? To that end, we use the following facts. Let :
  1. For all :
  2. If then
  3. For all and :
  4. For all and :
Then, using these facts, and momentarily ignoring the interchange of the integral and infinite sum, Therefore, A similar calculation shows that Finally, and therefore Of course, the above calculations are valid provided that the Fourier series converges uniformly to on since if all we have is pointwise convergence then in general we cannot interchange the integral sign and the infinite sum. Since the functions in the Fourier series are clearly continuous, and if we insist that the convergence is uniform, then we have restricted our investigation of Fourier series to continuous functions! Relaxing this restriction led to the development of what we now call the Lebesgue integral; Lebesgue was interested in extending the notion of integration beyond Riemann's development so that a wider class of functions could be integrated and, more importantly, this new integral would be more robust when it came to exchanging limits with integration, i.e., uniform convergence would not be needed. A full development of Lebesgue's theory of integration is beyond the scope of this book, however, we can still say some interesting things about Fourier series. Motivated by our calculations above, suppose that and define Assume that the Fourier series of converges uniformly on and let Then is continuous on . Does ? To answer this question, our computations above show that and therefore for all . Similarly, for all we have Let and recall that converges uniformly to . Consider for the moment A straightforward computation shows that and Therefore, Now since it follows that or equivalently that This proves that the series converges and In particular, and thus .