Administrative info HW8 due Wednesday Final exam Thursday 5-8pm in 10 Evans No regrades for HW8 (not enough time) or final exam (UCB policy) Review session tomorrow 3-5pm in 306 Soda Countability We have now completed the fourth unit of this course, continuous probability, and now move on to the last unit, diagonalization and self-reference. We have seen in probability the difference between discrete and continuous random variables. The former can only take on values in a discrete, but possibly infinite, subset of R, while the latter can take on values in a continuous subset of R. In the discrete case, we could talk about the event that a random variable takes on a particular value, and we could use summations to compute expectations and variances. In the continuous case, we had to resort to looking at intervals of values that the random variable takes on, and we had to use integrals to compute expectations and variances. Why were these two cases different? In both cases, a random variable could take on infinitely many different values. However, they appear to be different types of infinity, and we will now see exactly how they are different. We follow the reasoning of Cantor, starting by a principle that we have seen in counting: (4) Isomorphism Principle If two sets S1 and S2 can be put into a bijective correspondence, then |S1| = |S2|. Here, we use the term "cardinality" to refer to the size of a (possibly infinite) set. To understand what the above principle means, let's first define a bijection. Let f: A -> B be a function from set A to set B, i.e. with domain A and range B. Then f is "one-to-one" or "injective" if f(x) is different for different values of x. More formally, ∀x,y∈A . x ≠ y => f(x) ≠ f(y) ∀x,y∈A . f(x) = f(y) => x = y. (The above two statements are contrapositives of each other.) And f is "onto" or "surjective" if f takes on all possible values in B: ∀y∈B ∃x∈A . f(x) = y. Finally, f is "bijective" if it is one-to-one and onto. Then two sets S1 and S2 have a bijective correspondence if there is a bijective function f : S1 -> S2. As an example, we saw that the power set P(S) (set of all subsets) of a set S = {a1, a2, ..., an} has a bijective correspondence with the set of n-bit strings. The following function provided this bijection: f(x) = (g(x,a1), g(x,a2), ..., g(x,an)), where g(x,a) = { 1 if a∈x { 0 otherwise. As an example, let S = {1, 2, 3, 4, 5}, x = {1, 3, 4}. Then f(x) = (1, 0, 1, 1, 0). Since P(S) has a bijective correspondence with the set of n-bit strings, |P(S)| = |{0,1}^n| = 2^n. The principle we use, then, is that two sets S and T have the same cardinality if there is a bijective function between S and T. We say that a set S is "countable" if there is a bijection between S and some subset of N. This implies that either S has a finite cardinality (if the subset of N is finite) or that S is "countably infinite." In the latter case, we will see that S and N have the same cardinality. As an example, consider the set of positive integers Z^+. It seems that Z^+ and N should not have the same cardinality, since Z^+ is a strict subset of N. However, the following function demonstrates a bijection between N and Z^+: f:N->Z^+ f(n) = n + 1. This function is one-to-one, since f(n) is different for different n. It is onto, since there is an input n for every output in Z^+. Thus, it is a bijection, so N and Z^+ have the same cardinality and Z^+ is countable. Strange, but true, and this is what happens when we deal with infinities. What about the set of even integers E? The following is a bijection between N and E: f:N->E f(n) = 2n. So E is also countable, and N and E have the same cardinality, even though we'd expect the cardinality of N to be twice that of E! What about the integers Z? We can define the following bijection between N and Z: f:N->Z f(n) = { n/2 if n even { -(n+1)/2 if n odd. So we get n f(n) 0 0 1 -1 2 1 3 -2 4 2 ... ..., and we see that this is indeed a bijection. Thus, Z is countable. Enumerating the elements of S in a list implicitly defines a bijection between S and a subset of N. If the list is finite, then it defines a bijection between S and {0, 1, ..., |S|-1}. If the list is infinite, then it defines a bijection between S and N. For S = Z, we get elements in Z (<--> elements in N) 0 (<--> 0) -1 (<--> 1) 1 (<--> 2) -2 (<--> 3) 2 (<--> 4) ... (<--> ...) Thus, showing that S can be enumerated in a (possibly infinite) list is enough to show that S is countable, since we can use the list to demonstrate a bijection. In enumerating a set S, though, we have to make sure that our enumeration covers all elements in S. In other words, given any element s∈S, it must appear at some concrete position in the list of all elements in S. This is necessary for the implicit bijective function we are defining to be onto. Using the above technique, we can show that any subset T of a countable set S is also countable. We know that S can be enumerated in a list, so removing the elements not in T from the list for S results in a list for T. Thus, T is countable. This implies that any subset of N is countable, and any infinite subset U of N (e.g. Z^+) is countably infinite, since we can enumerate U in a list. This defines an implicit bijection between U and N, so they have the same cardinality. This further implies that all countably infinite sets have the same cardinality, since by definition, a set S is countably infinite if it has a bijection with a countably infinite subset of N. As another example of enumeration, consider the set BS of all finite binary strings, i.e. BS = {0,1}^*. We can enumerate BS in a list, ordering elements first by length and then by lexicographic order: BS = {ε, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, ...}, where ε is the empty string. Since BS can be enumerated, it is countable. A bad enumeration of BS is to enumerate its elements strictly in lexicographic order: BS = {ε, 0, 00, 000, 0000, ...} As you can see, this enumeration doesn't cover all elements in BS, so it is not a valid enumeration of BS. What about the set ES of all finite English sentences? Every English sentence can be encoded as a finite binary string (using ASCII or Unicode), so there is a bijection between ES and a subset of BS. Since BS is countable, any subset of BS is countable, implying that ES is countable. What about pairs of natural numbers, N x N? For Cartesian products of finite sets, the product rule tells us that |S1 x S2| = |S1| x |S2|. So it seems like the cardinality of N x N should follow a similar pattern, that we get some sort of ∞^2, whatever that means. However, we can enumerate the elements in N x N, first by sum and then by lexicographic order: N x N = {(0,0), (0,1), (1,0), (0,2), (1,1), (2,0), (0,3), (1,2), (2,1), (3,0), ...}. So N x N is countable and has the same cardinality as N! And in fact, we can use the same technique to show that the Cartesian product N^m is countably infinite for any m∈Z^+. What about Q, the set of rationals? Then Q = {a/b : a∈Z ∧ b∈Z^+ ∧ gcd(a,b) = 1}, where we represent a rational as a reduced fraction with a positive integer denominator. (By our definition, 0 has the unique representation 0/1; for 0/b for any other b, gcd(0, b) = b ≠ 1.) Then there is a bijection between Q and the following subset of N x N: S = {(a, b) : a∈Z ∧ b∈Z^+ ∧ gcd(a,b) = 1}. Since a subset of a countable set is countable, S is countable, and so is Q. Every set we've looked out so far, even the complicated ones, has been countable. Are there sets that are not countable? Diagonalization Let us now consider the set of real numbers in the interval [0, 1]. Is this set countable? Let's try enumerating all the elements in a list. All real numbers have decimal representations, though those representations may be infinite (e.g. 1/3 = 0.3333...). In fact, we will use only infinite representations in our list, by filling with 0's if necessary (e.g. 1/4 = 0.250000...). Finally, if a number has more than one valid decimal representation (e.g. 1 = 0.9999...), then we pick one arbitrarily (though we pick 0.9999... for 1 so that all representations in our list start with "0."). So let's assume that [0, 1] is countable and proceed to enumerate its elements by writing out their decimal representations in a list. (Since the list is infinite anyway, we don't mind writing down infinite decimal representations in the list.) We list them in some arbitrary order: i r∈[0, 1] 0 0.250000... 1 0.333333... 2 0.999999... 3 0.326244... 4 0.624601... 5 0.648756... ... ... In the above list, we've underlined the diagonal, so that for the ith element in the list, its ith digit after the decimal is underlined. This results in a new element in [0, 1]: s = 0.239206... Now consider a new number t, where each digit in t is the corresponding digit in s, plus 2, modulo 10. So we get t = 0.451428... Clearly t∈[0, 1]. Does t appear in our enumeration above? Well, t is not equal to element 0 in the list, since it differs from 0.250000... by 2 in digit 0 after the decimal. (Note that if two decimal representations differ by more than 1 in a digit, then they cannot represent the same number.) And t differs from 0.333333... by 2 in digit 1, so it is not the same as element 1. Similarly, t is not equal to any element i in the list, since it differs from element i by 2 in its ith digit. Thus, t is not in the list. This is a contradiction, since the list contains all real numbers in [0, 1], but t is in [0, 1] and is not in the list. Thus, our original assumption that we could enumerate [0, 1] is false, and [0, 1] is not countable. This technique was invented by Cantor and is called "diagonalization." It involves the following steps: (1) Assume that a set S can be enumerated. (2) Consider an arbitrary list of all the elements of S. (3) Use the diagonal from the list to construct a new element t. (4) Show that t is in S but is different from all elements in the list and so is not in the list. Contradiction. This shows that the original assumption that S is countable was false. Therefore, S is "uncountable" or "uncountably infinite," and its cardinality is strictly larger than that of N. Since [0, 1] is uncountable, by the contrapositive of our statement above that any subset of a countable set is countable, the set R of all real numbers is also uncountable. Let's look at another example of diagonalization. Consider the set IBS of all infinite binary strings. We assume that IBS is countable, so we can list all elements of IBS: i s∈IBS 0010101... 1 111001... 2 000000... 3 010011... 4 111101... 5 000111... ... ... We flip the digits in the diagonal to construct a new element t that is not on the list: t = 101110... Since t∈IBS but is not in the list of all elements in IBS, this is a contradiction, and the assumption that IBS is countable is false. What happens if we try to use diagonalization on a countable set? If we try to proceed as above for the set BS of finite binary strings, the resulting t will have infinite length. (We have to change the construction of t slightly. Since the ith string in the list of elements of BS may have length less than i, we may not be able to flip its ith bit to get the ith bit of t. In that case, we just choose the ith bit of t arbitrarily, which ensures that t is different than that ith string in the list.) Since t has infinite length, it is not in BS and should not appear on the list anyway. So step (4) of the diagonalization process fails, and no contradiction is generated.