Administrative info PA1 solutions on Piazza HW4 out today, due Monday PA2 out today MT1 next Tuesday Review session Sunday 7/10 5pm in 310 Soda Review Last time, we reviewed polynomials and saw how to reconstruct a polynomial of degree d given d+1 points. Then we saw a procedure for sharing a secret among n people such that k of them are needed to recover it: construct a degree k-1 polynomial P(x) modulo some large prime q such that P(0) ≡ s (mod q), the secret. Then give each person a different point from the polynomial. If k of them cooperate, they can reconstruct the degree k-1 P(x) and compute P(0) to retrieve the secret. Error Correction We know turn our attention to another application of polynomials, that of sending messages over unreliable networks. Consider the following situations: (1) FM radio has a range of ~10 miles, transmission power ~100 KW (2) GPS satellite: 10,000 miles, 50 W (reception power goes down by square of distance, so 2 billion times weaker) (3) Voyager space probe: 10 billion miles, 12-18 W (10^22 times weaker) Why is it that FM reception is flaky, but GPS works quite well, and we can communicate with Voyager? Error correcting codes! We want to be able to send long messages over unreliable channels but still be able to recover the message. (For example, sending a picture from a Neptune probe to Earth.) If a message is split into multiple packets, some of those may be lost. Our assumption is that the packets are labelled, so we know which packets are lost. We also assume that packets either arrive correctly in their entirety or are completely lost (i.e. they have been "erased"). (Maybe they arrive corrupted, but we assume we can detect this and throw them away.) Suppose we divide a message into n 100 bit packets, m_1, m_2, ..., m_n. How can we send them so that they can be recovered reliably? Let's assume that no more than k losses. (In practice, we'd pick some k such that there is a very high probability that no more than k are lost.) We could just send each packet m_i many times, so that we can ensure that at least one copy of each packet arrives at the destination. But this is inefficient, since it transmits a lot of redundant information. Instead, we use polynomials to reduce the amount of redundant information we have to send. We use the following scheme: (1) Pick a prime q, q > 2^100 (so all 100 bit packets can be represented). (Note: The receiver must know what q is! So transmitter and receiver must agree beforehand on the value of q.) (2) Construct the degree n-1 polynomial over GF(q) P(x) ≡ m_n x^{n-1} + m_{n-1} x^{n-1} + ... + m_2 x + m_1 (3) Transmit the n+k encoded packets P(1), P(2), ..., P(n+k). Does this scheme work? We assumed that no more than k packets are lost, so at least n packets arrive at Earth from Neptune. Then mission control performs Lagrange interpolation to reconstruct P(x) and reads off the resulting coefficients as the intended message. Note that this scheme sends a lot less information than the naive method of sending the same packet multiple times. It only requires n+k packets rather than some c*n. EX: Suppose we want to send the message (5, 0, 4, 1) with a maximum of 2 packets getting lost. Then: (1) We pick q = 7 > 5 so that all our packets can be represented. (2) We construct the degree 3 polynomial P(x) ≡ 1x^3 + 4x^2 + 0x + 5 (mod 7) ≡ x^3 + 4x^2 + 5 (mod 7). (3) We transmit n+k = 6 packets P(1) ≡ 3 (mod 7) P(2) ≡ 1 (mod 7) P(3) ≡ 5 (mod 7) P(4) ≡ 0 (mod 7) P(5) ≡ 6 (mod 7) P(6) ≡ 1 (mod 7) Now as long as at least 4 packets, the receivier can reconstruct P(x) ≡ 1x^3 + 4x^2 + 0x + 5 (mod 7) and read off the message (5, 0, 4, 1). This scheme is known as Reed-Solomon encoding. It was used by Voyager and is used in DVDs, satellite communcitions, some digital radio standards (e.g. DAB+), and in many other applications. Compare this scheme with secret sharing. Let n be the number of people who must be present out of a total of n+k people (where k is the number of people who are "lost"). We wanted to "send" a single packet, but we wanted to ensure that n packets arrived before the "message" could be recovered. So we essentially padded the secret with n-1 random packets so that it reached the required message size of n. The secret is P(0) ≡ m_1 ≡ s, and m_2, ..., m_n are random junk. Finally, what happens if we no longer assume that we can detect corrupted packets? We can still recover the message if we send n+2k packets using the above encoding scheme, where at most k can be corrupted or lost. The Berlekamp-Welch algorithm accomplishes this, though we won't cover it in this class. Summary We have now completed the second unit of this course, modular arithmetic. We studied the mathematical theory and then saw how to use it in three very cool applications: encryption, secret sharing, and error correction. We now turn our attention to the next unit in the course, probability theory. ======= Probability Theory Probability theory is central to many aspects of EECS. For example, we may want to determine how well an algorithm (e.g. quick sort) works in the "average" case. Or algorithms may include randomness to probabilistcally improve efficiency (e.g. hashing). Probability is also used in artificial intelligence (infer most likely value of an unknown quantity) and in many other areas of EECS. Let's look at some simple examples of computing probabilities. (1) If I flip a fair coin once, what is the chance of heads? There are two possibilities, each of which are equally likely, and only one of which is heads. So the probability should be 1/2. (2) If I flip a fair coin twice, what is the chance of two heads? There are now four possibilities: HH, TH, HT, and TT. (Note that TH and HT are different, since one has heads in the first flip and the other in the second.) Only one of these has two heads, so the probability should be 1/4. (3) If I flip a fair coin 100 times, what is the chance of exactly 50 heads? It is no longer trivial. We need to count how many sequences of 100 flips have exactly 50 heads and the divided by the total number of possible sequences. (The probability is ~8% or 0.08, by the way.) Thus, we need to learn how to count before we can move on to computing probabilities. Counting We want to know how to count large quantities without actually counting them. For example, how many seats are there in this room? Instead of counting one-by-one, we count how many seats are in each row and multiply by the number of rows. This is the basic idea: count big quantities by counting smaller quantities and then combining those to get a result for the big quantity. (Does this remind you of anything? Recursion!) Note that what we are doing in counting is counting the size of some set S (e.g. the set of sequences of flips with exactly 50 heads, the set of all possible sequences of flips). So if we can express S in terms of smaller sets (not necessarily subsets), we can count the sizes of the smaller sets to help us count the size of S. We will now look at some general counting principles that can help us in counting large quantities. (1) Enumeration Just list all possiblities. (We need to make sure that we've covered everything!) EX: How many bit strings of length 3 are there with no consecutive zeroes? ANS: 010, 011, 101, 110, 111 How do we make sure we covered every possiblity? One way is to draw a decision tree: Legal bit strings 0 010 / 0 - 1 - 1 011 / / 0 - 1 101 / / / / 0 110 / / / x ---- 1 - 1 - 1 111 Bit # 1 2 3 EX: How many 2-out-of-3 game playoffs are there? Winner sequence Winner b bb b / / b bab b b - a - a baa a / / b abb b / / b - a aba a x -- a - a aa a