Administrative info
  PA1 solutions on Piazza
  HW4 out today, due Monday
  PA2 out today
  MT1 next Tuesday
  Review session Sunday 7/10 5pm in 310 Soda

  Last time, we reviewed polynomials and saw how to reconstruct a
  polynomial of degree d given d+1 points. Then we saw a procedure for
  sharing a secret among n people such that k of them are needed to
  recover it: construct a degree k-1 polynomial P(x) modulo some large
  prime q such that P(0) ≡ s (mod q), the secret. Then give each
  person a different point from the polynomial. If k of them
  cooperate, they can reconstruct the degree k-1 P(x) and compute P(0)
  to retrieve the secret.

Error Correction
  We know turn our attention to another application of polynomials,
  that of sending messages over unreliable networks.

  Consider the following situations:
  (1) FM radio has a range of ~10 miles, transmission power ~100 KW
  (2) GPS satellite: 10,000 miles, 50 W
      (reception power goes down by square of distance, so 2 billion
      times weaker)
  (3) Voyager space probe: 10 billion miles, 12-18 W
      (10^22 times weaker)
  Why is it that FM reception is flaky, but GPS works quite well, and
  we can communicate with Voyager? Error correcting codes!

  We want to be able to send long messages over unreliable channels
  but still be able to recover the message. (For example, sending a
  picture from a Neptune probe to Earth.) If a message is split into
  multiple packets, some of those may be lost. Our assumption is that
  the packets are labelled, so we know which packets are lost. We also
  assume that packets either arrive correctly in their entirety or are
  completely lost (i.e. they have been "erased"). (Maybe they arrive
  corrupted, but we assume we can detect this and throw them away.)

  Suppose we divide a message into n 100 bit packets, m_1, m_2, ...,
  m_n. How can we send them so that they can be recovered reliably?
  Let's assume that no more than k losses. (In practice, we'd pick
  some k such that there is a very high probability that no more than
  k are lost.)

  We could just send each packet m_i many times, so that we can ensure
  that at least one copy of each packet arrives at the destination.
  But this is inefficient, since it transmits a lot of redundant

  Instead, we use polynomials to reduce the amount of redundant
  information we have to send. We use the following scheme:
  (1) Pick a prime q, q > 2^100 (so all 100 bit packets can be
      represented). (Note: The receiver must know what q is! So
      transmitter and receiver must agree beforehand on the value of
  (2) Construct the degree n-1 polynomial over GF(q)
        P(x) ≡ m_n x^{n-1} + m_{n-1} x^{n-1} + ... + m_2 x + m_1
  (3) Transmit the n+k encoded packets P(1), P(2), ..., P(n+k).
  Does this scheme work? We assumed that no more than k packets are
  lost, so at least n packets arrive at Earth from Neptune. Then
  mission control performs Lagrange interpolation to reconstruct P(x)
  and reads off the resulting coefficients as the intended message.

  Note that this scheme sends a lot less information than the naive
  method of sending the same packet multiple times. It only requires
  n+k packets rather than some c*n.

  EX: Suppose we want to send the message (5, 0, 4, 1) with a maximum
      of 2 packets getting lost. Then:
      (1) We pick q = 7 > 5 so that all our packets can be
      (2) We construct the degree 3 polynomial
            P(x) ≡ 1x^3 + 4x^2 + 0x + 5 (mod 7)
                 ≡ x^3 + 4x^2 + 5 (mod 7).
      (3) We transmit n+k = 6 packets
            P(1) ≡ 3 (mod 7)
            P(2) ≡ 1 (mod 7)
            P(3) ≡ 5 (mod 7)
            P(4) ≡ 0 (mod 7)
            P(5) ≡ 6 (mod 7)
            P(6) ≡ 1 (mod 7)
      Now as long as at least 4 packets, the receivier can reconstruct
        P(x) ≡ 1x^3 + 4x^2 + 0x + 5 (mod 7)
      and read off the message (5, 0, 4, 1).

  This scheme is known as Reed-Solomon encoding. It was used by
  Voyager and is used in DVDs, satellite communcitions, some digital
  radio standards (e.g. DAB+), and in many other applications.

  Compare this scheme with secret sharing. Let n be the number of
  people who must be present out of a total of n+k people (where k is
  the number of people who are "lost"). We wanted to "send" a single
  packet, but we wanted to ensure that n packets arrived before the
  "message" could be recovered. So we essentially padded the secret
  with n-1 random packets so that it reached the required message size
  of n. The secret is P(0) ≡ m_1 ≡ s, and m_2, ..., m_n are random

  Finally, what happens if we no longer assume that we can detect
  corrupted packets? We can still recover the message if we send n+2k
  packets using the above encoding scheme, where at most k can be
  corrupted or lost. The Berlekamp-Welch algorithm accomplishes this,
  though we won't cover it in this class.

  We have now completed the second unit of this course, modular
  arithmetic. We studied the mathematical theory and then saw how to
  use it in three very cool applications: encryption, secret sharing,
  and error correction. We now turn our attention to the next unit in
  the course, probability theory.


Probability Theory
  Probability theory is central to many aspects of EECS. For example,
  we may want to determine how well an algorithm (e.g. quick sort)
  works in the "average" case. Or algorithms may include randomness to
  probabilistcally improve efficiency (e.g. hashing). Probability is
  also used in artificial intelligence (infer most likely value of an
  unknown quantity) and in many other areas of EECS.

  Let's look at some simple examples of computing probabilities.
  (1) If I flip a fair coin once, what is the chance of heads?
      There are two possibilities, each of which are equally likely,
      and only one of which is heads. So the probability should be
  (2) If I flip a fair coin twice, what is the chance of two heads?
      There are now four possibilities: HH, TH, HT, and TT. (Note that
      TH and HT are different, since one has heads in the first flip
      and the other in the second.) Only one of these has two heads,
      so the probability should be 1/4.
  (3) If I flip a fair coin 100 times, what is the chance of exactly
      50 heads?
      It is no longer trivial. We need to count how many sequences of
      100 flips have exactly 50 heads and the divided by the total
      number of possible sequences. (The probability is ~8% or 0.08,
      by the way.)
  Thus, we need to learn how to count before we can move on to
  computing probabilities.

  We want to know how to count large quantities without actually
  counting them. For example, how many seats are there in this room?
  Instead of counting one-by-one, we count how many seats are in each
  row and multiply by the number of rows. This is the basic idea:
  count big quantities by counting smaller quantities and then
  combining those to get a result for the big quantity. (Does this
  remind you of anything? Recursion!)

  Note that what we are doing in counting is counting the size of some
  set S (e.g. the set of sequences of flips with exactly 50 heads, the
  set of all possible sequences of flips). So if we can express S in
  terms of smaller sets (not necessarily subsets), we can count the
  sizes of the smaller sets to help us count the size of S.

  We will now look at some general counting principles that can help
  us in counting large quantities.

  (1) Enumeration
      Just list all possiblities. (We need to make sure that we've
      covered everything!)
      EX: How many bit strings of length 3 are there with no
          consecutive zeroes?
      ANS: 010, 011, 101, 110, 111
      How do we make sure we covered every possiblity? One way is
      to draw a decision tree:
                         Legal bit strings
                     0   010
             0 - 1 - 1   011
           /     0 - 1   101
          /     /
         /     /     0   110
        /     /    /
      x ---- 1 - 1 - 1   111
       Bit # 1   2   3
      EX: How many 2-out-of-3 game playoffs are there?
                    Winner sequence   Winner
               b          bb            b
             /   / b      bab           b
           b - a - a      baa           a
         /       / b      abb           b
        /    / b - a      aba           a
      x -- a - a          aa            a