Administrative info
  HW5 due tomorrow
  HW6 out tonight

  In defining events, we noted that we often do not care about a
  specific outcome to a random experiment but whether or not the
  outcome is part of a special set of outcomes, which we called
  events. For example, when flipping a fair coin 100 times, we may
  care about whether or not the number of heads and tails is the same.
  So we define E as the event that we get 50 heads and compute Pr[E] =
  C(100, 50) / 2^100.

Random Variables
  Sometimes, what we care about is a numerical value of an outcome.
  For example, if we receive $1 for each heads in 100 flips of a fair
  coin and lose $1 for each tails, we care about how much total money
  we earn or lose. For any particular outcome ω, this is just
  the number of heads in ω minus the number of tails. We can
  compute the amount of money we win or lose for every one of the 2^n
  outcomes. This is a "random variable."

  A random variable is a function that assigns a value to each sample
  point. More formally, a random variable X is a function from
  Ω, the sample space to R, the set of real numbers. The value
  of the random variable at sample point ω is denoted as
  X(ω) (like any other function). (Note: a random variable is
  neither random nor a variable, since it is a function. Why is it
  called a random variable? I don't know, but since the outcome of an
  experiment is random, the value of the random variable is a function
  of a random outcome.)

  Let's go back to coin flipping. Suppose I flip a fair coin once. Let
  X be a random variable that is +1 if I get heads, -1 if I get tails.
  What is X(ω) for each outcome ω? Well, X(h) = +1 and
  X(t) = -1.

  What if I flip a fair coin three times, where X is the amount I win
  if I win $1 for each heads, lost $1 for each tails? Then
    X(hhh) = 3    X(hth) = 1     X(thh) = 1     X(tth) = -1
    X(hht) = 1    X(htt) = -1    X(tht) = -1    X(ttt) = -3

  In general, if I flip a fair coin n times, then if X is the amount I
  win, X(ω) = H(ω) - T(ω), where H(ω) is the
  number of heads in ω and T(ω) is the number of tails in
  ω. Notice that H and T are also random variables, since they
  assign a real number to each sample point. Defining a random
  variable in terms of simpler random variables is a very useful
  procedure. (Notice a common theme with induction, counting,
  probability, and now random variables? All involve reducing a hard
  problem to simpler problems.)

  We can actually note that H(ω) + T(&omega) = n, since each
  flip must be heads or tails. So we can further write
    X(ω) = H(ω) - (n - H(ω))
      = 2 H(ω) - n.

  Suppose that rather than handing back your exams individually in
  section, we hand back a random exam to each person in lecture. Let
  X be the number of students who get their own homeworks back. What
  is X(ω) for each sample point ω?

  First, let us determine the sample space. Each outcome is just a
  permutation of the n students in class {1, ..., s}. For example,
  ω = (2, 3, ..., n, 1) corresponds to student i getting the
  exam for student mod(i+1, n). Since each outcome is a permutation,
  there are n! outcomes, which we assume to have uniform probability.

  Now let's define a series of simpler random variables. Let X_i be a
  random variable that is 1 if the ith person gets his or her own
  homework back, 0 otherwise. Such a 0/1-valued random variable is
  called an "indicator random variable" and is a very common and
  useful type of random variable. Then we have
    X(ω) = X_1(ω) + ... + X_n(ω).

  As a concrete example, suppose n = 3. Then the outcomes are
    (1,2,3)  (1,3,2)  (2,1,3)  (2,3,1)  (3,1,2)  (3,2,1)
  Then the values of the X_i are
       1        1        0        0        0        0       X_1
       1        0        0        0        0        1       X_2
       1        0        1        0        0        0       X_3
  and the values of X are
       3        1        1        0        0        1        X.

  We can use the same procedure in the case of coin flipping. Here, X
  is the total amount won in n flips. Let X_i be the amount won in the
  ith flip, +1 if it is heads, -1 if it is tails. Then
    X(ω) = X_1(ω) + ... + X_n(ω).
  Then in the case of n = 3, we have
     hhh   hht   hth   htt   thh   tht   tth   ttt
      1     1     1     1    -1    -1    -1    -1           X_1
      1     1    -1    -1     1     1    -1    -1           X_1
      1    -1     1    -1     1    -1     1    -1           X_1
      3     1     1    -1     1    -1    -1    -3            X
  as before.

  As with events, we often don't care about the value of a random
  variable at each outcome. Rather, we care about the probability that
  the random variable takes any particular value. In fact, we can
  define events in terms of random variables. We define "X = a" to be
  the event that the random variable X takes on value a. More
    X = a ≡ {&omega : &omega ∈ Ω ∧ X(&omega) = a},
  i.e. X = a is the set of outcomes ω for which X(ω) = a.

  In the above example with three coin flips, we have
    (X = 3) = {hhh}
    (X = 1) = {hht, hth, thh}
    (X = -1) = {htt, tht, tth}
    (X = -3) = {ttt}.

  Now since each X = a is an event, we can compute the probability
  Pr[X = a]. With the coin flips, we have
    Pr[X = 3] = 1/8
    Pr[X = 1] = 3/8
    Pr[X = -1] = 3/8
    Pr[X = -3] = 1/8
  This set of probabilities is called the "distribution" of the random
  variable X.

  We can also draw a graph to depict the distribution.
       3/8 |       *     *
           |       *     *
       1/8 | *     *     *     *
            -3 -2 -1  0  1  2  3  a

  Note that since X is a function from Ω to R, each ω has
  exactly one value a such that X(ω) = a, so ω is in exactly
  one of the events X = a. Thus, the events X = a partition the sample
  space. This means that
  (1) (X = a_1 ∩ X = a_2) = ∅ if a_1 ≠ a_2
  (2) ∪_{a ∈ A} (X = a) = Ω, where A is the set of all
      possible values that X(ω) can take on.
  These two facts imply that the sum of all probabilities in the
  distribution of X is 1.

  In the example of passing back exams, what is the distribution of X,
  the number of students who get their own exam back? For n = 3, we
    Pr[X = 3] = 1/6
    Pr[X = 1] = 1/2
    Pr[X = 0] = 1/3.
  What about arbitrary n? Let's come back to that later.

  Let's take another look at the coin flipping example, but for
  arbitrary n. The distribution of X, the amount of money won, seems
  non-trivial. But since we know that X(&omega) = 2 H(ω) - n,
  let's first compute a distribution for H, the number of heads.

  If we flip a fair coin n times, in how many outcomes are there
  exactly i heads? This is just choosing i out of the n flips to be
  heads, so there are C(n, i) outcomes. Then |H = i| = C(n, i), so
  Pr[H = i] = C(n, i) / 2^n. This is the distribution of H, where i is
  an integer 0 <= i <= n.

  Here is a graph representation of the distribution of H when n = 5:
           |       *  *
      9/32 |       *  *
           |       *  *
      7/32 |       *  *
           |       *  *
      5/32 |    *  *  *  *
           |    *  *  *  *
      3/32 |    *  *  *  *
           |    *  *  *  *
      1/32 | *  *  *  *  *  *
             0  1  2  3  4  5  a
  You can see the beginnings of a bell curve.

  It follows that the distribution of X is Pr[X = i] = Pr[2H-n = i] =
  Pr[H = (i+n)/2], where -n <= i <= n. If (i+n)/2 is not an integer
  (e.g. i i is odd and n is even), then this is 0.

  Now suppose we are flipping a biased coin with probability p of heads.
  Then what is the distribution of H, the number of heads?

  First, how many outcomes are in H = i? As before, there are C(n, i).
  But now we can't just divide by the size of the sample space, since
  it is not uniform. Instead, we use the definition of the probability
  of an event, that it is the sum of the probabilities of the outcomes
  in the event. What is the probability of each outcome in H = i? We
  already computed this in a previous lecture as p^i (1-p)^(n-i). So
    Pr[H = i] = C(n, i) p^i (1-p)^(n-i),
  where i is an integer 0 <= i <= n.

  This is known as the "binomial distribution" with parameters p and
  n, where p is the probability of getting heads in any one flip and n
  is the number of flips. We use the shorthand H ~ Bin(n, p) to denote
  that H is a random variable with a binomial distribution with
  parameters n and p.

  The graph of a binomial distribution with parameters p and n is
  bell-shaped, though it will be skewed in one direction if p is not
  1/2. See the reader for an example.

  The binomial distribution comes up in any experiment with n
  independent trials, each with probability of success p. As another
  example, suppose we are sending n packets over a network, where we
  choose the path from source to destination randomly and
  independently for each packet. Suppose that the probability that a
  single packet reaches its destination is p. Then if X is the number
  of packets that reach the destination, X ~ Bin(n, p), so Pr[X = i] =
  C(n, i) p^i (1-p)^(n-i) for 0 <= i <= n.