Administrative info Updated MT2 solutions posted on Piazza Exams can be picked up from Soda front office PA3 due today HW8 due next Wednesday Final exam next Thursday No regrades for HW8 (not enough time) or final exam (UCB policy) Review In the continuous sample spaces we consider in this class, the probability of any particular outcome is 0. So instead of working with outcomes, we work directly with events. EX: James Bond jumps out of a plane and lands at a position uniformly distributed in [0, 1000]. The probability that he lands in an interval [a, b], where 0 <= a <= b <= 1000, is (b-a)/1000. Often, we work directly with random variables. A continuous random variable has a range that includes a continuous subset of R. Thus, for a random variable X, Pr[X = a] = 0 for any a. Again, we work with intervals, such as Pr[a < X <= b], which can have non-zero probability. We can describe a continuous random variable X in two ways. (1) The cumulative distribution function (cdf): F(x) = Pr[X <= x]. Then Pr[a < X <= b] = F(b) - F(a). The cdf F(x) must satisfy a few properties. (a) 0 <= F(x) <= 1 for all x∈R, since F(x) is a probability. (b) F(x) must be monotonically increasing: F(x) <= F(y) if x <= y. (2) The probability density function (pdf): f(x) = d/dx F(x). Then F(x) = ∫_{-∞}^y f(y) dy Pr[a < X <= b] = ∫_a^b f(y) dy. The pdf f(x) must satisfy a few properties. (a) f(x) >= 0 for all x∈R. Otherwise it would be possible to find an interval for which the integral is negative, resulting in a negative probability. (b) ∫_{-∞}^{+∞} f(x) dx = 1, i.e. the probability Pr[-∞ < X < +∞] is 1. The pdf tells us where there is a higher probability density. Its graph is similar to the distribution graph for a discrete random variable. EX: Let X be Bond's landing position when he jumps out of the plane. Then the cdf of X is { 0 if x < 0 G(x) = { x/1000 if 0 <= x <= 1 { 1 if x > 0 The pdf of X is { 0 if x < 0 g(x) = { 1/1000 if 0 <= x <= 1 { 0 if x > 0 A plot of the pdf: f(x) ____ 1/1000 _________ _________ 0 1000 x As you can see, he is likely likely to land anywhere in [0, 1000]. EX: James Bond shoots at a 1 foot radius gas tank, hitting any point on it with uniform probability. What is the pdf of the distance from the center to where he hits? Let Y = distance of hit from center. Then the cdf of Y is { 0 if y < 0 F(y) = { y^2 if 0 <= y <= 1 { 1 if y > 0 The pdf of Y is { 0 if y < 0 f(y) = { 2y if 0 <= y <= 1 { 0 if y > 0 A plot of the pdf: f(y) 2 / / 1 / _________/ _________ 0 1 y This shows us that there is higher density away from the center than closer to it. So Bond is more likely to hit further away from the center than closer to it. As you can see in the above example, the pdf is not restricted to the range [0, 1]. This is because the pdf is adensity, not an actual probability. We defined the pdf as f(y) = lim_{δ->0} Pr[y < Y <= y + δ] / δ, so it is the limit of a tiny probability divided by a tiny length, which can give us any non-negative value. We derived an expression for the expectation of a random variable: E(X) = ∫_{-∞}^{+∞} x f(x) dx. Then the variance is defined as in the discrete case, with E(X^2) = ∫_{-∞}^{+∞} x^2 f(x) dx. EX: What is E(X), Bond's expected landing position? It is E(X) = ∫_{-∞}^{+∞} x g(x) dx = ∫_{0}^{1000} x/1000 dx. = x^2/2000 |_0^{1000} = 1000^2/2000 = 1000/2 = 500. Then E(X^2) = ∫_{-∞}^{+∞} x^2 g(x) dx = ∫_{0}^{1000} x^2/1000 dx. = x^3/3000 |_0^{1000} = 1000^3/3000 = 1000^2/3. Then Var(X) = E(X^2) - E(X)^2 = 1000^2/3 - 1000^2/4 = 1000^2/12. In general, for a random variable Z that is uniformly distributed in the interval [0, d], we get E(Z) = d/2 Var(Z) = d^2/12. A random variable W that is uniformly distirbuted in the interval [a, a+d] is just W = Z + a, so we get E(W) = E(Z) + a = a + d/2 Var(W) = Var(Z) = d^2/12. Exponential Distribution Recall that if we have a number of independent trials, each of which has a probability p of success, then the number of trials T until the first success follows a geometric distribution T ~ Geom(p), so Pr[T = i] = p(1-p)^{i-1} for all i∈Z^+, and Pr[T > i] = (1-p)^i for all i∈N. Suppose now that we perform a large number of trials every second, where each trial has a small probability p of success, so that we perform a trial every δ seconds for some small δ. By linearity of expectation, the average rate of success λ per second is λ = p / δ, since there are 1/δ trials per second, each with probability of success p, so we have p = λ δ. Let S be the number of seconds until the first success. Then Pr[T > k] = Pr[S > kδ] (since each trial takes δ seconds) = Pr[S > t], where t = kδ and t >= 0 since k >= 0. Then Pr[S > t] = Pr[T > k] = (1 - p)^k (since T ~ Geom(p)) = (1 - p)^{t/δ} (since k = t / δ) = (1 - λδ)^{t/δ} (since p = λ δ) ≈ (e^{-λδ})^{t/δ} (since p = λ δ is small) = e^{-λ t}. Finally, we get, for t >= 0, Pr[S <= t] = 1 - Pr[S > t] = 1 - e^{-λ t} as the cdf of S, which gives us a pdf f(t) = d/dt (1 - e^{-λ t}) = λ e^{-λ t}. Both the pdf and cdf are 0 if t < 0. This is the "exponential distribution," which has pdf f(t) = { λ e^{-λ t} if t >= 0 { 0 if t < 0 It is the continuous version of the geometric distribution and tells us how long we need to wait for a success, if successes can occur at any time and λ is the average rate of success per unit time. We write S ~ Exp(λ). Computing the expectation and variance of an exponential random variable requires integration by parts, and we get E(S) = 1/λ Var(S) = 1/λ^2. These are similar to the geometric distribution, where we got an expectation of 1/p and a variance of (1-p)/p^2. Note that though p is restricted to [0, 1], since it is a probability, λ can be any non-negative value, since it is the average rate of success. In particular, it may be the case that we expect many successes in a unit of time, in which case λ will be greater than 1. Recall the relationship between the binomial and the geometric distribution. They both examine what happens when have a series of independent trials, each with probability p of success. The binomial distribution tells us how many successes we get in a fixed number of trials, while the geometric distribution tells us in which trial the first success occurs. The exponential distribution has a similar relationship to the Poisson distribution. They both examine what happens when we have a particular average rate of success λ. The Poisson tells us how many successes we get in a fixed unit length of time, and the exponential tells us at what time the first success occurs. EX: Suppose a web server processes on average 1.2 requests per second. Then the amount of time between requests follows an exponential distribution with λ = 1.2. Suppose a request comes in. What is the probability that a new request will come in within the next second? Let S be the amount of time until the next request. Then S ~ Exp(1.2). Then Pr[S <= 1] = ∫_0^1 1.2 e^{-1.2 t} dt = -e^{-1.2 t} |_0^1 = -e^{-1.2} + 1 ≈ 0.7. Note that we could use the Poisson distribution to solve this problem. Let R be the number of requests in the next second. Then R ~ Poiss(1.2). Then Pr[S <= 1] = Pr[R > 1] = 1 - Pr[R = 0] = 1 - 1.2^0/0! e^{-1.2} = 1 - e^{-1.2}, as we computed before. Normal Distribution A random variable X has a "normal distribution", also called a "Gaussian distribution,", if it has a pdf of the form f(x) = 1/√{2πσ^2} e^{-(x-μ)^2/(2σ^2)} for some values of μ and σ. It can then be shown that ∫_{-∞}^{+∞} f(x) dx = 1, as required for a pdf, and that E(X) = μ Var(X) = σ^2, hence the parameters μ and σ. We write X ~ N(μ, σ^2). The pdf of a normal distribution is a symmetric bell-shaped curve centered at μ, with a width determined by σ. The "standard normal distribution" has parameters μ = 0, σ = 1. So if Y is a standard normal, then Y ~ N(0, 1), and the cdf of Y is g(y) = 1/√{2π} e^{-y^2/2}. More on the normal distribution next time.