EECS 401 Spring 1997 Outline - Jeff Fessler

COURSE TEXT: Leon-Garcia, "Prob. and Random Processes for Engineers," 1994.

	Axiomatic Probability

Lecture#	(Sections from Leon-Garcia 2nd edition in parenthesis)

1 (2.1)
example: CD-ROM motivating 
2.1: Random experiment: trial, repeated trials, outcomes, sample space,
elementary events, compound events, examples

2 (1.3, 2.2, notes)
Sets: relationships, operations, algebra, partitions, De Morgan's Laws,
disjoint, properties relevant to probability,
cardinality, countable, uncountable (e.g. [0,1))
Relative Frequency
Axioms of probability
Equally likely probability law

3 (2.2 2.3)
Combinatorics
	- with and without replacement
	- ordered vs unordered

4 (2.3 2.2)
Properties of P
Joint Probability

5 (monday)
(2.4)
Conditional probability (given an event)
Sequential experiment
Law of total prob.

6 (2.4)
Bayes rule
examples: digital comm, urn
(2.5)
Independent events

7
(2.6)
Sequence of independent sub-experiments
Bernoulli trials
Binomial Probability Law
(3.1)
Random variables

8
Equivalent events
Discrete, continuous, mixed random variables
(3.2)
Cumulative distribution functions, properties

9
definition of cts vs. discrete r.v.s based on CDF
CDF of discrete r.v., step functions
probability mass function (pmf), histogram
Uniform distn
(3.3)
pdf definition, interpretation
uniform pdf

10 (monday)
pdf properties
pdf interpretation (dart players)
Dirac delta or unit impulse function, sifting property (not covered)
pdf of discrete RV (brief) 
Exponential distribution
example: vcr and P[3 < X < 4 | X > 3] (not covered)
example: system reliability
Scale and shift of r.v.: Y=aX+b
uniform pdf

11
(3.4)
Gaussian distribution, Q function
pmf of Bernoulli, Binomial, Poisson r.v.
(3.3)
Conditional cdf, pdf
"Total probability" for cdfs and pdfs

12
example: resistor tolerance
conditioning on a interval - see homework
conditioning on a point
example: binary comm 
total probability for point conditioning, continuous and discrete r.v.

13
(3.5)
cdf and pdf of functions of r.v.s
  - method of events (applicable to p.m.f. and p.d.f.)
  - method of differentials (p.d.f. only)
example: Y=X^2
example: half-wave rectifier (mixed r.v.) (not covered)

(friday was midterm 1)

(monday was memorial day)

14
more functions of r.v. (plug-and-chug formula)
linear xform of gaussian still gaussian
discrete r.v., multiple roots
(3.11)
computer generation of r.v. from uniform[0,1] by ``Transformation Method''
Computer generation of discrete r.v. (not covered)
example: triangular RV
(3.6)
Synopses of the properties of a r.v.
Mean, Average, Expectation
Age Example (for discrete r.v.)
Geometric r.v. example
E[X] for continuous r.v.
example: uniform r.v.
Symmetry Property
Gaussian

15
(3.6)
E[g(X)]
proof that 'easy way' and 'hard way' are equivalent for monotonic, cont. diff. g(x) (not covered)
discrete (dice) and continuous (gaussian) examples
scale and shift property, linearity
Conditional expectation, dice example
Var[X] = E[X^2] - (E[X])^2

16
(3.6)
Moments of r.v.
Moments about the origin
Central Moments
Effect of shift and scale on variance
Standard Deviation
Gaussian variance

17
(3.7)
Markov Inequality
Chebyshev Inequality
(3.9)
Characteristic Function and moment generating property
example: exponential moments
review of chapter 3, preview of chapter 4

18 (monday)
(4.1, 4.2, 4.5)
Vector r.v.s: n-tuple of functions on sample-space
equivalent events
PMF of discrete r.v.
joint CDF
dart board examples
Properties of JCDF
marginal CDF

19
(4.2, 4.5)
Joint pdf
Properties of jpdf
Marginal pdf
Computing P from jpdf
example: glass rod, 3 pieces, forms triangle (not covered)
example: P[R <= 20, pi/4 <= Theta <= pi/2] again
(4.3)
Independent r.v.s

20
(4.4)
Conditional JCDF, jpdf given an event
Total probability for JCDF, jpdf
Conditional JCDF and jpdf given an interval
Point Conditioning
Bayes rule for pdfs
"Total probability" for pdfs
chain rule	(not covered)
example: Packet arrivals at router: conditional prob. given r.v.

21
independence and conditional pdfs
conditional expectation of Y given X=x
independence and conditional expectation
law of iterated expectation E[X]=E[E[X|Y]]
example: mean number of packet arrivals
(4.6)
general pdf of g(X_1,...,X_n) using method of events
pdf of a function of X,Y
example: X+Y using method of events
independent X+Y: convolution of pdfs
convolution: associative, commutative
example: X+Y independent exponential yields Erlang

22
example: Y = max(X_1,X_2,X_3)
* (skip Jacobian approach)
(4.7)
E[g(X_1,...X_n)], continuous and discrete
mean of sum is sum of means
example: mean of binomial is np
conditional expectation given event, "total expectation"
linearity of E does not require independence: hat/virus example
independence and expectation of products

23 (monday)
correlation, orthogonality
covariance, correlation coefficient, uncorrelated r.v.s
properties of covariance
schwarz inequality (not proven)
Y = aX + b then corr. coef. is 1 or -1 (sign of a)
var(X+Y) = var(X) + var(Y) + 2 cov(X,Y)
(5.1)
mean and var of sum
example: var of binomial

24
(5.1, 5.2)
cdf of sum of indep. r.v.s via char. fun.
example: sum of gaussians is gaussian
example: sum of exponentials is Erlang (not covered)
i.i.d. r.v.s
sample mean: mean and var
(5.2, 5.3)
Laws of large numbers
   - sample averages as estimates of ensemble averages
   - weak law via Tchebychev
   - central limit theorem for i.i.d. case
   -  example: 4884 heads in 10000 tosses: fair coin?

25	(given by TA)
(4.8)
Bivariate Gaussian pdf
properties of Bivariate Gaussian
   - uncorr => indep
   - linear conditional expectation
   - Gaussian preserved by linear operations

thursday: exam
friday: no class

26 (monday)

(6.1, 6.2)
Random processes (indexed set of r.v.s)

27
example: perfectly predictable decaying sinusoid w/ random amplitude
	(mean function, variance function, correlation coefficient is 1)
Mean function
Variance function
deterministic r.p.
(6.2)
k-th order joint cdf, pdf, pmf
independent increments
Markov
Strict-Sense Stationarity
Wide-Sense Stationarity
Autocorrelation and autocovariance function and properties

28
(6.3, 6.5)
i.i.d. processes
Bernoulli process
Random walk: mean, variance, and autocovariance

(6.3)
Random walk
	1st- and 2nd-order pmf, generalizable to kth order
	independent increments
	stationary increments


29
Gambler' Ruin Process

(6.4)
Poisson process
	definition
	mean function, variance function
	P[multiple arrivals in time instant] -> 0
	McDonalds example for computing probability, CLT
	Autocovariance
	Interarrival times are expontential, Erlang arrivals
	Arrivals "at random" (uniform conditioning on N(t_0) = 1)

30
white noise
binary communications example
summarize ch. 4-6

----------------- end of spring term, below here not covered :-< ------------

(6.5)
Wide-sense stationarity: definition
s.s.s. => w.s.s., converse not true in general; exception: Gaussian

(6.5)
Properties of autocorrelation function for w.s.s.
	symmetric, max at origin, (skip: non-neg. definite),
	continuity at origin => continuous everywhere
Example: Sinusoid with random phase

Pairs of random processes
	joint cdfs
	joint strict-sense stationarity
	independence
	cross-correlation function and properties
	cross-covariance function (if independent then 0)
	joint wide-sense stationarity
	signal with random gain + noise example
	sum of jointly w.s.s. is w.s.s.
	product of independent and w.s.s. is w.s.s.