* Housekeeping Details - Lec: Mon/Wed 2:50-4:05 - OH: Dunno yet - HW: Approx 6 probs/wk; reviewed but not graded in detail - Txt: Comments welcome. - Sty: Read the book, do the problems, ask questions. My goal is not to spoon-feed the book, but rather to add perspective, illustrate and illuminate ideas, offer examples, and help show how the ideas and tools are useful in the theory and application of (especially Bayesian) statistics. 1. Sets and Events Motivation: Most students will have taken an undergraduate calculus-based course in probability theory (Duke's MTH135=STA104); such a course teaches about discrete and continuous random vbls and their distributions, joint distributions of 2 or 3 RV's, a little about conditional prob's and dist'ns. Most things are done twice: once for discrete rv's (binomial, geometric, poisson) and once for continuous (uniform, normal, exponential). This course builds a single coherent (beautiful) structure for one, two, or infinitely many random variables that are discrete or continuous or neither, and is especially concerned with limits of random variables (we will see there are many sorts of limits to consider) and with conditional distributions, when there may be many (even infinitely-many) others. A recurring theme is application within Bayesian statistics--- which we may view as simply probability theory on a grand scale, building a joint probability model for all the things we don't know (for example, the probability p of success in a clinical trial of an experimental drug) or haven't yet observed (for example, the number X of successes in the trial of N subjects); the object is usually to deduce more about the CONDITIONAL DISTRIBUTION of the things we care about, given the things we observed... like P[ p > 0.75 | X=8, N=10 ] Notation and Basic Mathematical Set-Up: \Omega: Set of possible outcomes of some "experiment" \omega: One of the outcomes in \Omega [Idea: nature or fate chooses an \omega from \Omega; alas she doesn't tell us about it] A, B, C: Subsets of \Omega; A is "true" if nature's \omega\in A. 2^\Omega: All subsets of \Omega ("Power set", sometimes denoted with a spikey P(\Omega)) P[ ]: Probability assignment of numbers 0<= P[A] <=1 to SOME (maybe not all) subsets A of \Omega... \cal{A}: Certain collections of sets. X,Y,Z: Random variables, functions X:\Omega -> E (maybe R or R^n) E[X]: Expectation of SOME (maybe not all) random variables X Big Ideas in Probability: LLN: If { X_i } are Indep Identically-Distributed RV's with same mean \mu = E[ X_i ], and partial sums S_n := \sum_{i<=n} X_i, then (1/n) S_n -> \mu [ what does it MEAN for a sequence RANDOM VARIABLES like Y_n = (1/n) S_n to "converge" to a constant \mu or to a random variable Y??? ] CLT: If { X_i } are IID with same mean \mu and finite variance \sig^2=E[(X_i-\mu)^2], and partial sums S_n := \sum_{i<=n} X_i, then Z_n = \sqrt {n} * (Y_n - \mu)/sig ==> No(0,1) [ what does it MEAN for a sequence of DISTRIBUTIONS to converge?? ] LIL: If { X_i } are IID with same mean \mu and finite variance \sig^2=E[(X_i-\mu)^2], and partial sums S_n := \sum_{i<=n} X_i, then \limsup_n (S_n - n\mu)/\sqrt{2 n \sigma^2 \log\log n} = 1 [ what is the "lim sup" of a sequence of random variables? ] MCT: If X_n = E[ X_{n+k} | X_1,...,X_n ] for every k>=0 and n, then UNDER SOME CONDITIONS (what conditions? why?), X_n -> X for some random variable X ( "->" in what way?) and, for SOME random times T (which ones? why?), E[ X_T | Info up to time n ] = X_n [ what does it MEAN to find expectation "given" some "info" ? ] ---------------------- Operations: Complement; A^c = "not A" = {w: w \notin A} Union over arbitrary index set; \cup A_\alpha = {w: w \in A_\alpha for at least one \alpha } A \cup B = "A or B (or perhaps both)" Intersection over arbitrary index set \cap A_\alpha = {w: w \in A_\alpha for all \alpha } A \cap B = A B = "both A and B" Set difference; symmetric difference; A /\ B = (A \cap B^c) \cup (A^c \cap B) -- = A\B u B\A = "exactly one of A, B occurs" Relations: containment: A \subset B : "A implies B" disjoint: A B = \emptyset equality; A = B: "A if-and-only-if B" De Morgan's Law: (\cup A_\alpha)^c = \cap (A_\alpha ^c) (\cap A_\alpha)^c = \cup (A_\alpha ^c) Indicator Functions: 1_A Lim Inf: union of intersections = EA = \cup_n \cap_{k>=n} A_k Lim Sup: intersection of unions = IO = \cap_n \cup_{k>=n} A_k Note (Lim Inf) \subset (Lim Sup); sometimes they coincide, but not always. Some examples, with \Omega = |N: A_n = {n,n+1,...} LimSup = LimInf = emptyset A_n = {1,2,...,n} LimSup = LimInf = |N A_{2n} = Evens, A_{2n+1} = Odds: LimSup = |N, LimInf = {} Example: a_n -> a <==> V eps>0 \exists NN -> |a_n-a| < eps {w: Xn(w) -> X(w) } ; set An = {w: |Xn(w)-X(w)| > 1/n }, consider LimInf An and LimSup An. Not every subset A of \omega will be an "event", but we will need to show that some are. Here are some rules: FIELD: i) \Omega in |A ii) A\in|A => A^c \in |A iii) A,B\in|A => AuB \in |A [==> FINITE unions] o'-FIELD (=\sigma-ALGEBRA) iii) Ai\in|A => Ui Ai \in |A Field not o'-Field: finite & co-finite sets (o')-Field generated by C: Borel Sets -------------- countable != infinite (present Cantor arg if time allows) ----- END OF FIRST PART --------- 1. Basic Definitions - Probability Space (_O_,F,P) ~ Prob prop'ties: 1) P(A)>=0; 2) o'-additive; 3) P(_O_)=1. ~ Inc/Exc rule; sub-additivity; continuity - Fatou's Lemma: P(Ai e.a.) <= lim-inf P(Ai) <= lim-sup P(Ai) <= P(Ai i.o.) - df: F(x)=F(x+); x F(x) <= F(y); F(-oo)=0, F(+oo)=1 P (a,b] := F(b) - F(a) 2. Dynkin's Theorem * Lambda System* * Pi System* l1: _O_ \in L l2: A \in L => A^c \in L p1: A,B\in L => A B \in L l3: Disjoint ctble unions Thm (ED): a) P a \pi-system in L a \lambda-system => o'(P) \subset L b) P a \pi-system => o'(P) = L(P) 3. Two Constructions i) Discrete: Countable \Omega, \sum{p_i} = 1, => B=2^\Omega okay ii) Continuous: Prob density iii) General 1-d: CDF 4. Constructions of Probability Spaces - Infinite Bernoulli sequence - Cantor distribution? - Big Thm 2.4.3: Ctbl-additive set function on a field F has ! extension to o'(F) 5. Measure Constructions 2 - Lebesgue measure on (0,1] (\lambda_2(dx)) 6. Counter-examples? E.g. Uniform on integers; Finitely-additive measures; ------------ Wed: Introduce: 1) { w: Xn(w) -> X(w) } = \cap{km} { w: |Xn(w) - X(w)| > 1/k } = \cap{k 1/k } Focus on: 1) Discrete probability spaces 2) Lebesgue measure 3) Continuous probability spaces 4) Cantor Example? Show: 1) Uniform distribution on rationals in [0,1] (what goes wrong?)