Course webpage

Instructor

Parsiad Azimzadeh

Course description

This is a fairly rigorous introduction to probability theory with some emphasis given to both theory and applications, although a knowledge of measure theory is not assumed. Topics covered include probability spaces, conditional probability, independence, moment generating and characteristic functions, expectations, convergence of random variables, law of large numbers, central limit theorem, Markov chains, and Monte Carlo methods.

An archive of the course (including all lecture notes, assigments, solutions, TeX files, etc.) can be downloaded directly.

Textbook

(Optional) Walsh, John B. Knowing the odds: an introduction to probability. Vol. 139. American Mathematical Soc., 2012.

Lecture notes

• Th Jan 04: sample spaces, sigma-algebras, and probability measures (Section 1.1 and 1.2) [LaTeX source]
• Tu Jan 09: conditional probability, independence, counting, generating a sigma-algebra, and Borel sets (Section 1.4 to 1.7) [LaTeX source]
• Th Jan 11: random variables, distribution functions, indicator random variables, Borel measurable functions (Section 2.1) [LaTeX source]
• Tu Jan 16: uniform distribution, existence of random variables, independence of random variables, types of distributions (Section 2.2 to 2.4) [LaTeX source]
• Th Jan 18: expectations and variances of discrete random variables (Sections 2.5 and 2.6) [LaTeX source]
• Tu Jan 23: moments, moment generating functions, and special discrete distributions (Section 2.6 and 2.8) [LaTeX source]
• Th Jan 25: expectations of general random variables (Section 3.1) [LaTeX source]
• Tu Jan 30: moment inequalities and Jensen’s inequality (Section 3.3 and 3.4) [LaTeX source]
• Th Feb 01: convergence of random variables and the Borel-Cantelli lemma (Section 4.1) [LaTeX source]
• Tu Feb 06: limits of expectations (Section 4.2) [LaTeX source]
• Th Feb 08: law of large numbers, empirical distribution, Bernstein polynomials (Section 5.1) [LaTeX source]
• Tu Feb 13: moment generating functions and characteristic functions (rigorously) (Section 4.3 and 6.1) [LaTeX source]
• Th Feb 15: midterm exam 1 (in class)
• Tu Feb 20: convergence in distribution and Helly’s theorem (Section 6.2) [LaTeX source]
• Th Feb 22: tightness, integration, and Levy’s continuity theorem (Section 6.2 and 6.3) [LaTeX source]
• Tu Feb 27: no class
• Th Mar 01: no class
• Tu Mar 06: normal random variables, central limit theorems, and confidence intervals (Section 6.4) [LaTeX source]
• Th Mar 08: Markov chains (Section 7.2) [LaTeX source]
• Tu Mar 13: irreducibile matrices and the period of a nonnegative matrix (Section 7.3) [LaTeX source]
• Th Mar 15: filtrations and stopping times (Section 7.4) [LaTeX source]
• Tu Mar 20: midterm exam 2 (in class)
• Th Mar 22: strong Markov property, recurrence, and transience (Section 7.5 and 7.6) [LaTeX source]
• Tu Mar 27: limiting distribution, primitive matrices, and Page Rank (Section 7.7 and 7.8) [LaTeX source]
• Th Mar 29: lecture cancelled
• Tu Apr 03: Markov decision processes, dynamic programming [LaTeX source]
• Th Apr 05: Monotone, M, and weakly chained diagonally dominant matrices and the Bellman equation [LaTeX source]
• Tu Apr 10: joint distributions, covariances, and correlations (Section 3.6) [LaTeX source]
• Th Apr 12: conditional distributions, conditional expectations, and intro to Monte Carlo methods (Section 3.7) [LaTeX source]
• Tu Apr 17: accuracy and computational effort of Monte Carlo, quasi-Monte Carlo [LaTeX source]

Assignments

All assignments are to be handed in at the beginning of class.

• Assignment 1: due Tu Jan 16 [Description] [LaTeX source] [Solutions]
• Assignment 2: due Tu Jan 23 [Description] [LaTeX source] [Solutions]
• Assignment 3: due Tu Jan 30 [Description] [LaTeX source] [Solutions]
• Assignment 4: due Th Feb 08 [Description] [LaTeX source] [Solutions]
• Assignment 5: practice only [Description] [LaTeX source] [Solutions]
• Assignment 6: due Tu Mar 13 [Description] [LaTeX source] [Solutions]
• Assignment 7: practice only [Description] [LaTeX source] [Solutions]
• Assignment 8: due Tu Apr 03 [Description] [LaTeX source] [Solutions]
• Assignment 9: practice only [Description] [LaTeX source] [Solutions]

Miscellany

You are not expected to know any material in this section for assignments/exams.

• Can the principle of inclusion-exclusion be extended to countably many events? The answer seems to be yes, but with some technical conditions. See, e.g., Friedland, Shmuel, and Elliot Krop. “Exact conditions for countable inclusion-exclusion identity and extensions.”
• Why do probabilists take random variables to be Borel (and not Lebesgue) measurable?
• Is a measure for a sigma algebra determined by its values for a generator of the sigma algebra?
• A bound on the star discrepancy of the Van der Corput sequence can be found in Haber, Seymour. “On a sequence of points of interest for numerical quadrature.” J. Res. Nat. Bur. Standards Sect. B 70 (1966): 127-136.

Grading

The grading scheme is as follows:

• Assignments: 20%
• Midterms: 40%
• Final: 40%

Grades are posted on Canvas.

The lowest of your assignment scores will not count towards your final grade. Midterms will be held in class. The final is on April 23rd, 1:30 - 3:30 pm in 296 Weiser Hall. All exams are closed book.

The final exam covers all lectures except for lectures 21 and 22.

Alternate final times for students who cannot make the regular time are April 25: 12:00 pm - 2:00 pm and April 27: 12:00 pm - 2:00 pm. Both alternate exams will be held in 2866 East Hall.