This page will be updated for Fall 2005.

STAT432 (STAT590A): Applied Probability Models
( Updated: Tue Nov 18 21:15:40 CST 2003 )

Information

News

• 2004-04-02: Homework solutions have been removed. Email me if interested.
• 2003-12-22: Final grades and final exam scores have been posted. You may obtain graded final exams or projects from me.
• Homework 9 solutions reposted:
  • Problem 1.5: There was one line incorrect on page 1. It is not true that P(W₂ > t+s | W₁ > t) = P((W₂ > t+s | W₂ > t), but it is true that P(W₂ > t+s | W₁ > t) = P(W₂ > s). A proof is provided, but you can take this as true. Also, there are cleaner proofs for this problem and they are acceptable.
  • Problem 1.9: I was counting the number of events incorrectly (I was counting pairs of events as 1, while they should count as 2). The corrected solutions are provided.
• Homework 9 solutions reposted with minor corrections to solution 2.4 (in the unnecessary and mostly unproductive simplification of the solution).
• Homework 7 solutions reposted with more detail on 3.9 and correction and more detail on 4.2.
• The List: the list of problems to be covered in the final exam.
• 2003-11-21: exam 2: solutions
• 2003-11-19: Some errors to report in notes.
  • Lecture 23 notes: In definition #2 for a nonhomogeneous Poisson Process, |A| should always be approaching 0 in the limit for the statements to be true. (Thanks M. Lawrence)
  • Lecture 24 notes: I defined p_e as the extinction probability, but then treated it as the survival probability, which would be 1-p_e. (Thanks F. Fang)
• 2003-11-18: It was brought to my attention that homework #9 had not been posted. Sorry, I've been unfairly sitting on it. It's up. More to come....is here.
• 2003-11-18: I am behind on posting materials that I promised; I will attempt to get the remaining material (homework #10; practice problems solutions) this evening.
• 2003-11-10: Homework 8 solutions posted. Since so many struggled with this homework, I hope you examine the solutions carefully. In one case at least, multiple solutions are provided.
• 2003-10-30: Homework help section added. BUT WE NEED MORE QUESTIONS! If you are struggling, you need to ask a question.
• 2003-10-29: Problem 6.3 retracted from homework #8. You can turn it in for extra credit.
• 2003-10-24: Homework 8 posted
• 2003-10-22: As announced in class, there will be no office hours tomorrow. Hongyue Dai's stat seminar is tomorrow at 4pm. You may be interested in attending that instead.
• 2003-10-20: Solutions to homework 6; lecture 22.
• 2003-10-16: Minor corrections to last two lectures (18 & 19) involving lecture titles. Content unchanged.
• 2003-10-08: Exam 1. Solutions posted.
• 2003-10-06: Solutions to homework 5 posted.
• 2003-09-29: A note of caution about the posted notes. The notes are designed to help you review your lecture notes and to allow you to spend less time taking notes and more time understanding the lectures. However, since I am preparing and writing up these notes for the first time, I fully expect there to be errors (and there are). Please use the book as your final, definitive source of correct and verified information.
• 2003-09-29: Corrections (thanks to Kent). A new version of lecture 2 (2003-08-27) has been posted.
  • Page 3: Two cases of intersection replaced with union for addition law for disjoint sets.
  • Page 3: Error in addition law for union of two non-disjoint sets.
  • Page 6: P(a<X<=x+dx) should be P(x<X<=x+dx).
  • Project is worth 10% not 30% as posted for a long time on this website.
• 2003-09-28: Corrections (thanks to Lixia). Please, let me know of other errors you find.
  • X_n and Y_n were swapped in writeup of lecture 12, page 3; Also in derivation of ogf for census of mature adults, use X₁ instead of X₀ since it is more natural to think of X_n as those immatures which matured to Y_n mature adults in generation n (rather than X_n-1 immatures maturing to Y_n matures) as was assumed in derivation of ogf for immature census.
  • Correct expression simplification error in writeup of solution for Homework 1 problem 2.5(b).
• 2003-09-27: Quick links to past lectures and homework assignments created above.
• 2003-09-27: Solutions to homework 2 re-posted. The new solutions provide a correct argument for why the wait time is not geometrically distributed.
• 2003-09-20: Solutions to homework 3 posted.
• 2003-09-20: Lecture 8 notes posted.
• 2003-09-18: Solutions to homework 1 have been posted.
• 2003-09-02: Solutions to last Friday's in-class questions have been posted.
• 2003-09-02: There was a typo on the assignments as posted here on this website until this am. The problems are from chapters 1 and 2 (not 1 and 3).
• 2003-08-29: Note, you are responsible for all sections in chapters 1 & 2 except those that are starred or footnoted, that is Chapter 1, section 2.8 and Chapter 2, sections 4 and 5.

Homework Help

• Homework 9
  • Problem 1.9: I'll make this problem a bonus problem worth up to 10 points. Because these are extra points, I'll provide limited help here.
    Hint 1: Consider the births in pairs and consider one pair of births as an event. For example, when the current state is 0, the first event happens when the chain reaches state 2. The wait time for that event is the sum of two random variables.
    Hint 2: In a Poisson process, the expected number of arrivals in time t is t divided by the average sojourn time 1/λ.
• Homework 8
  • Problem 5.3
  • Problem 5.5: Select a random point and put it at the center of a circle with radius r. Let D be the random distance to the closest point. What event is equivalent to {D > r}?
  • Problem 5.7
  • Problem 6.2: The problem asks you to show that the points at the end of the whole procedure are still Poisson distributed over the region. You must, therefore, choose one of the definitions of a Poisson process and show that the points still satisfy the postulates. Select some easy family of sets A and show the results for them. For example, you might let A_r,θ,φ be the wedge of points less than r from the origin and with angle between θ and φ.

    Another way to do this problem relies on an extension of the theorem in section 6. An outline is provided.

  • Problem 6.9: You must make the terms in the sum become independent before applying results from random sums. A major theorem for Poisson processes provides a standard way for making dependent W_i become iid random variables.
  • Problem 1.3: This problem poses a problem only in interpretation of its language. However you interpret the meaning of the parameter α, please clearly specify your definition of α in your answer.

Lectures

33. 2003-11-14 (Friday): Finite State Markov Chains
    • Homework 11: Practice Problems (not turned in) Chapter 6 5.1, 5.2
    • Main Concepts:
      • finite state markov chain
      • matrix exponential and solution of forward and backward equations
      • estimating matrix exponentials and transition probabilities
      • the general two-state CTMC
      • Cox process with underlying two-state CTMC
32. 2003-11-12 (Wednesday): Time Reversible CTMC
    • Main Concepts:
      • ergodic ctmc
      • relationships between DTMC and CTMC stationary distributions
      • reversed CTMC
      • time reversible CTMC
      • M neurons competing for resources
31. 2003-11-10 (Monday): Probability of Extinction
    • Main Concepts:
      • definition of absorbing state
      • definition of embedded DTMC
      • probability of ultimate absorption
      • expected time until absorption
30. 2003-11-07 (Friday): CTMC Stationary Distribution
    • Homework 10: Chapter 6 3.2, 3.3, 4.4, 4.5, 4.6 (solutions)
    • Main Concepts:
      • limit of transition probability function
      • stationary distribution definition and existence
      • balance equations and finding stationary distribution
      • stationary distribution of birth and death process
29. 2003-11-05 (Wednesday): Kolmogorov Backward and Forward Equations
    • Main Concepts:
      • simulating a birth and death process
      • Kolmogorov backward equations
      • general ctmc infinitesimal matrix generator
      • Kolmogorov forward equations
28. 2003-11-03 (Monday): Extinction Time in Death Process (and more)
    • Main Concepts:
      • time to extinction
      • sample problems
      • infinitesimal generator of birth and death process
      • Chapman-Kolmogorov equations
      • calculation of unconditional probabilities P(X(t)=i)
27. 2003-10-31 (Friday): Derivation of CTMC Differential Equations
    • Homework 9: Chapter 6 1.5, 1.8, 1.9, 1.10, 2.4, 2.6 (solutions)
    • Main Concepts:
      • definition of instantaneous transition rates
      • derivation of first order differential equations for Poisson process transition probabilities
      • pure death process
      • linear pure death process and binomial distribution
26. 2003-10-29 (Wednesday): CTMC and Differential Equations
    • Main Concepts:
      • transition probability function for pure birth process
      • linear pure birth process (Yule process) and negative binomial distribution
      • introduction to differential equations
      • solving differential equations for pure birth process
25. 2003-10-27 (Monday): Introduction to Continuous Time Markov Chain
    • Main Concepts:
      • definition of ctmc
      • ctmc and exponential distribution
      • birth and death process, yule process, linear growth process
      • transition probability function
      • transition probability function for pure birth process with distinct rates
24. 2003-10-24 (Friday): Poisson Process Wrap-Up
    • Homework 8: Chapter 5 5.3, 5.5, 5.7, 6.2, 6.9 (Problem 6.3 retracted); Chapter 6 1.2, 1.3 (solutions)
    • Main Concepts:
      • poissonization
      • marked poisson process
      • nonhomogeneous spatial poisson process
      • mapping theorem
23. 2003-10-22 (Wednesday): Nonhomogeneous and Compound Poisson Processes
    • Main Concepts:
      • nonhomogeneous poisson process
      • time inhomogeneous marking
      • compound poisson process: sums over poisson process arrival times
      • spatial poisson process: definition and uniform distribution of points
22. 2003-10-20 (Monday): Time-Dependent Marking
    • Main Concepts:
      • using uniform distribution of unordered arrivals
      • time-dependent marking
      • hiv prevalence example
      • shot noise: an example of deterministic, time-dependent marking
21. 2003-10-17 (Friday): Poisson Processes and Uniform Distribution
    • Homework 7: Chapter 5 3.6, 3.7, 3.9, 4.2, 4.8
      • Added part to 3.9: Find P[X₁(t)=n before X_2(t)=m].
      • And: You collect cards. There are m types of cards. Each time you get a new card it is of type j with probability p_j independent of all your previous cards (p_j's sum to 1). Use a simple marked PP to calculate the expected number of cards in your collecton when you first get all m cards.
      • solutions
    • Main Concepts:
      • simple marked poisson processes with more than 2 types (still independent poisson processes)
      • order statistics
      • arrival times as order statistics of iid uniform random variables
20. 2003-10-15 (Wednesday): Random Variables Associated with Poisson Processes
    • Main Concepts:
      • waiting times and sojourn times: definitions and distributions
      • definition 3 of poisson process
      • simple marked poisson processes lead to multiple, independent poisson processes
19. 2003-10-13 (Monday): Poisson Processes and Generalizations
    • Main Concepts:
      • two definitions of poisson process
      • little-o notation
      • nonhomogeneous poisson process
      • cox process
18. 2003-10-10 (Friday): Background for Poisson Process
    • Homework 6: Chapter 5 1.6, 1.9, 1.10, 1.11, 2.2, 2.7, 2.8 (solutions)
    • Main Concepts:
      • exponential distribution: memoryless property
      • poisson distribution: two theorems
      • counting process
      • poisson process
17. 2003-10-03 (Friday): Quick Presentations and Review
16. 2003-10-01 (Wednesday): Exercises in Modeling
15. 2003-09-29 (Monday): Markov Chain: Return Visits and Reversibility
    • Main Concepts:
      • stationary distribution
      • return time distribution and mean return time for recurrent states
      • reversed Markov chains
      • time-reversible Markov chains
14. 2003-09-26 (Friday): Markov Chain: Limiting Distribution
    • Homework 5: Chapter 4 1.5, 1.8, 1.13, 2.4, 3.3, 4.5, 4.7 (solutions)
    • Main Concepts:
      • limiting distribution: existence and calculation
      • ergodic, positive recurrent
      • long-run proportions
13. 2003-09-24 (Wednesday): Classification of States in State Space
    • Main Concepts:
      • communication of states and state space partitions
      • concept of irreducible Markov chains
      • transient and recurrent states
      • periodicity of states
12. 2003-09-22 (Monday): Discrete Time Branching Process III
    • Main Concepts:
      • probability of ultimate extinction
      • fundamental branching process theorem
      • modeling processes where reproduction occurs in sequential stages
      • modified branching processes
11. 2003-09-19 (Friday): Discrete Time Branching Process II
    • Homework 4: solutions
      • Chapter 3 problems 5.4, 8.3, 9.1, 9.5
      • What is the probability that an active cell passed through the latent cell in the HIV model described in lecture 9?
    • Main Concepts:
      • definition and main results for probability generating function (means, variances, product formula for sums of independent r.v., product formula for random sums)
      • probability generating function for branching processes: functional iteration
      • extinction probability calculation by generating functions
      • birth/death process
10. 2003-09-17 (Wednesday): Discrete Time Branching Process I
    • Main Concepts:
      • definition of branching process
      • fundamental branching process equation
      • means and variances of branching process
      • extinction probability by first-step analysis
9. 2003-09-15 (Monday): First-Step Analysis II and Viral Infection Example
   • Demonstration: Solving Virus Example Using R
   • Main Concepts:
     • summary of calculations for time to absorption, number of visits, and hitting probabilities
     • [review] matrix multiplication for matrices of different sizes
     • U=WR
     • example: model formulation and solving 3 questions
     • constructing markov chains from iid random variables
8. 2003-09-12 (Friday): First-Step Analysis I
   • Homework 3: solutions
     • Chapter 3, problems 3.6, 3.8, 4.12, 4.15, 4.17
     • Construct a Markov chain to find E[T] for the overlapping pattern ACGGAC in homework #2.
   • Main Concepts:
     • transient and absorbing states for finite DTMC
     • first-step analysis for chains with absorbing states
       • calculating absorption times
       • calculating hitting probabilities
7. 2003-09-10 (Wednesday): Markov Chain II
   • Main Concepts:
     • proof of Chapman-Kolmogorov
     • [review] matrix multiplication
     • matrix version of Chapman-Kolmogorov equation
6. 2003-09-08 (Monday): Markov Chain I
   • Main Concepts:
     • categorization of stochastic processes
     • definition of discrete time Markov chain
     • transition probability and transition matrix
     • stationary process
     • higher-order Markov chains
     • initial state distribution
     • a fully-defined Markov chain (initial distribution and transition matrix)
     • Chapman-Kolmogorov equation
5. 2003-09-05 (Friday): Simple Probability Models
   • Homework 2: handout | solutions
   • Main Concepts:
     • event indicator functions: E(1_A) = P(A)
     • independence and conditional probability: P(A|B) = P(A) if A and B are independent
     • practicing model formulation
4. 2003-09-03 (Wednesday): Probability Theory II
   • Main Concepts:
     • common discrete distributions: bernoulli, binomial, geometric, negative binomial, poisson
     • common continuous distributions: uniform, exponential, gamma, normal
     • expectation
     • >1 random variables
       • joint probabilities
       • functions of >1 r.v.
       • independence of r.v.'s
       • convolutions for sums of >1 independent r.v.
     • conditional distributions: definition, identities and tricks
3. 2003-08-29 (Friday): Work probability problems in groups.
   • Handouts: problems and solutions for group work
   • Homework 1: solutions
     • Chapter 1 Problems: 2.5, 2.7, 3.11, 3.12, 4.5, 5.7, 5.9
     • Chapter 2 Problems: 1.3, 1.8, 2.1, 3.2, 3.4
   • Main Concepts:
     • addition law for non-disjoint events
     • multiplication rule [ P(ABC) = P(A)P(B|A)P(C|AB) ]
     • practice with conditional probability, expectation, binomial r.v., modeling, ...
2. 2003-08-27 (Wednesday): Probability Theory I
   • Main Concepts:
     • sample space and events
     • mathematical definition of probability
     • conditional probability
     • independent events
     • random variable
1. 2003-08-25 (Monday): Introduction
   • Handouts: syllabus
   • Assignment: Read chapters 1 & 2.
   • Main Concepts:
     • mathematical model: definition and modeling process
     • stochastic vs. deterministic models
     • intuitive definition of probability
     • stochastic process, basic definition