Discrete Stochastic Processes

Discrete Stochastic Processes

Prof. Robert Gallager via MIT OpenCourseWare Direct link

7. Finite-state Markov Chains; The Matrix Approach

7 of 25

7 of 25

7. Finite-state Markov Chains; The Matrix Approach

Class Central Classrooms beta

YouTube videos curated by Class Central.

Classroom Contents

Discrete Stochastic Processes

Automatically move to the next video in the Classroom when playback concludes

  1. 1 1. Introduction and Probability Review
  2. 2 2. More Review; The Bernoulli Process
  3. 3 3. Law of Large Numbers, Convergence
  4. 4 4. Poisson (the Perfect Arrival Process)
  5. 5 5. Poisson Combining and Splitting
  6. 6 6. From Poisson to Markov
  7. 7 7. Finite-state Markov Chains; The Matrix Approach
  8. 8 8. Markov Eigenvalues and Eigenvectors
  9. 9 9. Markov Rewards and Dynamic Programming
  10. 10 10. Renewals and the Strong Law of Large Numbers
  11. 11 11. Renewals: Strong Law and Rewards
  12. 12 12. Renewal Rewards, Stopping Trials, and Wald's Inequality
  13. 13 13. Little, M/G/1, Ensemble Averages
  14. 14 14. Review
  15. 15 15. The Last Renewal
  16. 16 16. Renewals and Countable-state Markov
  17. 17 17. Countable-state Markov Chains
  18. 18 18. Countable-state Markov Chains and Processes
  19. 19 19. Countable-state Markov Processes
  20. 20 20. Markov Processes and Random Walks
  21. 21 21. Hypothesis Testing and Random Walks
  22. 22 22. Random Walks and Thresholds
  23. 23 23. Martingales (Plain, Sub, and Super)
  24. 24 24. Martingales: Stopping and Converging
  25. 25 25. Putting It All Together

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.