Stochastic systems analysis and simulation is divided in four blocks plus two introductory lectures. This page indexes the material prepared for different blocks


This introductory block consists of a single lecture. It previews the class’s contents and exemplifies topics by studying a simple stochastic system. Material available for this block is the following:

  • Slides for this introductory block. The latex fonts to generate these slides are available here.
  • Speech is an example of a phenomenon for which stochastic modeling is fruitful. Speech waveforms for the words “Hi”, “Good” and “Bye”, as well as the fricative sound “S” are available in this file. To play these sounds use this Matlab code.
  • Package from the MATLAB tutorial

Probability review

This block is a speedy review of Probability theory. We start reviewing the axiomatic definition of probability and introduce the concept of random variable. We then introduced commonly used distributions, the concept of expected value and joint distributions. We finish introducing different concepts of limit and limit theorems like the law of large numbers and the central limit theorem. We also devote a class to talk about conditional probability since this is something we will be using extensively in the rest of the class. Material available follows.

  • First set of slides covering definition of probability, random variables, commonly used distributions, expected value and joint distributions. Covering this set of slides takes 3 lectures.
  • Second set of slides covering Markov and Chebyshev’s inequalities, definitions of limits in probability, the law of large numbers and the central limit theorem. Two lectures are devoted to cover this slides
  • Third set of slides revisiting the definition of conditional probability in the context of random variables and emphasizing its use in the computation of probabilities and expected values. These slides are for a single lecture.

Markov chains

  • Slides containing first 7 lectures on Markov chains. The first two lectures cover definitions, notations and the introduction of Chapman-Kolmogorov equations for the evolution of probabilities. We discuss then two simple examples, gambler’s ruin and buffers in communication networks. We then follow with the introduction of the different classes of sates that may compose a Markov chain, and the concept of limiting distributions for irreducible aperiodic Markov chains. We also discuss ergodicity to some detail. The last class in this block expands the discussion of buffers in communication networks.
  • Slides for two classes covering ranking of nodes in graphs.

Continuous time Markov chains

Gaussian, Markov and Stationary random processes