Stochastic systems analysis and simulation is divided in four blocks plus two introductory lectures. This page indexes the material prepared for different blocks
Introduction
This introductory block consists of a single lecture. It previews the class’s contents and exemplifies topics by studying a simple stochastic system. Material available for this block is the following:
- Slides for this introductory block. The latex fonts to generate these slides are available here.
- Speech is an example of a phenomenon for which stochastic modeling is fruitful. Speech waveforms for the words “Hi”, “Good” and “Bye”, as well as the fricative sound “S” are available in this file. To play these sounds use this Matlab code.
- Package from the MATLAB tutorial
Probability review
This block is a speedy review of Probability theory. We start reviewing the axiomatic definition of probability and introduce the concept of random variable. We then introduced commonly used distributions, the concept of expected value and joint distributions. We finish introducing different concepts of limit and limit theorems like the law of large numbers and the central limit theorem. We also devote a class to talk about conditional probability since this is something we will be using extensively in the rest of the class. Material available follows.
- First set of slides covering definition of probability, random variables, commonly used distributions, expected value and joint distributions. Covering this set of slides takes 3 lectures.
- Second set of slides covering Markov and Chebyshev’s inequalities, definitions of limits in probability, the law of large numbers and the central limit theorem. Two lectures are devoted to cover this slides
- Third set of slides revisiting the definition of conditional probability in the context of random variables and emphasizing its use in the computation of probabilities and expected values. These slides are for a single lecture.
Markov chains
- Slides containing first 7 lectures on Markov chains. The first two lectures cover definitions, notations and the introduction of Chapman-Kolmogorov equations for the evolution of probabilities. We discuss then two simple examples, gambler’s ruin and buffers in communication networks. We then follow with the introduction of the different classes of sates that may compose a Markov chain, and the concept of limiting distributions for irreducible aperiodic Markov chains. We also discuss ergodicity to some detail. The last class in this block expands the discussion of buffers in communication networks.
- Slides for two classes covering ranking of nodes in graphs.
Continuous time Markov chains
- Slides containing first 7 lectures on continuous time Markov chains. The first lecture covers the exponential distribution. We follow with two lectures on Poisson processes and three lectures on the basic definitions and theory of continuous time Markov chains (CTMC). We cover the transition probability function, methods to compute the transition probability function and the limit distribution of CTMCs.
- Slides containing 2 lectures discussing basic aspects of queuing theory.
- Slides containing 3 lectures on modeling and simulation of biochemical reactions.
Gaussian, Markov and Stationary random processes
- Gaussian processes. This set of slides defines Markov processes, Gaussian processes and stationary processes. It then studies Gaussian processes to some detail putting emphasis on Brownian motion and white noise. We also study biased Brownian motion and geometric Brownian motion to facilitate the future discussion on arbitrages, stock and option pricing.
- Arbitrages and pricing of stocks and options.
- Stationary processes.
- Review Session