MATH220

Description

A stochastic process is a collection of random variables indexed by a time parameter and used to model phenomena over time. Markov chains, the focus of this course, are one class of stochastic processes. They possess the Markov property which asserts that future random behavior of the system under study depends only on its current state, and not its past. Such processes find wide modeling applications in fields such as physics, chemistry, biology, business, information theory, and many others. Additionally, Markov chains are an effective tool in numerical estimation, for instance in solving multi-dimensional integrals that appear in statistical learning models. This course will consider both discrete- and continuous- time Markov chains. Topics will include Markov chains with finite and countable state spaces; continuous-time Markov chains such as the Poisson process, birth and death process, and queueing systems; Markov chain Monte-Carlo algorithms.