http://en.wikipedia.org/wiki/Markov_chain DTMC Markov chain a Markov model that models the state of a system with a random variable that changes through time. In this context, the Markov property suggests that the distribution for this variable depends only on the distribution of the previous state. The system is autonomous and its state is fully observable. discrete-time Markov chain