About the Markov Chain Calculator
This Markov chain calculator works out where a process is likely to be after any number of steps and where it settles in the long run. Enter the transition matrix — one row per current state, with the probabilities of moving to each next state — plus a starting distribution and the number of steps. It returns the probability of being in each state after n steps, the full n-step transition matrix Pⁿ, the stationary (steady-state) distribution, mean return times and whether the chain is irreducible, periodic or has absorbing states.
Markov chains model systems that hop between states with fixed probabilities: weather from day to day, customers switching brands, credit ratings migrating, machines moving between working and broken, or board-game positions. Students use the calculator to check linear-algebra and probability homework; analysts use it for market-share forecasts and long-run equilibrium shares.
The model assumes the Markov property — the next state depends only on the current state, not the path that led to it — and that the transition probabilities do not change over time. Each row must add up to 1; if a row is slightly off (for example because of rounding), it is rescaled and you are told. Up to 10 states are supported.
How to use the markov chain calculator
- 1Type the transition matrix, one row per line, with the probabilities separated by spaces or commas.
- 2Optionally name the states in row order (one name per state).
- 3Enter the starting distribution (for example 1, 0, 0 to start in the first state).
- 4Set the number of steps and read the n-step distribution, Pⁿ and the steady state.
Formula and method
Row i of the transition matrix P gives the probabilities of moving from state i to every state in one step. Multiplying the starting row vector x₀ by P gives the distribution after one step, and multiplying by Pⁿ (computed by repeated squaring) gives the distribution after n steps. Entry (i, j) of Pⁿ is the probability of being in state j after n steps when starting in state i.
The steady-state (stationary) distribution π solves π·P = π with the probabilities summing to 1; the calculator solves this linear system directly. For an irreducible, aperiodic chain every row of Pⁿ converges to π whatever the start. For a periodic chain π still exists but describes the long-run fraction of time in each state rather than a limit. By Kac’s formula, the expected number of steps to return to state i is 1/πᵢ.
- P
- Transition matrix; Pᵢⱼ = probability of moving from state i to state j
- x₀
- Starting distribution (row vector)
- n
- Number of steps
- π
- Steady-state (stationary) distribution
Worked examples
Three-state weather model
Starting on a sunny day, the chance of sun three days later is 0.51 (cloudy 0.268, rainy 0.222). Solving π·P = π gives a long-run share of 21/46 ≈ 45.7% sunny days, 13/46 cloudy and 12/46 rainy, so a sunny day recurs on average every 46/21 ≈ 2.19 days.
Brand switching between two products
Each period 10% of Brand A buyers switch to B and 20% of B buyers switch to A. From a 50/50 split, A’s share is 0.5 × 0.9 + 0.5 × 0.2 = 0.55 after one period and 0.55 × 0.9 + 0.45 × 0.2 = 0.585 after two. In the long run A holds 0.2 ÷ (0.1 + 0.2) = 2/3 of the market.
A periodic chain
The chain flips between the two states every step, so after an odd number of steps it is certainly in state 2. A stationary distribution (0.5, 0.5) still exists — the chain spends half its time in each state — but Pⁿ never converges to it.
Frequently asked questions
What is a steady-state vector of a Markov chain?+
It is a probability vector π that does not change when multiplied by the transition matrix (π·P = π). For a regular chain, it is the long-run proportion of time spent in each state, whatever state you start in.
How do I calculate the n-step transition probabilities?+
Raise the transition matrix to the n-th power. Entry (i, j) of Pⁿ is the probability of moving from state i to state j in exactly n steps; multiplying a starting distribution by Pⁿ gives the distribution after n steps.
Why must each row of the transition matrix sum to 1?+
Row i lists the probabilities of every possible next state when the chain is in state i. Because the chain must go somewhere (possibly staying put), these probabilities have to total 1.
When does a Markov chain have a unique steady state?+
A finite chain has a unique stationary distribution when it has exactly one closed communicating class, which is guaranteed if it is irreducible. If it is also aperiodic, the n-step probabilities converge to that distribution.
What is an absorbing state?+
A state that, once entered, is never left — its row has a 1 on the diagonal. In an absorbing chain the process eventually ends up in an absorbing state, and the steady state puts all its probability there.