Markov Chain Calculator | Steady State, n-Step & Absorbing States
Analyze discrete-time Markov chains with up to 4 states. Enter the transition probability matrix and compute: the n-step transition matrix (matrix multiplication), the steady-state (stationary) distribution by power iteration, absorption probabilities for absorbing states, and the expected return time for each state.
State labels (optional)
Transition matrix P (rows must sum to 1)
| From \ To | Sunny | Rainy | Sum |
|---|---|---|---|
| Sunny | 1.000 | ||
| Rainy | 1.000 |
What Is the Markov Chain Calculator | Steady State, n-Step & Absorbing States?
A discrete-time Markov chain models a system that transitions between states according to fixed probabilities that depend only on the current state (the Markov property). The transition matrix P has Pᵢⱼ = P(next state = j | current state = i), so each row sums to 1. The n-step matrix Pⁿ gives the probability of being in each state after exactly n transitions. For an ergodic chain, Pⁿ converges to a matrix with identical rows — each row equal to the steady-state distribution π, which satisfies π = πP and Σπᵢ = 1. The expected return time to state i is 1/πᵢ steps. Absorbing states have Pᵢᵢ = 1 and are never left.
Formula
π = πP (steady-state) | Pⁿ = n-step matrix | E[Tᵢ] = 1/πᵢ (expected return time)
How to Use
- 1
Select the number of states: 2, 3, or 4
- 2
Enter optional state labels (e.g. Sunny, Rainy)
- 3
Fill in the transition probability matrix — entry (i, j) is P(go to j | in i)
- 4
Each row must sum to exactly 1.0 — a red warning shows if not
- 5
Set n for the n-step transition matrix computation
- 6
Click a preset to load a ready-made chain
- 7
Click "Analyze Markov Chain" — review steady-state π, state properties, and Pⁿ
Set number of states, enter labels, fill the transition matrix (rows must sum to 1), set n, then click Analyze.
Example Calculation
Weather: 2 states (Sunny=0, Rainy=1). P = [[0.7, 0.3],[0.4, 0.6]]. Steady-state: π = [4/7, 3/7] ≈ [0.571, 0.429]. Expected return to Sunny: 1/0.571 ≈ 1.75 days. P⁵ ≈ [[0.571, 0.429],[0.571, 0.429]] — both rows converge to π after 5 steps.
Understanding Markov Chain | Steady State, n-Step & Absorbing States
Chain Classification
| Property | Definition | Implication |
|---|---|---|
| Irreducible | Every state reachable from every other state | Single communicating class; unique steady-state π |
| Aperiodic | No state has a fixed return period (gcd = 1) | Pⁿ converges to the matrix with rows all equal to π |
| Ergodic | Irreducible + aperiodic | Time average = ensemble average; π exists and is unique |
| Absorbing state | Pᵢᵢ = 1 (self-loop probability = 1) | Chain eventually trapped; used in Gambler's Ruin |
| Absorbing chain | At least one absorbing state | From any transient state absorption is certain |
| Regular | Pⁿ has all positive entries for some n | Equivalent to ergodic for finite chains |
Real-World Applications
| Application | States | Transition | Output used |
|---|---|---|---|
| PageRank (Google) | Web pages | Links between pages | Steady-state π = page importance |
| Weather modelling | Sunny / Rainy | Historical daily weather | n-step probability of future weather |
| Credit risk | Credit ratings | Annual rating transitions | Default probability over k years |
| DNA sequence | A, C, G, T bases | Nucleotide substitution | Expected base composition |
| Queue analysis | Number in queue | Arrival/service rates | Steady-state queue length |
| Board games | Square positions | Dice roll probabilities | Expected turns to finish |
Key Theorems
- ›Perron-Frobenius theorem: every positive stochastic matrix has a unique stationary distribution π with all entries positive.
- ›Ergodic theorem: for an ergodic chain, the time average of any function converges to its expectation under π, independent of the starting state.
- ›Convergence rate: the rate at which Pⁿ → π is governed by the second-largest eigenvalue λ₂ of P — faster convergence when |λ₂| is small.
- ›Fundamental matrix: for absorbing chains, the fundamental matrix N = (I − Q)⁻¹ (where Q is the transient-to-transient sub-matrix) gives the expected number of visits to each transient state before absorption.
Frequently Asked Questions
What is a steady-state distribution?
The steady-state (stationary) distribution π is the long-run proportion of time spent in each state. It satisfies π = πP and sums to 1. For ergodic chains it is unique and independent of the starting state.
What is an absorbing state?
A state is absorbing if once entered it is never left — it has a self-loop probability of 1 (Pᵢᵢ = 1, Pᵢⱼ = 0 for j ≠ i). A chain with at least one absorbing state is an absorbing chain.
What does the n-step matrix tell me?
Entry (i, j) of Pⁿ is the probability of being in state j exactly n steps after starting in state i. As n → ∞, each row converges to the steady-state distribution for ergodic chains.
What is the expected return time?
For state i with steady-state probability πᵢ, the expected number of steps to return to i after leaving is 1/πᵢ. This is derived from the renewal theory interpretation of the ergodic theorem.
What is the Markov property?
The Markov property states that the future evolution of the chain depends only on the current state, not on the history of how that state was reached. This memorylessness makes the mathematics tractable and the model widely applicable.
You Might Also Like
Explore 360+ Free Calculators
From math and science to finance and everyday life — all free, no account needed.