Choose your reading route
eduKateSG · Why Mathematics? · Markov Reliability Models: State Transitions, Availability, Repair and Steady State
Start with the question, audit the worked example, or jump to the assumptions and practice. Every chapter stays open for uninterrupted reading.
Understand the question
Define the quantity and follow the mechanism.Route 2 · Chapters 9–16
Reproduce the example
Audit the numbers one step at a time.Route 3 · Chapters 17–24
Check assumptions
See where design and calibration matter.Route 4 · Chapters 25–32
Explore applications
Connect the method to a concrete investigation.Route 5 · Chapters 33–44
Study and verify
Practise, inspect failure cases and read further.
Full chapter index · Practice and checking · Mathematics Learning Hub
Why is mathematics important when systems can fail, recover and operate in degraded states? Markov reliability models represent those changes as transitions among clearly defined states. Matrices and exponential functions then calculate transient behaviour, steady availability, repair effects and expected performance.
Did You Know? A repairable system with failure rate 0.01 per hour and repair rate 0.09 per hour has long-run availability 0.09/(0.01+0.09)=0.90, even though the probability of no failure over a long mission keeps falling.
The model becomes useful when each state has an operational meaning. “Up” may mean full capacity, safe degraded service or merely the ability to respond, and those meanings lead to different rewards and decisions. Students should trace every arrow to a real failure, repair, switch or recovery event, check the time unit and ask whether the next transition depends on earlier history. These questions make matrix calculations honest and show when a richer state space, semi-Markov model or simulation is needed. They also keep transient mission questions separate from long-run maintenance planning. A clear state ledger lets another analyst verify that the matrix describes the same system and service promise.
Full chapter index
Chapters 1–8 · Understand the question
Chapters 9–16 · Reproduce the example
Chapters 17–24 · Check assumptions
Chapters 25–32 · Explore applications
Chapters 33–44 · Study and verify
Repairable and reconfigurable systems change among operating, degraded and failed states. A Markov model names those states and assigns transition rates or probabilities between them. It can calculate transient reliability, availability, expected occupancy and long-run behaviour. The value comes from making recovery and degradation explicit. A two-state model is not automatically realistic; state definitions and transition assumptions must match the system, maintenance and mission.
A discrete-time Markov chain advances in steps with a transition-probability matrix whose rows sum to one. A continuous-time Markov chain, or CTMC, uses rates per unit time in a generator matrix whose rows sum to zero. Reliability and availability work often uses CTMCs because failures and repairs can occur at arbitrary times. Do not mix a per-hour rate with a one-step probability without converting through a model.
The Markov property says the distribution of the next transition depends on the current state, not the full earlier path, given the model. In a homogeneous CTMC, transition rates are also constant over time, producing exponential state dwell times. Real ageing, repair learning or accumulated damage can violate this. Add states, use phase-type approximations, semi-Markov models or time-dependent rates when history matters.
Let a system be Up or Down. It fails from Up to Down at rate lambda and repairs from Down to Up at rate mu. This tiny model distinguishes failure and restoration, making it suitable for availability. It assumes one repair channel, constant rates and no degraded operation. The same state diagram can represent many systems, but lambda and mu must come from the population and conditions named in the claim.
With row-vector convention and state order Up, Down, the generator is Q=[[-lambda, lambda],[mu, -mu]]. Off-diagonal entries are transition rates. Each diagonal is the negative sum of outgoing rates, so every row sums to zero. For lambda=0.01/h and mu=0.09/h, Q=[[-0.01,0.01],[0.09,-0.09]]. A row that does not sum to zero signals a model-construction error.
For exponential time to failure, MTTF=1/lambda=100 hours. For exponential repair duration, MTTR=1/mu≈11.11 hours. These are model means, not scheduled times or guarantees. If repair includes diagnosis, logistics and restart, mu should reflect the full restoration process. A mixture of quick and slow repairs may not be exponential. Plot dwell times and check whether one constant rate is defensible.
| Quantity | Expression | Value |
|---|---|---|
| Failure rate lambda | given | 0.01/h |
| Repair rate mu | given | 0.09/h |
| MTTF | 1/lambda | 100 h |
| MTTR | 1/mu | 11.11 h |
| Steady availability | mu/(lambda+mu) | 0.90 |
At equilibrium, flow from Up to Down equals flow from Down to Up: pi_up lambda=pi_down mu, with probabilities summing to one. Thus pi_up=mu/(lambda+mu). For 0.01 and 0.09, availability is 0.90 and unavailability is 0.10. This is long-run point availability under the model. It is not the probability of no failure during a 100-hour mission.
If the system starts Up, A(t)=mu/(lambda+mu)+lambda/(lambda+mu)exp[−(lambda+mu)t]. At t=0 this equals one; as time grows, it approaches 0.90. Transient availability matters after startup, reset or a maintenance campaign. The matrix exponential p(t)=p(0)exp(Qt) generalises the calculation. State the initial condition; steady state may be inappropriate for a short mission.
Reliability for a mission often means no failure has occurred by time t. In the two-state model, if any failure ends mission success, R(t)=exp(−lambda t), regardless of repair. Availability permits failures followed by repair and asks whether the system is Up at t. A system can have modest reliability yet high availability when repair is quick. Mixing these metrics creates misleading claims.
For non-repairable mission reliability, Failed can be an absorbing state with no outgoing transition. The probability of remaining in working states gives reliability. Mean time to absorption follows linear equations or matrix methods. Repairable availability models keep return transitions. Decide whether mission failure can be recovered; the state diagram must represent the operational rule rather than a generic repair arrow.
A system with two redundant units may operate fully with two, degraded with one and fail with zero. States 2, 1 and 0 capture this. If each working unit fails at rate lambda independently, transition 2→1 has rate 2lambda and 1→0 has rate lambda. Repairs can add reverse transitions. The factor two counts two possible failures. Common-cause transitions from 2 directly to 0 must be added explicitly when relevant.
A Markov reward model assigns a value to each state. Up states can receive availability reward one and Down states zero. A degraded state might receive capacity 0.5, cost per hour or another outcome. Expected reward combines state probabilities with these values. NASA notes that upstates with reward one and down states zero yield availability-type measures. Rewards extend the model without changing transition logic, but their units and meaning must be stated.
Systems with several failed components may share one repair crew, so only one repair transition can occur at a time. Unlimited parallel repair assumptions can overstate availability. Add states for repair queue or crew assignment where important. Repair priority affects which state is reached next and can be optimised using rewards and costs. Field maintenance data should include waiting, parts and administrative delay, not only hands-on time.
A standby system changes state when an active unit fails, detection succeeds and a spare starts. States can distinguish active-plus-healthy-spare, active-plus-failed-spare, switched operation and total failure. Dormant failure, imperfect coverage and switching time require transitions. A static parallel formula often assumes away this sequence. Markov models make it explicit, at the cost of more states and parameters.
| State | Meaning | Availability reward |
|---|---|---|
| 2 | two units working | 1 |
| 1 | one unit working | 1 or degraded |
| 0 | no units working | 0 |
If two working units can fail together from a shared cause, include a direct transition from state 2 to state 0 with its own rate or condition. Modelling only independent single-unit transitions exaggerates redundancy. Separate common-cause and independent rates carefully to avoid double counting. Evidence may be sparse, so sensitivity ranges are often more honest than one precise rate.
A system with many components, modes and repair conditions can produce an enormous state space. Aggregate states only when they have equivalent future behaviour for the quantities of interest. Symmetry can collapse identities into counts of working units. Truncation, decomposition, simulation or specialised tools may be needed. A smaller transparent model can be better than a huge unvalidated one, but simplifications must be recorded.
Estimate rates from exposure time and transition counts under an appropriate model. For a constant failure rate, lambda may be failures divided by Up exposure; repair rate may be repairs divided by Down exposure. Censoring, covariates, fleet heterogeneity and repeated events complicate inference. Confidence intervals and Bayesian methods can express uncertainty. Do not infer an hourly rate from a catalogue probability without a stated conversion model.
A homogeneous CTMC implies exponential waiting times in each state. Plot survival of Up and Down durations, inspect hazard shape and stratify by environment or repair type. Increasing failure hazard suggests ageing; repair time may be lognormal or have a long logistics tail. Phase-type models can approximate non-exponential durations by adding hidden stages. The Markov property can be improved through better state definitions.
Rates may change with mission phase, temperature, load or calendar time. A non-homogeneous Markov model allows Q(t) to vary. Phased missions can use different generators and transfer state probabilities at phase boundaries. NASA research highlights limitations of simple homogeneous assumptions for phased missions. Record phase duration and state-restoration rules. Do not average rates across very different phases unless the resulting metric answers a clear question.
The matrix exponential solves continuous-time state probabilities. Uniformisation converts a CTMC into a Poisson-weighted discrete chain using a rate at least as large as all exit rates. Eigenvalue methods and numerical ODE solvers are alternatives. Whichever method is used, verify nonnegative probabilities, row sums and boundary cases. Numerical precision is secondary to correct states and rates.
Differentiate or perturb lambda, mu, coverage and common-cause rates to see which inputs control availability or reliability. In the two-state steady model, increasing mu and decreasing lambda both improve availability, but costs and feasible changes differ. A high sensitivity to an uncertain common-cause rate signals a need for evidence or design separation. Report ranges rather than one optimistic scenario.
A repairable online service can be Up, Degraded or Down. Incidents move it downward; mitigation and repair move it upward. Availability rewards may be one, partial and zero. Deployment phases can change rates. Shared dependencies create direct transitions to Down. Logs provide exposure and transition counts, but detection delays and unobserved degradation matter. The model complements, rather than replaces, process monitoring and incident analysis.
A two-pump system can occupy states with two, one or zero working pumps. Independent failures move 2→1 at rate 2lambda and 1→0 at lambda. One repair crew moves 0→1 and 1→2 at rate mu. A common power loss moves 2 or 1 directly to 0. If one pump supplies enough flow, states 2 and 1 are available; if not, state 1 receives a degraded reward.
Students can model a school printer as Ready, Printing, Jammed and Under Repair using illustrative transition probabilities. They calculate short-run state probabilities and steady availability, then question whether repair time is exponential. The goal is learning states and transitions, not judging people. A familiar low-stakes system helps students see why current state, history and repair definition matter.
NASA’s reliability resources define availability as the probability a repairable item performs its intended function when operated and maintained as prescribed. NASA technical reports describe Markov models for dependability, steady-state distributions, rewards and repair policies. Read the NASA reliability definitions, introduction to Markov modelling and Markov reward processes.
Check that the generator rows sum to zero. Calculate MTTF 100 h, MTTR 11.11 h, steady availability 0.90 and unavailability 0.10. Verify A(0)=1 and its limit 0.90. Use a numerical matrix exponential at several times and compare the closed form. Simulate many failure-repair histories and compare occupancy. Test lambda=0, mu→infinity and no-repair boundaries.
Markov models can become large, data-hungry and sensitive to state definitions. Constant rates and memorylessness may be wrong. Aggregation can hide history, load and common causes. Stationary availability may not describe short missions. Transition estimates from one fleet may not transfer. A mathematically valid generator cannot prove that important states or transitions are present. Use diagnostics, failure analysis and operational review.
Show the state diagram and define every state operationally. Report transition directions, rates, units, sources, uncertainty, repair resources, coverage, common causes and initial condition. Distinguish absorbing and repairable models, reliability and availability, transient and steady metrics. Describe the solver and validation. Present sensitivity and limitations. A reader should be able to rebuild Q and check each row.
Begin with Up and Down, write Q and calculate steady availability. Derive transient availability and compare with mission reliability. Add a degraded state, then model two redundant units with combinatorial rates. Add one repair crew, imperfect coverage and a common-cause transition. Simulate the chain and compare with matrix results. Finish by checking dwell-time assumptions. This links algebra, exponentials, matrices, probability and system design.
Markov reliability models turn repair and reconfiguration into a living mathematical system. Students see matrices describe motion through states, while engineers can compare immediate, long-run and degraded performance. The method is most useful when states are concrete and assumptions are tested. Mathematics then supports better maintenance and architecture choices without pretending that every future transition is memoryless or known.
Rehearse the model with a state ledger before trusting the matrix. For every state, record which components work, what function is delivered, whether repair is active and which next transitions are physically possible. Build Q from that ledger and verify rate units, nonnegative off-diagonals and zero row sums. Solve the two-state subset by hand, then compare the larger model with a matrix exponential and event simulation. Check initial probabilities, steady limits and rewards. Plot observed dwell-time survival for each state and challenge the exponential assumption. Add one common-cause transition and one repair-crew constraint separately so their effects are visible. Perturb rates across evidence-based ranges and compare short-mission reliability with long-run availability. If aggregation combines states with different future behaviour, split them and test whether the result changes materially. This rehearsal links the abstract matrix to operations and exposes unsupported transitions before they become precise-looking availability estimates.
A start-up mission, a daily service window and a mature fleet can produce different answers from the same generator. Transient probabilities retain the initial condition; steady state forgets it. If a system starts after preventive maintenance, assuming equilibrium may overstate early failure or understate early availability depending on the process. Report results at decision times and show convergence rather than quoting only a stationary value. For phased operations, carry the state distribution between phase-specific generators. Time horizon is part of the question, not an output formatting choice.
List states in rows and columns and build a generator matrix with off-diagonal rates. Calculate each diagonal as the negative outgoing sum and add a row-sum check. Store initial probabilities separately. For a two-state model, calculate MTTF, MTTR, transient and steady availability with labelled units. For larger models, import a validated matrix-exponential result rather than constructing fragile formulas. Add a reward vector, sensitivity scenarios and probability-sum checks at every time point.
Represent states and transitions in a table, then construct Q programmatically. Assert nonnegative off-diagonal rates and zero row sums. Compute p(t)=p(0)exp(Qt), steady distribution and expected rewards using tested numerical libraries. Verify against the two-state closed form and stochastic simulation. Preserve state ordering, units, initial condition and solver tolerances. Estimate rates from exposure with intervals, simulate parameter uncertainty and export diagrams, probability trajectories and sensitivity results.
Markov reliability models support aerospace, power, telecommunications, maintenance, cloud services, transport, medical devices and fault-tolerant computing. Careers combine linear algebra, probability, operations research, engineering, data and domain knowledge. A state model does not certify a system or guarantee employment. Students can build foundations through matrices, exponentials, coding and systems thinking, then explore reliability, operations and engineering roles while keeping pathways open.
Simulate an Up/Down repairable system with lambda=0.01/h and mu=0.09/h. Estimate long-run Up fraction and compare it with 0.90. Plot transient availability from an Up start. Replace exponential repair with a fixed or lognormal duration and observe how the CTMC approximation changes. Add a Degraded state and reward 0.5. Explain which differences come from simulation error and which from model assumptions.
Reliability and availability are not synonyms. A rate is not a one-step probability. Generator rows sum to zero, not one. Steady state does not describe every short mission. MTTF plus MTTR alone may not capture queues or degraded states. A Markov model does not prove memorylessness. Adding states can improve realism but also create unsupported parameters. Independent unit failures need combinatorial rate multipliers; common causes need separate transitions.
The reliability block diagram article provides static functional architecture. The fault tree article checks undesired-event logic. The Weibull reliability article models non-exponential lifetimes and censoring. Markov models add repair, state changes and availability.
Students should draw the state diagram before writing a matrix. Parents can ask what each arrow means, which unit the rate uses and why steady state is relevant. Encourage boundary checks such as zero failure or instant repair. Use simulated or low-stakes systems and avoid labelling people as failed states. The deepest transferable habit is checking whether the current state contains enough information for the next transition model to be credible.
1. Build Q for the two-state model. 2. Reproduce MTTF, MTTR and 0.90 availability. 3. Verify transient boundary values. 4. Distinguish mission reliability from availability. 5. Add one degraded state and reward. 6. Derive 2lambda for the first failure among two units. 7. Add common-cause and imperfect-coverage transitions. 8. Identify evidence against exponential dwell times. 9. Write a sensitivity conclusion.
### Why do generator rows sum to zero? The diagonal balances the rates leaving each state.
### Is steady availability the chance of no failure? No. It permits failure and repair.
### What if repair times are not exponential? Use richer states, semi-Markov models or simulation.
### Why add rewards? They translate state occupancy into availability, capacity, cost or another defined outcome.
Report system function; state definitions; transition diagram; generator convention; rates, units, sources and intervals; initial condition; absorbing and repairable states; repair resources and queues; coverage and common causes; memoryless and homogeneity assumptions; transient times; steady distribution; reward definitions; reliability and availability outputs; solver; simulation checks; sensitivity; validation; software and version; and limitations.
Rebuild the state list and every transition from operational evidence. Check Q off-diagonals, diagonals, units and row sums. Compare closed-form and numerical two-state results. Verify initial and limiting probabilities and that all probabilities remain nonnegative and sum to one. Audit combinatorial rate factors, shared repair crews, coverage and common causes. Plot dwell-time data against exponential assumptions. Challenge state aggregation and run parameter sensitivity before accepting a steady-state headline.
Continue through the Mathematics Learning Hub. Explore NASA’s Markov model for reliability growth and decay and phased-mission reliability work for extensions. Next implement the two-state model, verify it by simulation and then add only one new state at a time, with evidence for every new transition.
Extended practice laboratory
Practice case 1: Build two states
Setup: Use Up and Down states.
Task: Draw failure rate lambda from Up to Down and repair rate mu back.
Checking target: Verify each transition has units per time.
Practice case 2: Write the generator
Setup: Use lambda 0.01/h and mu 0.09/h.
Task: Write Q=[[-0.01,0.01],[0.09,-0.09]].
Checking target: Each row must sum to zero.
Practice case 3: Compute steady availability
Setup: Use mu/(lambda+mu).
Task: Calculate 0.09/0.10.
Checking target: Obtain 0.90 under constant-rate assumptions.
Practice case 4: Compute steady unavailability
Setup: Use lambda/(lambda+mu).
Task: Calculate 0.01/0.10.
Checking target: Obtain 0.10 and confirm availability plus unavailability equals one.
Practice case 5: Compute transient availability
Setup: Start Up at time zero.
Task: Use A(t)=mu/(lambda+mu)+lambda/(lambda+mu)exp[−(lambda+mu)t].
Checking target: Check A(0)=1 and convergence to 0.90.
Practice case 6: Find mean times
Setup: Convert rates to MTTF and MTTR.
Task: Calculate 1/lambda and 1/mu.
Checking target: Obtain 100 h and 11.11 h.
Practice case 7: Add degraded state
Setup: Use Full, Degraded and Down.
Task: Define allowed transitions and working-state rewards.
Checking target: Do not hide degraded capacity inside one Up state.
Practice case 8: Check Markov assumptions
Setup: Ask whether next transitions depend only on current state with constant rates.
Task: Inspect dwell times and history effects.
Checking target: Use semi-Markov or phase-type refinements when needed.
Practice case 9: Model redundancy
Setup: Create states for two, one and zero working units.
Task: Add failure, repair and common-cause transitions.
Checking target: Check combinatorial rate multipliers.
Practice case 10: Report the analysis
Setup: State states, rates, units and initial condition.
Task: Report transient and steady metrics with sensitivity.
Checking target: Avoid confusing reliability with availability.