Stochastic processes are the mathematics of systems that evolve through time under uncertainty. Instead of one random variable, a stochastic process is a family of random variables indexed by time, space or another parameter.
Queues, financial prices, population counts, reliability states, network traffic, molecular motion and weather variables all change while remaining uncertain. Stochastic processes provide a language for that combination of dynamics and probability.
Series route: Mathematics Learning Hub → How Mathematics Works → Stochastic Processes.
1. What a stochastic process is
A stochastic process {X_t} assigns a random variable to each index t. For every outcome of the underlying probability space, the process produces a sample path through time.
The mathematics studies distributions of states, dependence across time and long-run behaviour.
2. State space and index set define the architecture
Time may be discrete or continuous. States may also be discrete or continuous. This creates four broad combinations: discrete-time chains, continuous-time jump processes, discrete-time continuous-state models and continuous-state diffusions.
3. Sample paths are realised histories
A process is a distribution over possible histories. One observed trajectory is only one sample path.
This distinction prevents a common error: inferring the entire process law from one visible path without sufficient evidence.
4. Markov processes compress memory
A Markov process satisfies a conditional independence property: given the present state, the future is independent of the past under the model.
This does not mean history is physically irrelevant. It means the current state has been defined richly enough to carry all predictive information the model retains.
5. Transition probabilities define Markov chains
For a discrete-state Markov chain, transition probabilities describe the chance of moving from state i to state j in one step.
These probabilities form a matrix whose rows sum to one.
6. Matrix powers predict multi-step behaviour
If P is the transition matrix, P^n contains n-step transition probabilities. Linear algebra therefore becomes a direct tool for stochastic dynamics.
7. Stationary distributions represent statistical equilibrium
A stationary distribution π satisfies πP=π. If the chain starts in π, its marginal distribution remains unchanged over time.
Under additional conditions, chains can converge toward a unique stationary distribution from many starting states.
8. Recurrence and transience classify return behaviour
A recurrent state is revisited with probability one under the relevant model; a transient state may be left forever.
This classification describes long-run network behaviour beyond one-step transitions.
9. Absorbing states trap the process
An absorbing state, once entered, cannot be left. Reliability failure, completion and bankruptcy can be modelled this way in simplified chains.
Absorbing-chain analysis computes probabilities and expected times to absorption.
10. Poisson processes model random event arrivals
A homogeneous Poisson process models counts of events arriving independently at a constant average rate under its assumptions.
Interarrival times are exponential, and counts over disjoint intervals are independent.
11. Memorylessness is a structural property
The exponential distribution is memoryless: conditional on survival to the present, the remaining waiting-time distribution has the same form.
This is mathematically elegant but must not be assumed for real waiting times without evidence.
12. Counting processes connect probability to operations
Customer arrivals, failures, messages and claims can be represented by counting processes. Queueing systems then combine random arrivals with random service.
13. Brownian motion models continuous random fluctuation
Brownian motion has continuous sample paths, independent increments and Gaussian increment distributions with variance proportional to elapsed time.
Its paths are continuous but almost surely nowhere classically differentiable.
14. Diffusion emerges from many tiny random moves
Scaled random walks can converge toward Brownian motion. This connects discrete stochastic models to continuous diffusion.
The bridge is one of probability’s most important limit structures.
15. Martingales formalise fair evolution
A martingale has the property that, conditional on current information, the expected future value equals the current value.
Martingales do not mean paths remain flat. They mean there is no predictable expected drift relative to the information filtration.
16. Filtrations represent information growth
A filtration is an increasing family of sigma-algebras describing information available over time.
Adapted processes cannot use future information at present times. This makes information timing explicit.
17. Stopping times depend only on current information
A stopping time is a random time whose occurrence can be determined using information available up to that time.
First-hit times and failure times are common examples.
18. Optional stopping requires conditions
Martingale stopping results can preserve expectations under suitable integrability and stopping assumptions.
Without those conditions, intuitive “fair game” arguments can fail.
19. Stationarity separates time origin from law
A stationary process has statistical properties invariant under time shifts in an appropriate sense.
Stationarity supports time-series modelling but is a model assumption, not a universal property of data.
20. Autocorrelation measures temporal dependence
Autocorrelation measures how values at different time lags co-vary. Strong autocorrelation means observations are not independent across time.
This affects forecasting, uncertainty estimates and effective sample size.
21. Stochastic differential equations combine noise and dynamics
An SDE adds random forcing to a differential law, often written schematically as dX_t=b(X_t,t)dt+σ(X_t,t)dW_t.
The dW term is not an ordinary derivative; stochastic calculus is required to define the integral consistently.
22. Ito calculus changes ordinary rules
Because Brownian increments have quadratic variation, the chain rule acquires an additional second-derivative term in Ito calculus.
Stochastic dynamics therefore cannot be handled by ordinary calculus alone.
23. A worked mechanism: two-state reliability chain
Suppose a device is either Working or Failed, and Failed is absorbing.
- Let probability of remaining Working each step be 0.99.
- Failure probability each step is 0.01.
- After n independent Markov steps, probability of still Working is 0.99^n.
- The model predicts a distribution of failure time, not one deterministic failure date.
24. Common failure modes
- One-path overconfidence: treating one realised history as the full process law.
- Markov assumption blindness: assuming present state contains all predictive information without testing or justification.
- Stationarity overreach: using stationary models when the data-generating mechanism changes.
- Independence drift: ignoring temporal dependence.
- Ordinary-calculus misuse: treating Brownian paths as differentiable.
25. Stochastic processes and probability
Probability provides the measure-theoretic machinery; stochastic processes add indexing and dependence through time or space.
26. Stochastic processes and statistics
Time-series analysis observes stochastic processes indirectly and estimates their parameters, dependence and predictive structure from finite data.
27. Stochastic processes in operations and finance
Queueing, inventory, reliability and financial models all use random evolution. The mathematics helps estimate waiting times, risk distributions and long-run behaviour.
28. Stochastic processes as a machine
State Space → Probability Law → Time Dependence → Transition/Increment Structure → Long-Run or Hitting Behaviour → Calibration and Decision.
29. What mastery looks like
- distinguish a process law from one sample path;
- classify discrete/continuous time and state spaces;
- work with transition matrices and stationary distributions;
- interpret recurrence, absorption and hitting times;
- understand Poisson processes and Brownian motion structurally;
- use filtrations, martingales and stopping times with information discipline;
- recognise dependence and stationarity assumptions;
- connect stochastic dynamics to real operational systems.
30. Conclusion
Stochastic processes work by placing probability inside time-evolving systems. Markov chains model state transitions. Poisson processes model event arrivals. Brownian motion models continuous random fluctuation. Martingales control conditional expectation. Stochastic differential equations combine dynamics with noise.
Probability describes uncertain outcomes. Stochastic processes describe uncertain histories.
How Mathematics Works | Batch 06
- How Mathematics Works | Functional Analysis
- How Mathematics Works | Partial Differential Equations
- Stochastic Processes — this article
- How Mathematics Works | Algebraic Geometry
Return to the Mathematics Learning Hub or the How Mathematics Works root.