A controller cannot correct what it cannot know well enough.
Many important system states are not measured directly. A controller may measure position but need velocity. It may measure terminal voltage but need an estimate of battery state of charge. It may observe several temperatures while trying to infer heat stored inside a process.
Observability asks whether the hidden internal state of a system can, in principle, be reconstructed from the outputs we can observe over time, given the known inputs and a model of the dynamics.
This is a specialist control concept beneath How Control Systems Work. It is related to How Measurement Works, but not identical. Measurement asks what we observe. Observability asks whether those observations are sufficient to reconstruct the state needed for control.
State Is Larger Than Output
A system can contain internal variables that determine future behaviour even though only some of them are exposed as outputs.
Consider a moving object. Measuring its position at one instant does not tell us its velocity. Yet velocity matters for where the object will be a moment later.
If we measure position repeatedly through time, the sequence can reveal velocity. With a suitable model, observed output becomes evidence about hidden state.
Observability is therefore about information carried by dynamics, not merely the number of sensors installed.
Direct Sensing Is Not Always Necessary
Some states are difficult, expensive or impossible to measure directly.
A system can instead infer them from other variables whose evolution depends on the hidden state.
- velocity can be estimated from position history;
- internal temperature can be estimated from surface temperatures and heat-flow models;
- battery state can be estimated from voltage, current and model history;
- vehicle orientation can be estimated from combinations of inertial and other sensors.
This inferred-state machinery is often implemented through an observer or state estimator.
An Observer Is a Model Running Beside the Real System
An observer predicts how the internal state should evolve using the model and known control inputs. It then compares predicted outputs with actual measurements.
The difference between prediction and observation is used to correct the estimated state.
The broad sequence is:
- estimate current internal state;
- predict how that state should evolve;
- predict what outputs should be observed;
- compare predicted and actual outputs;
- use the mismatch to update the hidden-state estimate;
- feed the improved estimate to the controller or diagnostic system.
This is feedback applied to knowledge of the state itself.
Not Every Hidden State Is Observable
If two different internal states always produce the same observable output under every relevant input, the measurements cannot distinguish them.
That hidden difference is unobservable from the available measurement structure.
The repair may require:
- another sensor;
- a different sensor location;
- a deliberately informative input or test manoeuvre;
- a better model;
- accepting that the state cannot be identified sufficiently for the intended controller.
Observability Depends on the Model
State inference is only as meaningful as the relationship connecting hidden state to measured output.
If the model omits an important dynamic, the observer can become confidently wrong. Sensor bias can also be mistaken for hidden-state change.
A mature estimator therefore needs model validation, noise assumptions and independent checks where consequences are serious.
Noise Does Not Destroy Observability, but It Changes Estimation Quality
Mathematical observability is often a structural question: can the state be reconstructed in principle?
Real measurement adds noise, drift and finite precision. A state can be theoretically observable and practically difficult to estimate accurately.
This is where filters and probabilistic estimators become important. They combine model prediction and uncertain measurements rather than treating either as perfect truth.
Observability and Monitoring Are Related but Different
In software operations, “observability” is often used broadly for logs, metrics and traces that help operators understand system behaviour.
Control-theoretic observability is more specific: whether the internal state can be reconstructed from available outputs.
The two ideas share a deep intuition — hidden state should leave enough evidence in observable channels — but they should not be flattened into one definition.
Worked Example: Position and Velocity
A cart moves along a track. Its internal state includes position and velocity. The sensor measures position only.
One position measurement cannot reveal velocity. A sequence of positions over time can.
If the dynamics are known well enough, an observer can estimate both states and let the controller use velocity information without a separate direct velocity sensor.
Worked Example: Battery State
Battery state of charge is not usually measured by looking directly at a visible “amount of electricity.”
Current history, terminal voltage, temperature and a battery model can be combined to estimate internal charge state.
The estimate improves when the model and measurements contain enough information to distinguish plausible states. It degrades when conditions make several states look similar at the outputs.
Worked Example: Railway State
A railway control system needs more than one sensor reading. Position, speed, route state, train identity and movement authority interact.
Some state is measured directly, some is computed from sequences and known infrastructure, and some is validated through multiple channels.
The important control question is whether the system knows enough about current state to make the next movement decision safely.
A Careful Analogy: Learning
A teacher cannot directly observe “understanding” as an internal cognitive state.
The teacher sees outputs: answers, explanations, errors, response times, transfer attempts and retrieval after delay.
The educational problem is analogous to observability: do these outputs contain enough evidence to infer the learner state relevant to the next instructional decision?
One correct answer may be insufficient because several hidden states can produce it: genuine understanding, memorised procedure, cueing, guessing or external support.
Changing the task can make the learner state more observable. Ask for explanation, delayed retrieval or transfer to a new representation. The canonical mechanism remains educational assessment and evidence, not literal state-space control, but the analogy is powerful.
A Careful Analogy: Institutions
An institution’s health is also partly hidden.
Headline service metrics may look normal while staff turnover, maintenance backlog, exception rates or manual workarounds reveal a weakening internal state.
The analogy asks whether the chosen metrics make the state that matters visible enough for governance. It does not imply that every social institution has a neat linear state model.
Sensor Placement Is an Observability Decision
Adding more sensors is not automatically better. Their location and relationship to hidden dynamics matter.
Two sensors measuring nearly the same thing may add little ability to distinguish internal states. One strategically placed measurement can sometimes reveal much more.
Observability therefore connects instrumentation to model structure: measure where hidden differences become visible.
Observability Can Be Lost During Failure
A sensor failure does more than remove one number. It may make an important state impossible to reconstruct reliably.
The controller may need to move into a degraded mode, rely on redundant sensing or reduce its operating envelope until state confidence returns.
This is why redundancy should be analysed for information quality, not just hardware count.
An Observability Diagnostic
- Define the internal state required for the decision.
- List which state variables are measured directly.
- List which must be inferred.
- Map how hidden states influence observable outputs over time.
- Ask whether two important hidden states can produce indistinguishable measurements.
- Test sensor noise and bias.
- Test whether the estimator converges after a wrong initial guess.
- Test sensor failures and degraded modes.
- Validate inferred state against independent evidence where possible.
The CivDJ Rotation
- Forward: hidden state → dynamics → measured outputs → observer → state estimate → control decision.
- Backward: start from a decision that must be trustworthy and ask what hidden state needs to be distinguishable upstream.
- Rotate: inspect the same measurement set as controller, maintainer, operator, auditor and receiver.
The operator may need a simple alarm. The maintainer may need diagnostic resolution. The controller may need a fast numerical state estimate. One measurement architecture may not serve all three jobs equally well.
The Civilisation Lesson
Large systems routinely act on states they cannot observe directly: risk, capacity margin, asset health, learner capability, institutional readiness and future demand.
The responsible question is not “Do we have data?” It is “Does the evidence we can observe actually distinguish the hidden states that would change our decision?”
Observability is the discipline of making hidden state inferable enough that control does not have to guess blindly.
Continue through How Measurement Works, How State Validation Works and the master How X Works hub. Next: controllability — whether the actions available can actually move the hidden and visible state where the system needs it to go.