Find your next learning step
A plant controller sees temperatures, pressures and alarms, not the hidden degradation state itself. This guide turns noisy evidence into a belief state and asks when a sensor is worth its cost because it changes the operating decision.
Open the chapter contents · Thermodynamics root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the question, or read from Chapter 1 for the complete argument. The teaching chapters remain open; these grouped routes form the index.
Route 1 · See the hidden-state problem
Route 2 · Build the switching rule
Route 3 · Value the information
Route 4 · Test the policy
Route 5 · Check and transfer
CHAPTER 1 OF 12 · SEE THE HIDDEN-STATE PROBLEM
1. A controller sees evidence, not the hidden plant state
An energy system may be healthy, fouled, degraded or near failure, but the controller observes temperatures, pressures, vibration and alarms. These measurements are noisy and incomplete. Treating an alarm as the state itself throws away uncertainty; ignoring it throws away information.
Partial observability begins by separating the hidden state from the observation. A belief state is a probability distribution over possible plant states given the previous belief, the action taken and the new observation. It is the compact object used to choose among operating modes when the true condition is not directly visible.
CHAPTER 2 OF 12 · SEE THE HIDDEN-STATE PROBLEM
2. The POMDP pieces in thermodynamic language
A partially observable Markov decision process specifies hidden states, available actions, transition probabilities, observation probabilities, rewards or costs and a time horizon. In an exergoeconomic application, costs can include fuel, exergy destruction, product-cost penalties, wear, emissions where authorised, sensor expense and switching costs.
The transition model says how actions change the chance of degradation or recovery. The observation model says how likely each sensor reading is under each hidden state. Bayes’ rule updates the belief after a reading. Dynamic programming then compares current operating cost with future consequences on the continuous belief space.
The classic formulation by Kaelbling, Littman and Cassandra explains why the belief state can support planning under partial observation. It does not imply that every industrial model is exactly known or computationally easy.
CHAPTER 3 OF 12 · SEE THE HIDDEN-STATE PROBLEM
3. Worked alarm update
Consider two hidden states: healthy H and degraded D. Before a reading, the probability of D is 0.20. A sensor alarms with probability 0.80 when degraded and 0.10 when healthy. After an alarm, Bayes’ numerator is 0.20 times 0.80 equals 0.16; the total alarm probability is 0.16 plus 0.80 times 0.10 equals 0.24, so the posterior degradation belief is 0.667.
| Observation | Degraded-path probability | Healthy-path probability | Posterior probability of degradation | Reading |
|---|---|---|---|---|
| Alarm | 0.20 × 0.80 = 0.160 | 0.80 × 0.10 = 0.080 | 0.160 / 0.240 = 0.667 | Strong upward update |
| No alarm | 0.20 × 0.20 = 0.040 | 0.80 × 0.90 = 0.720 | 0.040 / 0.760 = 0.053 | Strong downward update |
| No reading | Not available | Not available | Remains at prediction step | Sensor outage is information too |
The posterior is not a diagnosis of certainty. It is the probability carried into the operating decision and the next transition.
CHAPTER 4 OF 12 · BUILD THE SWITCHING RULE
4. Belief-state switching thresholds
Suppose normal mode is cheaper when the plant is healthy, while protective mode reduces degradation loss at a production cost. In a one-step teaching rule, switch to protective mode when the degradation belief exceeds 0.40. The alarm posterior of 0.667 triggers protection; the no-alarm posterior of 0.053 does not.
The threshold comes from costs, not from the sensor alone. If the consequence of degraded normal operation rises, the threshold falls. If protective operation is very costly or creates its own wear, the threshold rises. In a multi-period model, future state transitions, information gained while operating and switching costs can make the policy curved or history-dependent through belief.
CHAPTER 5 OF 12 · BUILD THE SWITCHING RULE
5. Add exergoeconomic costs without double counting
Exergoeconomics combines exergy analysis with economic costing to locate and value inefficiencies in an energy-conversion system. The foundational methodology by Tsatsaronis and Winhold supports allocating costs to exergy streams and component irreversibilities.
For a partially observed controller, define each mode’s cost consistently. Fuel expenditure and the cost consequence of exergy destruction may overlap if accounting boundaries are mixed. Maintenance and lost production should be added on the same time basis. A sensor is valuable only through decisions it changes or uncertainty it reduces for a future decision; its price is not an exergetic loss.
The exergoeconomic switching-portfolio guide develops multiple operating modes and loop costs under observable assumptions. Partial observability replaces the known state with a belief over states.
CHAPTER 6 OF 12 · VALUE THE INFORMATION
6. The value of a sensor is decision-dependent
Value of information compares expected cost with and without an observation, using the best action in each case. A technically accurate sensor may have little value if every plausible reading leads to the same mode. A modest sensor can be valuable near a decision boundary because its readings split the action.
For the teaching rule above, the prior belief 0.20 calls for normal mode. The alarm moves belief above 0.40 and changes the action; a no-alarm reading confirms normal operation. To value the sensor, average the optimised post-reading costs over alarm and no-alarm probabilities, subtract the best no-sensor cost, then subtract purchase, calibration, latency and failure costs.
This connects to the exergoeconomic value-of-information guide. The new element is sequential belief updating rather than one static uncertain input.
CHAPTER 7 OF 12 · VALUE THE INFORMATION
7. Changed context: the same alarm can lead to another action
Keep the sensor sensitivity at 0.80, false-alarm probability at 0.10 and switching threshold at 0.40. If preventive maintenance lowers the prior degradation probability from 0.20 to 0.05, an alarm gives posterior 0.040 divided by 0.040 plus 0.095, or about 0.296. The same alarm no longer crosses the threshold.
| Prior degradation belief | Posterior after alarm | Posterior after no alarm | Action after alarm at threshold 0.40 |
|---|---|---|---|
| 0.05 | 0.296 | 0.012 | Stay normal; inspect or gather more evidence |
| 0.20 | 0.667 | 0.053 | Switch to protective mode |
| 0.50 | 0.889 | 0.182 | Switch to protective mode |
| Unknown prior | Not calibrated | Not calibrated | Do not treat alarm as a universal probability |
The transfer check is to ask whether the observation model and prior belong to the current regime. A policy calibrated before maintenance, a fuel change or a sensor replacement may no longer have the same posterior thresholds.
CHAPTER 8 OF 12 · VALUE THE INFORMATION
8. Transitions, observations and timing
Belief updating usually has a prediction step and a correction step. The action first changes the distribution of the next hidden state through the transition model. The new observation then changes the predicted belief through the observation likelihood. Reversing the timing without documenting it can produce a different policy.
Sensor delays also matter. A reading that describes the state ten minutes ago must be propagated through possible transitions before the current action. Mode changes can alter sensor behaviour, so observation probabilities may depend on the action. Model outages and ambiguous readings explicitly rather than dropping those intervals as if nothing happened.
CHAPTER 9 OF 12 · TEST THE POLICY
9. Policy shape, switching costs and hysteresis
With two states and simple costs, a threshold policy may emerge. With several modes, non-linear degradation, maintenance opportunities and costly information, the optimal action regions can be non-convex. Monahan’s partially observable stopping model with costly information is an early warning that information-purchase regions need not have an intuitive control-limit shape.
Switching costs can create hysteresis: the belief required to enter protective mode may differ from the belief required to leave it. Minimum dwell times and transition feasibility also matter. First prune physically impossible or economically dominated moves using the mode-graph guide, then solve or approximate the belief-state problem on the remaining graph.
CHAPTER 10 OF 12 · TEST THE POLICY
10. Validation, limits and evidence checks
Transition and observation probabilities are estimates. Validate sensor sensitivity and false-alarm rates by regime, compare predicted belief frequencies with inspected states, and backtest action consequences. Use simulation to test rare degradation paths and sensor outages, but distinguish simulated evidence from plant observations.
POMDP solutions can be computationally demanding, and approximate solvers may hide policy error. Model misspecification can be more important than optimisation tolerance. A low expected cost under one fitted model is not a safety certificate. Keep engineering limits, interlocks and human escalation rules outside the optimiser where required.
Common misreadings are to equate a belief with a physical measurement, assume one alarm has a fixed posterior meaning, value a sensor by accuracy alone, ignore transition timing, or report a switching threshold without its cost assumptions.
CHAPTER 11 OF 12 · CHECK AND TRANSFER
11. Questions readers usually ask
Is the belief state just a confidence score?
No. It is a probability distribution tied to an explicit transition and observation model. A free-form score may still guide action, but it is not automatically a POMDP belief.
Does a better sensor always have more value?
Not necessarily. It must change a decision enough to outweigh its cost, delay, maintenance and failure modes.
Can operators override the policy?
Yes. Human override, inspection and escalation can be modelled as actions or retained as governance constraints. Their costs and information should not be silently omitted.
Must the exact optimal POMDP be solved?
No. A transparent approximate policy can be preferable when its error and safety envelope are tested. The approximation should be compared with simpler baselines and stress scenarios.
CHAPTER 12 OF 12 · CHECK AND TRANSFER
12. A transfer routine and connected reading
Name the hidden states, actions and observations; estimate transition and observation models by regime; build an exergoeconomic cost ledger without overlap; update beliefs with correct timing; and compare policies on expected cost, exergy performance, switching frequency and safety constraints. Value a sensor by the optimised decisions it changes, then stress the result under prior drift and sensor degradation.
Continue with exergoeconomic expected-regret curves, Monte Carlo analysis with correlated inputs and mothballing with restart costs and hysteresis. The general lesson is compact: information has value when it moves a belief across a consequential and feasible decision boundary.
Top · Contents · Previous · How X Works