THE GOLD STANDARD OF JUDGEMENT · SPECIALIST STUDY
Choose a portfolio when good options compete for the same scarce capacity
Portfolio decisions rarely ask whether one option is good. They ask which combination fits a budget, capacity, risk limit and set of obligations. Multi-choice knapsack models represent that structure: choose at most or exactly one option from each group while respecting shared constraints.
The calculation can reveal opportunity cost, but a model is only as defensible as its values, constraints and uncertainty. Shadow prices describe local marginal value in a relaxed linear model; they are not permanent market prices. The gold standard combines optimisation with transparent judgement and scenario testing.
Start with the closest route. This guide belongs to the Judgement subject guide, the Gold Standard of Education, and the How X Works learning library.
Choose a reading route
Formulate choices · chapters 1–3
Solve the portfolio · chapters 4–5
Interpret scarcity · chapters 6–8
Test robustness · chapters 9–10
Transfer and govern · chapters 11–12
The whole problem at a glance
| Object | Meaning | Audit question |
|---|---|---|
| Choice group | Mutually exclusive versions of one need | Must one, at most one or no option be chosen? |
| Capacity constraint | Shared scarce resource | Is the limit hard, uncertain or negotiable? |
| Objective value | Modelled benefit or cost | Whose value is represented or omitted? |
| Shadow price | Local value of relaxing a linear constraint | Does integrality make the next unit lumpy? |
A mathematically optimal portfolio is conditional on the model. Constraints can encode law and safety; objectives encode contestable value choices.
CHAPTER 1 OF 12 · Formulate choices
1. Define items, groups and decision variables
Let xᵢⱼ equal one when option j from group i is selected and zero otherwise. A group may represent one project with small, medium and large versions.
Use exactly-one when every group must receive a version, at-most-one when omission is allowed, and include an explicit do-nothing option when its consequences need comparison.
Avoid double listing. Two options that share the same underlying action can be selected twice unless exclusivity is encoded.
Attach owner, cost, benefit, timing and dependencies to each option. The binary variable is only a switch; meaning stays in the catalogue.
CHAPTER 2 OF 12 · Formulate choices
2. Separate hard constraints from preferences
Budget, staff-hours, physical capacity, legal duties and risk ceilings may constrain the portfolio. State which are inviolable and which are planning estimates.
Preferences such as variety, geographic spread or strategic balance may be objective terms, goals or soft constraints with penalties. Do not disguise them as facts.
Dependencies need explicit logic: an advanced option may require a foundation, and two projects may compete for the same site even if budget allows both.
Run a feasibility check before optimisation. If no portfolio satisfies obligations, identify conflicting constraints rather than weakening them silently.
CHAPTER 3 OF 12 · Formulate choices
3. Build benefits without false commensurability
A single objective often sums option values, but scores across safety, learning, revenue and access may not share a natural unit.
Use monetisation only when defensible. Otherwise apply multi-criteria value elicitation, goal programming or report a Pareto set. Normalisation choices must be visible.
Avoid counting the same benefit twice across correlated criteria. Trace each score to evidence and a decision meaning.
Include uncertainty ranges and downside, not only expected benefit. A portfolio with a slightly lower score may be more resilient.
CHAPTER 4 OF 12 · Solve the portfolio
4. Work a three-group example
A centre chooses one upgrade from each of three rooms: basic or advanced equipment. The six options have costs and learning values, while total budget is 30 units.
Exactly-one constraints ensure one version per room. The budget couples choices: selecting one advanced option can displace another elsewhere.
Enumerating eight combinations is possible here and useful for checking. Larger portfolios need integer optimisation, but small examples reveal the mechanics.
Report chosen and rejected options with the binding constraint. A result is easier to trust when readers can see the closest alternative.
CHAPTER 5 OF 12 · Solve the portfolio
5. Use dynamic programming, branch-and-bound or solvers carefully
Dynamic programming can exploit integer capacity; branch-and-bound solves linear relaxations and prunes impossible regions; modern solvers combine cuts and heuristics.
Choose algorithms by size, structure, precision and time limit. A feasible incumbent under a gap is not the same as proven optimality.
Record solver version, parameters, runtime, bound and optimality gap. Reproducibility requires more than the final item list.
Validate with conservation checks and small enumerated cases. Solver success cannot repair wrong coefficients or missing constraints.
CHAPTER 6 OF 12 · Interpret scarcity
6. Understand the linear relaxation
Relaxing binary variables to values between zero and one creates a linear program that provides an upper bound for maximisation.
Fractional choices may be operationally impossible, but the relaxation reveals which constraints limit value and guides branch-and-bound.
Compare the integer solution with the relaxation. A large integrality gap signals that lumpy choices make marginal reasoning unreliable.
Do not present fractional portfolios as implementation plans unless partial funding or scaling is genuinely allowed.
CHAPTER 7 OF 12 · Interpret scarcity
7. Read shadow prices as local marginal values
In a linear model, a shadow price estimates how the objective changes when a constraint right-hand side increases by one unit, within a valid range.
If budget’s shadow price is three, one more unit may improve modelled value by three locally. It does not mean every future budget unit is worth three.
Integer portfolios are lumpy. One extra unit may change nothing until enough accumulates to unlock an option. Reduced costs and discrete scenario runs complement dual values.
Sign, unit and objective direction matter. Publish them in plain language and never treat solver duals as observed market prices.
CHAPTER 8 OF 12 · Interpret scarcity
8. Analyse rejected options and opportunity cost
A rejected option may be individually attractive yet block a more valuable combination. Compare the best portfolio with and without that option forced in.
The objective difference is a modelled opportunity cost under current assumptions. It explains trade-offs more clearly than saying budget was insufficient.
Check whether rejection arises from one binding resource, exclusivity, dependency or objective score. Each suggests a different remedy.
Some obligations should never be described as opportunity costs alone. Rights, safety and minimum access may belong in constraints before optimisation.
CHAPTER 9 OF 12 · Test robustness
9. Test budgets, scores and constraints
Vary budget across a range and trace portfolio changes. Stair-step behaviour is normal because options are indivisible.
Vary uncertain benefits and costs jointly, not one at a time only. Correlated overruns can make an apparently diversified portfolio fragile.
Relax each constraint carefully and examine value, feasibility and distribution. A high shadow price identifies a bottleneck worth investigation, not automatic expansion.
Report switch points: the assumptions or resource levels at which another portfolio becomes preferred.
CHAPTER 10 OF 12 · Test robustness
10. Protect fairness and resilience
Aggregate value can concentrate resources. Add minimum service, geographic access or subgroup constraints when justified and review their effects transparently.
Diversification constraints can reduce exposure to one supplier, technology or scenario. They also carry opportunity cost that should be shown.
Stress-test loss of one project, cost escalation and delayed benefit. A robust portfolio may outperform the nominal optimum when conditions change.
Avoid token constraints that are easy to satisfy without real access. Measure outcomes, not only counts of selected items.
CHAPTER 11 OF 12 · Transfer and govern
11. Transfer the method to a learner’s course portfolio
A learner chooses one module from each requirement group under timetable and workload limits. Interests and prerequisites shape value and feasibility.
The highest score should not erase wellbeing, required foundations or genuine career uncertainty. Some constraints represent health and institutional rules.
Shadow price on weekly hours can reveal whether another hour would help locally, but real schedules are lumpy and attention is not perfectly divisible.
The transfer keeps the mechanism while changing the values: use the model to expose trade-offs, not dictate a life choice.
CHAPTER 12 OF 12 · Transfer and govern
12. Publish a portfolio decision record
List options, groups, coefficients, constraints, uncertainty, solver settings, selected portfolio, alternatives and sensitivity.
Separate observed data, forecasts and stakeholder weights. Readers should know which numbers can be verified and which express judgement.
Record excluded options and reasons. Omission can shape the result more than optimisation.
The final audit asks which reasonable change reverses the portfolio and who bears the cost of each binding constraint.
Practice and evidence check
- Write exactly-one and at-most-one constraints for a choice group.
- Enumerate a two-group toy portfolio and verify the solver result.
- Compare integer and relaxed solutions.
- Explain one shadow price with unit and validity range.
- Force a rejected option and measure portfolio opportunity cost.
A defensible portfolio exposes what was selectable, what was scarce, which values were used and how robust the combination is.
Portfolio optimisation audit
Are all real alternatives represented?
A solver cannot select an omitted option. Record catalogue scope, ownership and update date.
Is every exclusivity rule correct?
Exactly-one, at-most-one and do-nothing choices create different decisions. Test each group manually.
Which constraint is binding?
Report slack and operational meaning, then check whether the limit is measured, forecast or policy.
Is the shadow price valid for an integer decision?
Use it as local relaxed-model evidence and confirm discrete resource increments with scenario solves.
How large is the optimality gap?
Publish incumbent, bound, runtime and gap when the solver stops early.
Who receives value and who bears scarcity?
Disaggregate outcomes; aggregate score can hide concentrated exclusion.
What if costs rise together?
Use correlated scenarios, not isolated one-variable sensitivity only.
Can another reader reproduce the portfolio?
Preserve data version, model, solver, parameters and decision record.
Frequently asked questions
Is multi-choice knapsack the same as ordinary knapsack?
It adds groups and rules choosing at most or exactly one option from each group.
What is a shadow price?
A local marginal objective change from relaxing a linear constraint by one unit.
Why can one extra budget unit have no effect?
Binary choices are lumpy; the next feasible upgrade may require several units.
Does the optimal portfolio maximise real value?
Only the value represented by the model under stated assumptions.
How should fairness enter?
Through transparent objectives, minimum constraints, alternatives and distributional reporting.
Continue through the ecosystem
The Gold Standard of Judgement — the subject root for defensible choice
Goal Programming, Priority Levels and Deviation Variables — compare portfolio objectives with aspiration-based optimisation
Satisficing, Aspiration Levels and Robust Feasibility — define acceptable portfolios before ranking
Dominance, Pareto Frontiers and Non-Compensable Constraints — separate efficiency from prohibited trade-offs
Sources and further reading
- Google OR-Tools — knapsack problems
- Google OR-Tools — mixed-integer programming example
- NEOS Guide — mixed-integer linear programming
- Gurobi Optimizer Reference — dual values and shadow prices
Use sources to inspect definitions and limits, not as decoration. Apply methods in ways that fit the actual decision, data and people affected.
Portfolio scenarios and failure checks
Choose one service level per site
Each site has basic, standard and enhanced versions with distinct cost and benefit. Exactly-one rules make omission impossible; adding a do-nothing option makes non-investment visible. The portfolio record explains whether every site has a minimum entitlement or whether scarce resources may leave one site unchanged. That ethical choice belongs before the solver, not inside a hidden coefficient.
Budget and staff constraints bind differently
Two portfolios may cost the same while one needs scarce specialist hours. Reporting only the monetary budget hides the true bottleneck. The model includes both resources and publishes slack. A high shadow price on staff time may justify training, rescheduling or redesign, but the discrete portfolio is resolved again before action.
Mutually dependent projects
A software upgrade creates value only if training is selected, while training alone has limited value. Binary linking constraints prevent the solver from choosing the attractive score without its prerequisite. The catalogue records whether value estimates already include dependency effects; otherwise the pair may be double-counted when both enter the objective.
Projects with shared fixed cost
Several options use one platform licence. Charging the full cost to every option makes the portfolio look unaffordable; ignoring it understates cost. Introduce a binary activation variable for the shared platform and link chosen projects to it. This small modelling detail can change the preferred combination more than a sophisticated solver.
Risk concentration
The highest expected-value set relies on one supplier. A diversification constraint limits exposure, or scenarios assign joint failure loss. The team reports the value surrendered for resilience. Calling that difference inefficiency would misunderstand the decision: the purpose of the constraint is to buy performance when the nominal scenario fails.
Fairness minimums
An aggregate benefit objective repeatedly selects already well-served areas because their measured returns are easier. Minimum-access constraints guarantee a floor, while alternative optima reveal the cost and distribution. The floor should be based on rights, need or policy, not a symbolic number that can be met without usable service.
Integer step in budget value
The relaxed model shows a positive shadow price, yet one extra budget unit changes nothing because the next upgrade costs five. Solve the portfolio at budgets plus one through plus five and show the stair step. Decision-makers then understand both local scarcity and the practical funding increment required to unlock value.
Cost overrun scenario
Costs are forecasts, not constants. Apply correlated ten- and twenty-percent overruns, identify portfolios that become infeasible and reserve contingency. A slightly lower nominal score may preserve obligations under stress. Robustness should examine whether the plan can be repaired, not merely whether its expected value is highest.
Time-staged portfolio
Benefits and capacity occur across years. A project consumes implementation staff now and maintenance later. Multi-period constraints prevent a portfolio that fits total budget but overwhelms one quarter. Discounting is explained, and irreversible dependencies receive scenario analysis rather than disappearing into one present-value number.
Decision after the solver
The optimisation team presents the recommended portfolio, nearest alternatives, binding constraints, distribution and switch points. Governance checks omitted options, legal duties and data quality before approval. If leaders choose another feasible portfolio, they record the value judgement and evidence. The model clarifies accountability; it does not transfer responsibility to software.
Extended practice laboratory: choosing a constrained programme portfolio
A multi-choice knapsack is useful when a decision must choose at most one option from each group while respecting shared limits. Imagine a community learning programme selecting one delivery plan for each of three districts. The annual budget is 18 units and the trained-facilitator capacity is 11 days. Estimated benefit points are a common planning scale, not a claim that every kind of benefit is interchangeable.
| District option | Budget | Facilitator days | Estimated benefit |
|---|---|---|---|
| North basic | 4 | 2 | 7 |
| North extended | 7 | 5 | 12 |
| East basic | 5 | 3 | 8 |
| East extended | 8 | 5 | 13 |
| South basic | 3 | 3 | 6 |
| South extended | 6 | 5 | 11 |
The group rule prevents selecting both basic and extended plans for the same district. A policy rule requires every district to receive one plan, so “choose none” is not available. The budget and facilitator constraints operate across all districts.
Write the decision before solving it
Let each option have a binary variable equal to one when selected. For each district, the basic and extended variables must sum to one. The sum of selected budgets cannot exceed 18, and the sum of facilitator days cannot exceed 11. The objective maximises the sum of estimated benefit points. Writing these rules exposes choices that a spreadsheet ranking can hide: universal coverage is mandatory, options are mutually exclusive, resources are shared, and benefit is treated as additive for this exercise.
Now test feasible portfolios. Selecting all extended plans costs 21 and uses 15 facilitator days, so it is infeasible. North extended, East basic and South basic cost 15, use 11 days and produce 26 benefit points. North basic, East extended and South basic also cost 15, use 10 days and produce 26. North basic, East basic and South extended cost 15, use 10 days and produce 26. The apparent optimum is not unique. That matters: an optimisation tool may return one portfolio even though several portfolios share the objective value.
| Portfolio | Budget used | Days used | Benefit | Slack |
|---|---|---|---|---|
| North extended, East basic, South basic | 15 | 11 | 26 | 3 budget, 0 days |
| North basic, East extended, South basic | 15 | 10 | 26 | 3 budget, 1 day |
| North basic, East basic, South extended | 15 | 10 | 26 | 3 budget, 1 day |
| All basic | 12 | 8 | 21 | 6 budget, 3 days |
If the objective values tie, secondary judgement becomes visible. The second or third portfolio preserves one facilitator day. The first concentrates the extended plan in North; the second does so in East; the third does so in South. A decision-maker can apply a documented secondary rule—greater resilience, higher unmet need, lower implementation risk or a rotation commitment—instead of pretending the solver identified a uniquely correct answer.
Use marginal values carefully
For a linear relaxation, a dual value can describe how much the relaxed objective would improve with one more unit of a binding resource near the current solution. That local interpretation is often called a shadow price. In an integer model, one extra unit may do nothing until it enables a discrete switch. Here the all-basic portfolio can be upgraded in steps: moving North from basic to extended adds 3 budget, 3 days and 5 benefit; moving East adds 3 budget, 2 days and 5 benefit; moving South adds 3 budget, 2 days and 5 benefit. The marginal value depends on which upgrade remains feasible.
At the portfolio North extended, East basic, South basic, facilitator days are binding while budget has slack. One extra facilitator day still may not improve the objective because the next useful upgrade also needs budget and at least one more day than is available after replacing choices. A continuous shadow price should not be reported as a guaranteed value of a real facilitator day. Run the integer model again with capacity 12, 13 and 14. Report the stepwise improvement curve.
Stress-test the estimates
Estimated benefits are rarely exact. Suppose North extended could deliver between 9 and 12 points because recruitment is uncertain, while East extended remains between 12 and 13. The North-heavy portfolio can fall from 26 to 23; the East-heavy portfolio falls at most to 25 if other estimates stay fixed. A one-point optimum under a single estimate would not justify choosing North without examining this uncertainty.
- Change one benefit estimate at a time to find the value at which the selected portfolio changes.
- Test joint pessimistic scenarios rather than assuming every estimate misses independently.
- Increase and decrease each capacity. Observe whether the recommendation is stable or sits at a cliff.
- Force each plausible portfolio in turn and calculate the objective sacrifice. This reveals the cost of policy preferences.
- Ask whether resource use and benefit are measured consistently across groups. Apparent precision cannot repair incomparable inputs.
Add non-compensable safeguards
A high total benefit should not compensate for an unacceptable outcome. The universal-coverage constraint already prevents excluding a district. Other safeguards might require an accessibility standard, a minimum share of places in underserved neighbourhoods, travel-time limits or a cap on implementation risk. Model these as constraints when they are true boundaries. Do not hide them as small weights in the objective, where a sufficiently large benefit elsewhere can overwhelm them.
Safeguards can create infeasibility. That result is information, not a modelling failure. If the budget cannot fund one accessible plan in every district, the decision is not to quietly relax accessibility. It is to expose the resource gap, redesign options, phase delivery or seek more capacity. Record which requirement caused the conflict.
Changed-context transfer: device replacement
Transfer the method to a school replacing devices. Each year group must receive one package: repair, standard replacement or enhanced replacement. Shared constraints include money, technician hours and delivery slots. Benefits include reliability and learning continuity. The group structure transfers directly, but the ethical limits change. A minimum accessibility specification and a maximum outage period may be hard constraints. Security support dates can make a cheap repair option inadmissible. A model that ignores those conditions can optimise an unsafe choice.
Transfer succeeds when the decision-maker rebuilds groups, resources, objective and safeguards from the new problem. Copying the old benefit scale or its shadow prices would be a category error. The structure is reusable; the evidence is local.
Distinguish model error from decision error
A model error occurs when the formulation, data or implementation fails to represent the stated problem: a missing mutual-exclusion constraint, a wrong cost, a duplicated option or an incorrect unit. A decision error can occur even when the model is solved perfectly: leaders may accept an unrealistic benefit estimate, omit a harmed group or treat a narrow objective as the whole public purpose. Verification checks the mathematics. Validation checks whether the mathematics is a useful representation. Governance checks whether the representation is legitimate for the decision.
Portfolio decision record
Preserve the option catalogue, excluded options and reasons, units, source dates, group rules, resource limits, objective definition, safeguards, solver status, optimality gap, alternate optima, sensitivity scenarios and final human choice. If leaders choose a portfolio other than the model’s first output, state why. A transparent override is often better judgement than automatic compliance with fragile inputs.
The final check is practical: another reader should be able to reproduce feasibility, identify the binding constraints, see which assumption changes the portfolio and explain what the objective leaves out. The model earns trust by making trade-offs inspectable, not by turning them into a single impressive number.
Final verification drill: the value of a new option
Add an East hybrid option costing 6 budget units, using 4 facilitator days and delivering an estimated 10 benefit points. Do not assume that a good standalone ratio makes it useful. Re-solve the complete portfolio. The hybrid can matter when it fits a resource combination that the basic and extended options cannot, but it may never appear if another option dominates it on every relevant dimension.
Next add a fixed setup cost of 2 budget units whenever any extended plan is chosen. This creates a linking condition across groups: the cost is paid once, not once per district. A naive option-by-option table may double-count or omit it. Verify the formulation with small hand-enumerated cases before trusting a large solve. Then inspect whether the fixed cost makes all-basic preferable or strengthens the case for choosing several extended plans together.
Finally, ask what would make the model obsolete. A new universal programme could remove the group structure; a staffing agreement could make facilitator days nonbinding; evidence might show that benefit is not additive because districts share participants. Schedule a model review around those assumptions. Optimisation supports judgement only while its representation remains connected to the decision environment.