Parametric optimization studies how the best decision and its achievable value change when the inputs to an optimisation problem change. Instead of solving only one demand level, capacity or price, it describes a family of related problems. The gold standard is to identify the parameter range, derive the decision within each regime, locate the boundaries where the regime changes, and check feasibility before using the result.
This guide develops that process through original worked allocation, capacity, pricing and control examples. It distinguishes the optimal decision from the optimal value, explains why derivatives can fail at switching points, and shows how to validate a numerical parameter sweep. The examples are simplified mathematical teaching models, not measured operating rules for a particular organisation or machine.
Use the Gold Standard master, the Education guide to transferable learning and the How X Works hub for wider reading. For the broader question of how conclusions depend on assumptions, continue to How Sensitivity Analysis Works.
Choose your parametric-optimization route
Chapters 1–3: Define parameters and regimes
Chapters 4–6: Allocate resources and interpret values
Chapters 7–9: Handle ties and derive control laws
Chapters 10–12: Compute, differentiate and diagnose
Chapter index
Define parameters and regimes · Chapters 1–3
1. Separate the parameter from the decision
Allocate resources and interpret values · Chapters 4–6
4. Derive a resource-allocation map with two decisions
Handle ties and derive control laws · Chapters 7–9
7. Recognise ties and discontinuous decisions
Compute, differentiate and diagnose · Chapters 10–12
10. Compute a parameter sweep without losing its meaning
11. Check derivatives with one-sided experiments
12. Diagnose conditioning, model changes and misleading robustness
Transfer and continue · Chapters 13–14
13. Transfer the allocation calculation to a changed capacity
14. Answer common questions and build a dependable response map
CHAPTER 1 OF 14 · Define parameters and regimes
1. Separate the parameter from the decision
Imagine a workshop choosing an output level x after receiving a target demand θ. Its simplified loss is half the squared mismatch: f(x, θ) = (x − θ)²/2. Output must lie between zero and four units. The parameter θ is an input to the problem; x is the quantity being chosen. This distinction is the starting point of parametric optimisation.
For a fixed θ, the task is ordinary constrained minimisation. For a range of θ values, we want a rule describing the best x for every allowed input. Write x*(θ) for that rule when the optimum is unique. Write v(θ) for the minimum loss. These functions answer different questions: what should we do, and how well can we do it?
The interpretation of θ needs care. If it represents physical demand, negative values may be irrelevant. Allowing negative θ in a mathematical example is still useful because it exposes the lower bound. In an application, state the physically meaningful parameter domain rather than letting a convenient formula silently define it.
A family is more than a list of unrelated runs
Solving the model at θ = 1, 2 and 3 gives three answers. Understanding the family explains why those answers follow the same rule, where that rule stops working, and whether interpolation is legitimate. A spreadsheet containing many solver outputs is not yet a parametric explanation.
Record which elements may change. Demand can enter a target, a balance equation or a minimum-service constraint; those are different models. Capacity can be fixed or parameterised. Prices can enter objective coefficients. The same numeric change can have different consequences depending on where it appears. Before differentiating anything, write the optimisation problem clearly enough that another person can identify the changing input without guessing from its name.
CHAPTER 2 OF 14 · Define parameters and regimes
2. Solve the bounded tracking problem completely
Without the output bounds, differentiating (x − θ)²/2 with respect to x gives x − θ. Its zero is x = θ. The second derivative is one, so this is the unique unconstrained minimum. When θ lies between zero and four, that decision is feasible and produces zero mismatch loss.
If θ is below zero, every feasible x is above the target. Increasing x moves farther from θ, so the best feasible decision is x = 0. If θ exceeds four, every feasible x is below the target; increasing x reduces the mismatch until the upper bound is reached. Then x = 4 is best.
The complete solution is therefore x*(θ) = 0 for θ ≤ 0, x*(θ) = θ for 0 ≤ θ ≤ 4, and x*(θ) = 4 for θ ≥ 4. The overlapping boundary descriptions agree. This is a clipping rule, or projection of θ onto the interval [0, 4].
Substitute back to obtain the value
The minimum loss is θ²/2 below zero, zero between zero and four, and (θ − 4)²/2 above four. At θ = 5, the best output is 4 and the loss is 1/2. At θ = 6, output remains 4 but the loss rises to 2. An unchanged decision therefore does not imply an unchanged outcome.
The solution map is continuous but has slope changes at zero and four. Its derivative is zero outside the interval and one inside it; an ordinary two-sided derivative does not exist at the switching points. The value function behaves differently: its first derivative exists at both boundaries and equals zero there. Its second derivative changes. These observations come directly from the formulas and show why “the optimisation is differentiable” is too vague a claim.
CHAPTER 3 OF 14 · Define parameters and regimes
3. Explain regimes through active constraints
A constraint is active when it holds with equality at the chosen point. In the tracking example, the lower bound is active for negative targets, neither bound is active for targets strictly between zero and four, and the upper bound is active for targets above four. These three cases explain the three pieces of the decision rule.
An active-set argument is not simply an exercise in naming constraints. It supplies candidate equations. If the upper bound is active, set x = 4 and check when that candidate is optimal. If neither bound is active, solve x − θ = 0 and check when the resulting x lies strictly inside the interval. Each candidate is valid only on the parameter range that passes all required checks.
Verify multipliers as well as feasibility
Write the bounds as −x ≤ 0 and x − 4 ≤ 0, with nonnegative multipliers α and β. Stationarity is x − θ − α + β = 0. Complementarity requires αx = 0 and β(x − 4) = 0.
For θ < 0, x = 0, β = 0 and α = −θ satisfy these equations with α positive. For 0 < θ < 4, both multipliers vanish and x = θ. For θ > 4, x = 4, α = 0 and β = θ − 4. At a boundary, an active constraint can have a zero multiplier. Tightness alone does not prove that relaxing the constraint has a positive first-order benefit.
Because this example has a strictly convex objective and affine constraints, these conditions certify the unique optimum. In a nonconvex problem, similar-looking stationarity equations may describe a local minimum, a maximum or another stationary point. The Convex Duality guide explains the connection between feasible decisions, multipliers and valid optimality certificates.
CHAPTER 4 OF 14 · Allocate resources and interpret values
4. Derive a resource-allocation map with two decisions
Now allocate a total requirement θ between two activities x and y. Minimise x²/2 + y² subject to x + y = θ, 0 ≤ x ≤ 2 and y ≥ 0. The first activity is cheaper at the margin for a given output, but it has a capacity limit. Assume θ ≥ 0 and continuously divisible quantities.
Before the capacity becomes active, substitute y = θ − x. The objective becomes x²/2 + (θ − x)². Its derivative is 3x − 2θ, so x = 2θ/3 and y = θ/3. This candidate is feasible while 2θ/3 ≤ 2, or θ ≤ 3. Its second derivative is three, confirming a unique minimum.
For θ > 3, the unconstrained allocation asks for more than two units of x. The optimum is x = 2 and y = θ − 2. Thus the complete map changes at θ = 3: below it, each extra unit of demand is split two-thirds to x and one-third to y; above it, all extra demand must go to y.
Compute costs on both sides
For 0 ≤ θ ≤ 3, substituting the allocation gives v(θ) = θ²/3. For θ ≥ 3, v(θ) = 2 + (θ − 2)². At θ = 3 both formulas give 3. Their first derivatives also agree there: 2θ/3 and 2(θ − 2) both equal 2.
At θ = 2, the allocation is (4/3, 2/3) and the value is 4/3. At θ = 4, the allocation is (2, 2) and the value is 6. A single linear extrapolation from the first regime would incorrectly prescribe x = 8/3 at θ = 4, violating capacity. The switch is operationally meaningful: the cheaper activity has run out of room.
CHAPTER 5 OF 14 · Allocate resources and interpret values
5. Interpret marginal values without extrapolating too far
In the allocation example, v′(θ) measures the local increase in optimal cost per additional unit of total requirement. Below θ = 3 it equals 2θ/3. Above θ = 3 it equals 2(θ − 2). The derivative reflects re-optimisation: both activities are allowed to adjust according to the current regime.
At θ = 4, the marginal cost is 4. Raising demand by 0.1 gives an actual cost increase of 2.1² − 2² = 0.41, while the first-order prediction is 4 × 0.1 = 0.4. The difference, 0.01, is the quadratic remainder. A derivative is a local approximation, not an exact promise for a finite change.
Relaxing capacity has a different sign
Let the x capacity be c instead of two, while θ remains four. In the capacity-binding regime, x = c and y = 4 − c, so the value is c²/2 + (4 − c)². Differentiating with respect to c gives 3c − 8. At c = 2 this equals −2: a small capacity increase reduces the optimal cost by about two units per capacity unit.
Increasing c from 2 to 2.1 changes the value from 6 to 2.205 + 3.61 = 5.815, a reduction of 0.185. The linear prediction was 0.2. Once c reaches 8/3, the unconstrained allocation is feasible and further capacity has no benefit in this model.
Boyd and Vandenberghe’s perturbation and sensitivity analysis develops the relationship between optimal dual variables and changes in optimal value under appropriate duality assumptions. Always check the sign convention: increasing a resource bound and increasing a required output need not have the same effect, even if both changes are called a “one-unit increase.”
CHAPTER 6 OF 14 · Allocate resources and interpret values
6. Locate the parameter domain where a solution exists
A solution formula is incomplete if it does not say where the problem is feasible. Add an upper bound y ≤ 3 to the allocation model, retaining x ≤ 2. The total requirement can then be met only for 0 ≤ θ ≤ 5. For θ > 5, no allocation satisfies the balance equation. For θ < 0, nonnegative outputs cannot sum to θ.
Within [0, 5], the earlier formula remains valid: x = 2θ/3 and y = θ/3 until θ = 3, then x = 2 and y = θ − 2. At θ = 5 both capacities are saturated. The right-hand derivative of a finite optimal cost is not available beyond that point because the original problem has no feasible solution there.
Infeasible is not expensive
Replacing infeasibility with a large arbitrary cost can conceal an important distinction. An expensive plan exists; an infeasible plan does not. If unmet demand is allowed, introduce it explicitly, for example x + y + s = θ with s ≥ 0 and a stated penalty or priority for s. That is a new model, and its recommendations should be labelled accordingly.
Suppose θ = 5.2. The original model fails feasibility. A model allowing shortage can choose x = 2, y = 3 and s = 0.2, but whether that is acceptable depends on the meaning of the requirement. A soft service target and a physical conservation law cannot be treated interchangeably.
Also distinguish infeasibility from an unbounded objective and from a finite infimum that is not attained. For example, minimising x over x > 0 has infimum zero but no minimiser. These cases require different responses. A dependable parameter map records status as well as the decision and value; otherwise a blank solver cell can be mistaken for a zero-cost operating point.
CHAPTER 7 OF 14 · Handle ties and derive control laws
7. Recognise ties and discontinuous decisions
Parametric optimisation does not always produce a single continuous decision rule. Consider minimising θx over 0 ≤ x ≤ 1. If θ > 0, the unique optimum is x = 0. If θ < 0, it is x = 1. At θ = 0, every x in the interval is optimal because the objective is identically zero.
The solution is therefore a set-valued map at the switching point. Its optimal value is v(θ) = min{0, θ}, a continuous function with a kink at zero. Any rule selecting the unique optimum on each side jumps from one endpoint to the other. Smooth input data have not prevented a discontinuous selected decision.
A solver can choose a tie without explaining it
At θ = 0, one solver may return zero, another one, and another an interior point. These can all be correct. If the application needs repeatability, add a documented tie-breaking rule. For example, among primary minimisers, choose the point closest to a previous operating decision. This secondary preference should not be confused with evidence that the primary problem had a unique answer.
Adding a small quadratic term also changes the problem. Minimise θx + εx²/2 for ε > 0. The optimum is the clipping of −θ/ε to [0, 1]. The resulting transition is continuous across a band of width ε, but it no longer solves the original linear objective exactly throughout that band. Regularisation trades fidelity to the original objective for another property.
Report that trade clearly. In scheduling or configuration selection, a tiny estimated price difference can cause a large switch in the chosen plan while barely changing objective value. Decision instability and value instability are different phenomena. Which matters more depends on switching costs, implementation risk and the purpose of the model.
Discrete decisions create additional switches
If the tracking decision must be an integer in {0, 1, 2, 3, 4}, the optimum is the feasible integer nearest to θ. For example, θ = 2.4 selects x = 2 with loss 0.08, while θ = 2.6 selects x = 3 with the same loss. At θ = 2.5, both choices have loss 0.125. A small parameter change across 2.5 causes a full unit change in the optimal decision.
Between integers k and k + 1, the switch follows by equating (k − θ)² and (k + 1 − θ)², which gives θ = k + 0.5. This identifies the thresholds exactly. It also shows why rounding a continuous answer needs a problem-specific argument: nearest-integer rounding works here because the objective is a scalar squared distance and the feasible integers form a simple interval. In a coupled allocation problem, independently rounding each variable can break the balance equation. Recheck the complete constraints after any rounding step.
CHAPTER 8 OF 14 · Handle ties and derive control laws
8. Understand why quadratic programmes produce regions
A common structured family minimises xᵀHx/2 + (q + Fθ)ᵀx subject to Ax ≤ b + Eθ. Here H, A, q, F, b and E are fixed, θ is a vector of parameters, and H is positive definite. For every feasible parameter value, strict convexity gives a unique minimising decision.
Choose a candidate active set I. Its constraints are equations A_I x = b_I + E_I θ. Stationarity is Hx + q + Fθ + A_Iᵀλ_I = 0. If the associated block system is nonsingular, solving these linear equations expresses x and λ_I as affine functions of θ.
The candidate is valid where its inactive inequalities hold and its active multipliers are nonnegative. Because the candidate formulas and bounds depend affinely on θ, these checks define linear inequalities in parameter space. Their intersection is a polyhedral region. Moving outside that region means the assumed active set no longer supplies a valid optimality certificate.
The assumptions carry the conclusion
This derivation does not imply that every nonlinear parametric problem has a piecewise-affine solution. If the Hessian itself depends on θ, if constraints are nonlinear, or if discrete choices are introduced, different structures can arise. Degenerate active sets also need care because a naive block-system inversion may fail even when the optimisation problem has a unique solution.
Bemporad, Morari, Dua and Pistikopoulos’s original work on multiparametric quadratic programming for model predictive control established a foundational route to explicit control laws for the constrained systems studied. The useful idea is to solve and organise parameter regions in advance, then evaluate the appropriate decision formula when a new parameter arrives. The derivation above explains the linear algebra behind that idea without promising that the region count will remain small.
CHAPTER 9 OF 14 · Handle ties and derive control laws
9. Work a complete one-step control example
Consider a scalar system with current state s and next state s + u. Choose an input u between −1 and 1 to minimise (s + u)² + u². The first term penalises the next state; the second penalises input effort. Treat s as the parameter and u as the decision. This is a one-step teaching model, not a general stability guarantee.
Differentiating with respect to u gives 2s + 4u. The unconstrained optimum is u = −s/2. It satisfies the input bounds when −2 ≤ s ≤ 2. For s > 2, the best feasible input is −1. For s < −2, it is 1. Thus the optimal input is an affine law in the central region and a constant saturated law on each side.
At s = 1, choose u = −0.5, giving next state 0.5 and cost 0.5. At s = 3, choose u = −1, giving next state 2 and cost 5. The unconstrained recommendation −1.5 would have cost 4.5 but violates the actuator limit, so that lower number is not achievable under the stated constraints.
Add a state constraint and recheck feasibility
Suppose the next state must also satisfy |s + u| ≤ 1. With |u| ≤ 1, feasibility requires |s| ≤ 2. At s = 3 there is no acceptable input. Clipping the unconstrained command to −1 does not repair the state constraint: the next state remains 2.
This example distinguishes input saturation from full constraint handling. A clipped controller solves the earlier problem but not necessarily the new one. When changing a model, re-derive the feasible parameter domain as well as the decision formula. In a multi-step controller, future feasibility and stability depend on additional design choices; they cannot be inferred from this single-step calculation alone.
CHAPTER 10 OF 14 · Compute, differentiate and diagnose
10. Compute a parameter sweep without losing its meaning
A numerical sweep solves the same mathematical family at selected parameter values. Before running it, decide whether the purpose is to discover switching points, check a derived formula, assess a local derivative or compare operational scenarios. Those purposes call for different sample locations and different output records.
For the allocation example, include θ values just below, at and just above three, such as 2.99, 3 and 3.01. Also include zero, five if the second capacity is present, and a value above five to verify the expected infeasible status. A grid containing only whole numbers can miss a narrow regime or blur where a change occurs.
Store the parameter, solver status, decision, objective value, maximum constraint violation and relevant multipliers. Compute the objective again from the returned decision rather than relying only on a reported scalar. Keep the units and model version with the results so a later run can be compared meaningfully.
Reuse computation without reusing an invalid conclusion
CVXPY’s disciplined parametrized programming documentation describes how suitable parameterised problems can reuse a cached mapping from parameters to solver data. Its solver documentation discusses warm starts, which can reuse information from previous solves. These are computational conveniences; they do not certify that the previous active set remains correct.
Warm starting a nonconvex local solver can keep it on one branch while another branch becomes globally better. Compare multiple initialisations or use a global method when global optimality is required. Even in a convex problem, numerical tolerances can make a nearly active constraint look alternately active and inactive. Report residuals and the scale of the ambiguity instead of treating every last decimal as a physical regime change.
CHAPTER 11 OF 14 · Compute, differentiate and diagnose
11. Check derivatives with one-sided experiments
For the tracking problem, the derivative of x*(θ) is one inside (0, 4) and zero above four. At θ = 4, a central finite difference gives [x*(4 + h) − x*(4 − h)]/(2h) = 1/2 for every small positive h. That number is not an ordinary derivative at the boundary. It averages two different one-sided slopes.
The left difference is [4 − (4 − h)]/h = 1. The right difference is [4 − 4]/h = 0. Comparing them reveals the kink immediately. Shrinking h does not make the central estimate become a valid two-sided derivative; the underlying map is not differentiable there.
Choose perturbations that answer the actual question
If a capacity can only increase, a right-hand derivative may be the relevant quantity. If an input is measured with error on both sides of a threshold, both regimes matter. If a parameter must remain nonnegative, a symmetric perturbation at zero may leave the model’s allowed domain. Numerical differentiation must respect that domain.
Far from a switch, use several perturbation sizes. Large steps can cross regimes and measure a secant rather than a local slope. Extremely small steps can amplify solver error and floating-point cancellation. Agreement over a sensible range of step sizes is more informative than one attractive decimal.
Automatic differentiation does not remove these mathematical issues. It may differentiate a smooth local solution map under suitable conditions, or return a convention-dependent result at a nonsmooth point. Verify the meaning of the derivative exposed by the software. In particular, the derivative of the optimal decision is not the derivative of the optimal value, and differentiating the objective while holding x fixed omits the way an optimal decision changes with the parameter.
CHAPTER 12 OF 14 · Compute, differentiate and diagnose
12. Diagnose conditioning, model changes and misleading robustness
A smooth solution map can still be highly sensitive. Minimise εx²/2 − θx over the real line, with fixed ε > 0. Stationarity gives x*(θ) = θ/ε. Its derivative is 1/ε. If ε = 0.01, a parameter change of 0.001 shifts the optimal decision by 0.1. The map is differentiable everywhere, but small input errors can have substantial decision effects.
This is a conditioning issue. The objective is relatively flat, so a small linear tilt moves its minimiser far. Adding bounds may limit the movement, but then switching regions appear. Neither a smooth graph nor a successful solver status establishes practical stability.
Distinguish a changed parameter from a changed model
If demand rises while the production-cost formula remains valid, a parametric response can be appropriate. If higher output changes the production technology, failure rate or staffing rules, the original family may no longer describe reality. Extrapolating its formula is then a modelling error rather than a calculus error.
Parametric analysis also does not automatically choose a robust action. Solving separately for each possible θ assumes the decision can adapt to that value. If one decision must be committed before θ is known, evaluating x*(θ) for each scenario gives an optimistic comparison that may be impossible to implement. The timing of information and action belongs in the model.
Similarly, sweeping a preference weight generates different scalar optimisation problems; it does not prove that the weight is known. The Multiobjective Optimization guide explains how changing trade-off weights differs from changing factual inputs. When an upper-level choice intentionally changes another optimiser’s response, the Bilevel Optimization guide supplies the additional hierarchical structure. These connections help identify which question the parameter map can actually answer.
CHAPTER 13 OF 14 · Transfer and continue
13. Transfer the allocation calculation to a changed capacity
Try the following exercise before reading the answer. Keep the objective x²/2 + y² and balance x + y = θ, but change the capacities to 0 ≤ x ≤ 3 and 0 ≤ y ≤ 2. Determine the feasible demand interval, the switching demand, the optimal allocation at θ = 4 and θ = 5, and the marginal cost just below the maximum feasible demand.
The feasible interval is [0, 5], because the capacities add to five and both outputs are nonnegative. The unconstrained allocation remains x = 2θ/3, y = θ/3. The x capacity is reached at θ = 4.5, while the y capacity would be reached at θ = 6 if x were unconstrained. Therefore x becomes binding first.
For 0 ≤ θ ≤ 4.5, the unconstrained allocation is feasible. For 4.5 ≤ θ ≤ 5, the optimum is x = 3 and y = θ − 3. At θ = 4, the allocation is (8/3, 4/3) and the value is 16/3. At θ = 5, it is (3, 2) and the value is 4.5 + 4 = 8.5.
Check continuity and the boundary interpretation
At θ = 4.5, the first value formula gives θ²/3 = 6.75. The second gives 4.5 + (θ − 3)² = 6.75. Their slopes agree at three. Just below θ = 5, the derivative is 2(θ − 3), approaching four. Above five the model is infeasible, so that left-hand marginal cost cannot justify accepting additional demand without changing capacity or allowing shortage.
The earlier switching point was three. Copying it into this revised model would be wrong even though the objective and total maximum capacity look familiar. The exercise tests the transferable skill: derive the unconstrained candidate, find which bound is reached first, verify the remaining constraints, and state the domain of every resulting formula.
CHAPTER 14 OF 14 · Transfer and continue
14. Answer common questions and build a dependable response map
Is parametric optimisation just sensitivity analysis?
They overlap, but their emphasis differs. Sensitivity analysis broadly examines dependence on assumptions and inputs, including models with no optimisation. Parametric optimisation studies a family of optimisation problems, often seeking solution formulas, regions or value functions over a parameter domain. A local sensitivity derivative is one useful part of that wider map.
Does an active constraint always have a positive price?
No. In the tracking example at θ = 4, the upper bound is active but its multiplier is zero. The target is exactly achievable without first-order loss. Tightness describes geometry; the multiplier describes a particular marginal relationship under the model’s conventions and regularity conditions.
Can I interpolate between computed optimal decisions?
Sometimes, but check the structure. Within an affine solution region, interpolation reproduces the affine rule. Across a switching boundary, interpolating can give a feasible but suboptimal decision, or even an infeasible one when the feasible set changes nontrivially. Integer decisions make naive interpolation especially problematic because a fractional configuration may not exist.
What should a first implementation demonstrate?
Begin with bounded tracking and reproduce the three regimes exactly. Next implement the two-activity allocation problem and test both sides of its capacity switch. Include deliberately infeasible demands and the linear tie example. A useful implementation should explain its status rather than merely return a number for every input.
For an applied project, keep a response-map record containing the parameter definition and units, allowed domain, objective and constraints, regime formulas or numerical method, boundary tests, feasibility residuals and conditions requiring a model review. Have another person reproduce one interior case and one boundary case. The goal is a decision rule whose limits are as clear as its successful predictions, so that a changed input prompts the correct adjustment rather than an unjustified extrapolation.