Bilevel optimization chooses an upper-level decision while requiring a lower-level decision to solve its own optimisation problem. It is useful when a price changes a customer’s best response, a policy changes an operator’s choices, or a hyperparameter changes the model obtained from training. The gold standard is to preserve that response constraint rather than assume the upper level can command both decisions.
This guide derives a complete price-and-demand example, examines ambiguous follower choices, checks when optimality conditions are valid, and connects the same mathematics to machine learning. The calculations are original, deliberately simplified teaching cases. They show what a model establishes, which assumptions make the result possible, and what must be checked before transferring the method to a real decision.
Use the Gold Standard master, Education parent and How X Works hub for wider routes. The Convex Duality root supports the optimality arguments below, while the Stochastic Programming root explains a different kind of staged decision: adaptation to uncertain outcomes.
Choose your bilevel-optimization route
Chapters 1–3: Build the response and solve the leader
Chapters 4–6: Handle hierarchy, ties and optimality
Chapters 7–9: Reformulate and differentiate
Chapters 10–12: Check kinks, uncertainty and validity
Chapter index
Build the response and solve the leader · Chapters 1–3
1. Identify whose decision each variable represents
Handle hierarchy, ties and optimality · Chapters 4–6
4. See what goes wrong when both levels are merged
5. Treat ties as a substantive modelling choice
6. Use optimality conditions only when they characterise the follower
Reformulate and differentiate · Chapters 7–9
7. Understand complementarity, bounds and reformulation risks
8. Differentiate a response when the required smoothness exists
Check kinks, uncertainty and validity · Chapters 10–12
10. Expect kinks when constraints become active
11. Model approximate response and behavioural uncertainty honestly
Transfer and continue · Chapters 13–14
CHAPTER 1 OF 14 · Build the response and solve the leader
1. Identify whose decision each variable represents
Suppose a supplier chooses a price p and a customer then chooses quantity y. The supplier controls p. The customer controls y according to its own preferences and constraints. A model that lets the supplier choose both freely describes a different institution, even if the same variables appear in the equations.
Write the lower-level solution set as S(p): all customer quantities that minimise the customer’s cost, or maximise its utility, at price p. The supplier’s feasible decisions must then satisfy y∈S(p). This is the distinctive constraint in a bilevel problem. Ordinary feasibility, such as 0≤y≤4, is necessary but does not establish that the customer would choose y.
Sequence and commitment matter
The leader–follower interpretation requires the first decision to be observable and sufficiently committed for the follower to respond to it. If the supplier can change the price after seeing the customer’s choice, the assumed timing is wrong. If both choose simultaneously, a simultaneous-game model may be more appropriate.
The two levels do not always represent two people. In hyperparameter tuning, one computational process chooses a regularisation parameter and another trains model weights. The lower problem still restricts which weights arise from a parameter, even though there is no strategic customer. The mathematical hierarchy concerns dependence and optimisation, not necessarily power between organisations.
Before solving anything, make a control map: which variables belong to which level, what each level observes, what each level optimises and what limits its choices. Include whether the follower optimises globally, stops after a fixed algorithm or uses an empirical behavioural rule. These alternatives lead to different response maps.
A good model description should let another reader answer a simple question: why is this lower-level action expected to occur? If the answer is only that it helps the leader, the response constraint has probably been omitted or an optimistic tie-breaking assumption has gone unstated.
CHAPTER 2 OF 14 · Build the response and solve the leader
2. Solve the follower before optimising the leader
Consider a customer buying divisible quantity y between zero and four. Its benefit is U(y)=6y−y²/2, and it pays py. At a nonnegative price p, it maximises U(y)−py, equivalently minimising f(p,y)=y²/2+(p−6)y over 0≤y≤4.
The derivative with respect to y is y+p−6. The second derivative is one, so the objective is strictly convex and has a unique minimiser on the interval. Without the interval bounds, the minimiser would be y=6−p. Enforcing the bounds clips that value into [0,4].
Derive the complete response function
For 0≤p≤2, the response is y*=4 because the unconstrained quantity is at least four. For 2≤p≤6, the response is y*=6−p. For p≥6, the response is y*=0. At the boundary prices two and six, the adjacent formulas agree, so the quantity is continuous.
At price one, the customer buys four. At price three, it buys three. At price five, it buys one. At price seven, it buys nothing. These are calculated consequences of the assumed utility, not observations of actual consumers.
The cap of four can represent a purchase limit or a maximum usable quantity. Its interpretation matters. If the cap is a supplier capacity that changes with another leader decision, the lower feasible set must include that dependence. If a customer can substitute another product, its outside option may also change the response.
Solving the follower first reveals which regions the leader must consider. A derivative based only on the interior expression y=6−p would miss both saturated demand at low prices and zero demand at high prices. For more complicated models, the same principle applies through active constraints, discrete choices or changes in the set of optimal responses.
CHAPTER 3 OF 14 · Build the response and solve the leader
3. Optimise the leader over the response graph
Suppose the supplier’s production cost is one per unit and it chooses p∈[0,8]. Its profit is Π(p,y)=(p−1)y. Substituting the customer’s response yields three price regions rather than an unconstrained choice of p and y.
For 0≤p≤2, profit is 4(p−1), increasing to four at p=2. For 2≤p≤6, profit is (p−1)(6−p)=−p²+7p−6. This concave quadratic reaches its maximum at p=3.5, with y=2.5 and profit 6.25. For 6≤p≤8, profit is zero because the customer buys nothing.
Check all regions and endpoints
The interior candidate beats the best value four in the low-price region and zero in the no-demand region. Thus p*=3.5 is the global optimum of this teaching model. The calculation is complete because it checks every possible response regime, not merely one stationary point.
At this price the customer’s gross benefit is 6·2.5−2.5²/2=11.875. Its payment is 8.75, leaving net utility 3.125. Production costs 2.5, leaving supplier profit 6.25. The payment appears as a cost to one party and revenue to another; it should not be counted twice when calculating their combined surplus.
Combined surplus is benefit minus production cost, or 9.375. A central planner maximising combined surplus over 0≤y≤4 would maximise 5y−y²/2 and select y=4, producing surplus twelve. This comparison shows that maximising the leader’s profit is not identical to maximising total welfare.
The existing specialist guide to incentive response and leader–follower decisions develops the judgement implications. The arithmetic here provides a small, auditable foundation for asking whose objective is being optimised and whose outcome is merely being predicted.
CHAPTER 4 OF 14 · Handle hierarchy, ties and optimality
4. See what goes wrong when both levels are merged
If the supplier ignores the customer’s optimisation and simply chooses p∈[0,8] and y∈[0,4] to maximise (p−1)y, it chooses p=8,y=4 and reports profit twenty-eight. But at price eight the customer optimally chooses zero. The attractive pair is physically allowed by the box constraints and behaviourally inconsistent with the response model.
This is a useful diagnostic example because nothing is wrong with the arithmetic of the flattened problem. It solves the wrong feasible set. The bilevel feasible set is the graph of the response function, not the full price–quantity rectangle.
A weighted sum is another different problem
Adding the leader’s and follower’s objectives with chosen weights does not usually enforce that the follower independently optimises. A weighted objective can describe cooperation, bargaining or a central social planner, but those interpretations require their own justification. It does not automatically reproduce a hierarchy.
Similarly, minimising the follower’s objective first and then improving the leader’s objective globally is not generally the same as letting the leader change the follower’s problem. The follower’s optimal solution depends on the leader’s chosen price. It is a family of lower problems, one for each admissible upper decision.
There are special structures in which a simpler reformulation is valid. The important habit is to prove the relationship rather than infer it from similar notation. A reformulation should preserve the intended feasible pairs and objective values, or be clearly labelled as a relaxation, approximation or alternative decision concept.
When reviewing a proposed solution, hold the leader decision fixed and solve the follower again. In the false solution above, this single check changes y from four to zero and exposes the error immediately. Independent follower verification is often the most informative first test of a bilevel computation.
CHAPTER 5 OF 14 · Handle hierarchy, ties and optimality
5. Treat ties as a substantive modelling choice
The previous customer problem has a unique optimum. Many lower problems do not. Suppose a follower chooses y∈[0,1] to minimise (1−x)y, where the leader selects x∈[0,2]. For x<1, the unique follower optimum is y=0. For x>1, it is y=1. At x=1, every y in [0,1] is optimal.
Let the leader maximise y−x/2. At x=1, the leader prefers y=1, but the follower is indifferent. An optimistic formulation lets the leader benefit from the most favourable lower optimum. A pessimistic formulation evaluates the least favourable lower optimum. Neither convention should be hidden inside a solver’s arbitrary choice.
Calculate the difference at the tie
Under optimistic selection, x=1,y=1 gives leader value 0.5 and is optimal. Under pessimistic selection, x=1 gives y=0 and value −0.5. For x>1, the value is 1−x/2, which approaches 0.5 as x decreases towards one but never reaches it.
For x<1 the value is −x/2, at most zero. Consequently, the pessimistic problem has supremum 0.5 but no maximiser on this feasible interval. This simple discontinuity shows why existence of a best decision cannot always be assumed merely because x lies in a closed bounded interval.
A tie-breaking rule can sometimes be observed or designed: a contractual priority, a deterministic dispatch convention or a stable algorithmic rule. Then model that rule directly. If it is unknown, report the range of possible leader outcomes or justify an uncertainty model for selection.
Optimistic does not mean generally cheerful, and pessimistic does not mean the follower deliberately sacrifices its own objective to hurt the leader. Both consider follower-optimal decisions; the distinction concerns which of those equally good responses is selected. Near-optimal behaviour introduces a further uncertainty set, discussed later.
CHAPTER 6 OF 14 · Handle hierarchy, ties and optimality
6. Use optimality conditions only when they characterise the follower
For the convex customer problem, introduce nonnegative multipliers μ for −y≤0 and ν for y−4≤0. The Karush–Kuhn–Tucker conditions are stationarity y+p−6−μ+ν=0, primal feasibility 0≤y≤4, dual feasibility μ,ν≥0, and complementarity μy=0 and ν(4−y)=0.
At p=3.5,y=2.5, both bounds are inactive, so μ=ν=0 and stationarity holds. At p=1,y=4, the upper bound is active: μ=0 and ν=1 satisfy stationarity. This multiplier records the marginal pressure to buy beyond the permitted cap.
Why the convex assumptions matter
Here the lower objective is differentiable and strictly convex, the constraints are affine, and a strictly feasible point exists. The conditions characterise the unique global lower optimum. Replacing the lower problem with these conditions therefore preserves its response in this example.
The Stanford convex optimisation text develops convex optimality and constraint qualifications. Applying those results requires checking the lower problem for each relevant leader decision; a convenient formula does not remove those assumptions.
Now change the lower objective to f(y)=(y²−1)² on the real line. Its derivative vanishes at y=−1,0,1. The two outer points attain the global minimum zero, while y=0 has value one and is a local maximum. If a leader minimises y², a stationarity-only replacement incorrectly permits y=0 and reports zero instead of the true value one.
For nonconvex lower problems, necessary stationarity conditions generally include points that are not global lower optima. A local lower solver can create the same problem. State whether the model assumes a global best response, a local equilibrium or the output of a specified algorithm, and choose verification methods that match that interpretation.
CHAPTER 7 OF 14 · Reformulate and differentiate
7. Understand complementarity, bounds and reformulation risks
Complementarity says that a nonnegative multiplier and its associated nonnegative slack cannot both be positive. In the customer example, ν(4−y)=0 means either the upper quantity bound is tight or its multiplier is zero. It creates a logical alternative inside an otherwise smooth-looking system.
One mixed-integer encoding for nonnegative variables a,b with ab=0 uses a≤Mₐz and b≤Mᵦ(1−z), where z is binary. This is equivalent only when Mₐ and Mᵦ are valid upper bounds for the variables in the intended formulation.
A large number is not a proof
If a bound is too small, valid responses can be removed. For example, the customer at p=0 has y=4 and upper-bound multiplier ν=2. Artificially bounding ν by one would exclude that correct response. Increasing a bound without justification may avoid that particular exclusion while worsening numerical conditioning or the relaxation.
A solver certificate for the encoded mixed-integer problem certifies that encoded model. It does not prove that an arbitrary bound preserved the original bilevel feasible set. Record how bounds were derived and test critical responses against them.
Other approaches include value-function constraints, primal–dual reformulations for suitable lower problems, specialised branching and direct nested solution methods. Each exploits different structure. Continuous convex followers, integer followers and nonconvex followers should not be treated as interchangeable simply because all are called lower-level problems.
A practical comparison should examine whether the method returns a feasible follower response, whether it establishes global or local optimality, how it handles ties, and how much computational work is needed. An algorithm that finds a useful candidate may be valuable without proving a global optimum, provided the limitation is visible and the candidate is independently checked.
Check a follower who must choose a whole unit
Suppose a buyer can purchase either zero or one indivisible item, values the item at five, and pays price p. The buyer minimises (p−5)y over y∈{0,1}. Its response is one for p<5, zero for p>5, and either choice at p=5. A supplier with unit cost two earns (p−2)y.
With favourable tie-breaking and a price interval containing five, the supplier can charge five, sell one and earn three. With unfavourable tie-breaking, charging five produces no sale. Prices just below five produce profits approaching three but do not attain three when arbitrary real prices are allowed. The exact response and tie convention both matter.
Now restrict prices to whole cents. Under unfavourable tie-breaking the best selling price is 4.99, giving profit 2.99, provided the admissible price range includes it. This small implementation detail restores an attained best price. It also shows why replacing a physical price grid by a continuous interval can change the existence and interpretation of a solution.
A derivative of the lower objective with respect to y does not characterise the binary follower. The variable cannot move through infinitesimal quantities. Enumerating its two choices gives an exact response immediately; larger integer followers may require specialised discrete optimisation. Relaxing y to [0,1] can be useful for a bound, but any resulting fractional action needs a valid physical interpretation before it is issued as a decision.
The lesson is not that discrete followers always make an application impossible. It is that the domain is part of the model. A route, machine or contract selected as a whole is different from a divisible flow or quantity. Preserve that distinction when choosing a reformulation, and verify the lower response in its original domain.
CHAPTER 8 OF 14 · Reformulate and differentiate
8. Differentiate a response when the required smoothness exists
Let a lower problem be unconstrained and have a locally unique smooth solution y*(x) satisfying ∇ᵧf(x,y*)=0. If its Hessian with respect to y is nonsingular, differentiating that equation gives a linear system for the response derivative: fᵧᵧ dy*/dx+fᵧₓ=0.
For an upper objective F(x,y*(x)), the total derivative includes both direct dependence on x and the effect through y*. Ignoring the latter treats the follower as frozen and usually computes the wrong gradient for the composed objective.
Derive a scalar hyperparameter example
Take lower training loss f(λ,w)=(w−4)²/2+λw²/2, with λ≥0. The unique trained weight is w*(λ)=4/(1+λ). Let validation loss be J(λ)=(w*(λ)−1)²/2. The best λ is three, which gives w*=1 and validation loss zero.
Implicit differentiation gives dw*/dλ=−4/(1+λ)². At λ=1, the trained weight is two, its derivative is −1 and the validation derivative is (2−1)(−1)=−1. Increasing λ slightly therefore decreases validation loss at that point.
These numbers can be checked by substitution and finite differences. They do not establish that a real learning problem has a unique smooth global minimiser. Multiple minima, singular curvature, constraints or changing active sets can invalidate this simple derivative formula or require a different sensitivity analysis.
Research on hypergradient computation studies approximation methods under explicit mathematical conditions. Its usefulness lies partly in making those conditions visible. A numerical gradient is not automatically the derivative of the ideal bilevel objective just because automatic differentiation produced it.
CHAPTER 9 OF 14 · Reformulate and differentiate
9. Separate finite training from exact lower optimisation
Suppose the scalar training problem is solved by gradient descent from w₀=0 with step size one quarter. One update gives w₁=1 for every λ, because the regularisation term is zero at the starting point. The one-step validation loss is therefore zero for every λ, and its derivative with respect to λ is zero.
At λ=1, however, the exact trained solution is w*=2, with validation loss one half and derivative −1. Differentiating through one training step is correct for that one-step procedure and incorrect as a claim about the fully optimised response.
Compute the second update
The update is wₜ₊₁=wₜ−0.25[(1+λ)wₜ−4]. Starting from w₁=1 gives w₂=1.75−0.25λ. At λ=1, w₂=1.5, validation loss is 0.125 and its derivative is (1.5−1)(−0.25)=−0.125.
The two-step derivative differs from both the one-step derivative and the exact derivative. There is no contradiction: the three objectives evaluate different lower procedures. If deployment intentionally uses two steps, the finite procedure may be the correct object to optimise. If the claim concerns convergence to a lower optimum, approximation error must be assessed.
The primary paper on bilevel programming for hyperparameter optimisation and meta-learning develops an approach that explicitly accounts for inner optimisation dynamics and conditions for approximation. The local teaching calculation above illustrates why the distinction matters before any large model is involved.
The existing specialist guide to inner training, hypergradients and validation boundaries explores the implementation in more depth. Keep training data, validation data and final test evidence separate: repeatedly selecting parameters using validation outcomes can overfit the outer problem too.
CHAPTER 10 OF 14 · Check kinks, uncertainty and validity
10. Expect kinks when constraints become active
Consider a lower problem minimising (y−x)²/2 subject to y≥0. Its solution is y*(x)=max(0,x). This response is continuous but has a kink at zero. The derivative is zero for x<0 and one for x>0, with no ordinary derivative at the switching point.
Let the upper objective be (y−1)²+0.1x². For x≤0 it becomes 1+0.1x², minimised at zero within that region. For x≥0 it becomes (x−1)²+0.1x², whose derivative is 2.2x−2. The candidate x=10/11 has value 1/11 and beats the value one at zero.
Verify both sides of a switching point
The upper derivative approaches zero from the left at x=0 and minus two from the right. A derivative computed using only the inactive or active branch can be misleading at the boundary. The correct analysis considers the piecewise structure and compares feasible candidates.
Finite-difference checks must also respect the question being tested. A central difference that crosses a kink estimates a blended slope, not a smooth derivative that actually exists there. One-sided checks can reveal the two regimes and help diagnose why an implicit-gradient formula disagrees with numerical perturbations.
In higher dimensions, active-set changes can create more complicated response geometry. Strong convexity of the lower objective can secure uniqueness without making every constrained response globally differentiable. Uniqueness and smoothness are separate properties and should be checked separately.
For learning, draw the response y=max(0,x), then draw the composed upper objective. Mark the switching point and the optimum. This visual exercise explains why a method can behave well within a regime yet need special care when the identity of binding constraints changes.
CHAPTER 11 OF 14 · Check kinks, uncertainty and validity
11. Model approximate response and behavioural uncertainty honestly
An exact lower optimum is a modelling assumption. A person may use a heuristic, face information limits or accept a nearly best alternative. A computation may stop at a tolerance. These are different reasons for approximate response, but each requires more than asserting that the lower objective is nearly optimal.
At price 3.5 in the customer example, the lower objective gap from its optimum is exactly (y−2.5)²/2. If the follower accepts any quantity within objective tolerance ε=0.02, its allowable quantities are [2.3,2.7]. The supplier’s profit then ranges from 2.5·2.3=5.75 to 2.5·2.7=6.75.
Translate objective tolerance into outcome uncertainty
The lower objective tolerance looks small, yet the leader’s outcome varies by one across the allowed range. Tolerances are measured in the lower objective’s units. Their effect on quantities and upper outcomes depends on curvature, scaling and the upper objective’s sensitivity.
In this quadratic example the distance bound is √(2ε). Flatter lower objectives permit larger movements for the same objective gap. With a nonunique optimum, even zero gap can permit very different leader outcomes. Reporting a lower objective residual alone therefore does not settle upper-level reliability.
The research literature on bilevel optimisation under uncertainty distinguishes uncertainty in data, observations and response quality. A useful application should identify which of those is present rather than hiding all discrepancies inside one generic noise term.
Empirical validation is essential when the follower represents people or organisations. Compare predicted responses with observed choices under relevant conditions, including outside options and information available to the follower. A perfectly solved mathematical model cannot establish that a utility function accurately describes behaviour. If observed response is systematically different, revise the response model before refining the leader’s optimum.
CHAPTER 12 OF 14 · Check kinks, uncertainty and validity
12. Audit a candidate with independent checks
A defensible bilevel result should include the leader decision, the follower response, both objective values, the lower feasibility residuals and evidence that the follower response matches the intended solution concept. Add the global or local status of the upper optimisation and any unresolved bound gap.
For the price example, the certificate is compact: p=3.5, y=2.5, customer stationarity residual zero, both quantity slacks positive, multipliers zero, supplier profit 6.25, and comparison with the best values in the other price regimes. Each component answers a different question.
Check the lower problem independently
Hold p fixed and solve the customer problem using direct substitution or a second implementation. If the returned response improves the follower objective substantially, the proposed bilevel pair is not valid under exact best-response assumptions. If the follower has several optima, explore their upper outcomes rather than accepting the first one returned.
For a smooth hypergradient, compare the analytic result with several finite-difference step sizes away from switching points. Very large steps measure nonlocal changes; very small steps may amplify solver error and floating-point cancellation. A stable range of agreement is more informative than one coincidental match.
For numerical optimisation, separate lower-solve tolerance from upper stopping tolerance. Tightening only the outer loop while keeping a noisy or inaccurate response map may create the appearance of precision without improving the result. Record solver settings and rerun critical cases with stricter lower accuracy.
For the pricing model, a grid with increments of 0.2 contains prices 3.4 and 3.6 but misses 3.5. Both grid candidates earn 6.24, leaving a gap of 0.01 from the exact maximum 6.25. The identity Π(p)=6.25−(p−3.5)² in the interior region explains this error exactly. Grid resolution therefore affects the reported decision even when the follower is solved perfectly. In a less regular response map, an equally fine grid need not give an equally small error, so this numerical bound should not be transferred without checking the objective geometry.
Finally, validate implementation assumptions. Can the leader commit to the chosen rule? Does the follower observe it? Is the response feasible under actual capacity, timing and information? A failure in any of these questions changes the decision problem itself; it cannot be repaired merely by displaying more digits in the objective value.
CHAPTER 13 OF 14 · Transfer and continue
13. Transfer the price model to a smaller capacity
Change the customer’s maximum quantity from four to two, leaving benefit U(y)=6y−y²/2 and supplier unit cost one unchanged. The follower response becomes y*=2 for 0≤p≤4, y*=6−p for 4≤p≤6, and zero above six.
In the first region profit is 2(p−1), increasing to six at p=4. In the second region the same quadratic −p²+7p−6 applies, but its unconstrained maximum at 3.5 now lies outside that region. The quadratic is decreasing for p≥4, so its best value there is also six at p=4.
Explain the new optimum rather than copying the old one
The new global optimum is p=4,y=2, profit six. Keeping the old price 3.5 would sell two and produce profit five. The smaller capacity changes the threshold where demand leaves its cap, so the old interior solution no longer describes the relevant regime.
This is a useful changed-context test because the utility curve and unit production cost did not change. The only changed assumption is a bound. A solution method that blindly reuses the old first-order condition misses the new boundary optimum.
Now ask what happens if the supplier can pay to expand that cap. Expansion becomes an additional leader decision that changes the follower’s feasible set. Its cost must be included in the leader objective, and the lower response must be recomputed for each capacity. The original fixed-cap problem does not answer that investment question by itself.
For an independent exercise, raise unit production cost to two while returning the cap to four. Interior profit becomes (p−2)(6−p), maximised at p=4 with y=2 and profit four. The low-price capped region cannot beat it, and zero demand gives zero. State both changes clearly: this exercise changes production cost, whereas the preceding one changed usable quantity.
CHAPTER 14 OF 14 · Transfer and continue
14. Answer common questions and choose a learning route
Is every nested calculation a bilevel problem?
No. A numerical routine can call another routine without requiring its output to solve a lower optimisation problem. Conversely, a bilevel model can sometimes be reformulated and solved without an explicit nested loop. The defining feature is the response constraint, not the software’s call structure.
Does a unique follower make the whole problem easy?
No. Uniqueness removes ambiguity about which exact lower optimum is selected, but the response may still be nonlinear or nonsmooth. Composing it with the upper objective can create a difficult problem. The price example is easy because its few response regimes can be solved and compared explicitly.
Can a local lower optimum be acceptable?
Yes, if the intended response is explicitly a local optimisation process or a specified algorithm and the model reflects its initialisation and dynamics. It is not acceptable to label that result a global best response without evidence. The distinction is especially important in nonconvex learning and engineering design.
Which skills should a beginner build first?
Start with one-variable optimisation, constrained quadratics, piecewise functions and the difference between a decision and a parameter. Reproduce the customer response before solving the supplier problem. Then work through the tie example and explain why its pessimistic supremum is not attained.
For a computational exercise, build a grid of prices, solve the follower for each price, and calculate leader profit. Compare the grid result with the exact optimum and quantify the resolution error. Next deliberately flatten the hierarchy and explain why the impressive answer fails the follower check.
For a machine-learning route, derive the scalar regularisation example, compare exact and finite-step training, and check the outer derivative numerically. Preserve independent evaluation data and distinguish algorithmic performance from a claim about an exact lower optimum.
Use the linked convex optimisation text for optimality conditions, the primary hyperparameter papers for differentiation through training, and the uncertainty survey for response ambiguity. The practical finish is a model another reader can audit: who chooses, what they know, what they optimise, which responses are allowed, and what evidence supports the final decision.