A model can be correct only because we chose the weights that made it correct.
Codicology: 30%.
Palaeography: 20%.
Image ancestry: 15%.
Documentary provenance: 25%.
Astronomical parameters: 10%.
Change those numbers and the winner may change.
The Weight Sensitivity Problem asks whether the localisation conclusion survives reasonable changes in the uncertain choices used to combine evidence.
If Padua wins only when one evidence family receives a very specific weight, the conclusion is fragile.
If Padua remains preferred across a broad defensible range of weights, the conclusion is more robust.
Sensitivity Analysis Is About Robustness
Sensitivity analysis is widely used to study how uncertain model inputs affect model outputs. In model evaluation, one of its central purposes is to ask whether a conclusion depends on a narrow parameter choice or survives a wider range of plausible assumptions.
The Voynich localisation framework contains several uncertain inputs even after validation.
- how much weight codicology deserves relative to palaeography;
- how strongly documentary absence should reduce confidence;
- how much correlated evidence should be discounted;
- how expensive an extra geographic handoff should be;
- where the null threshold should sit;
- how wide the plausible candidate region should be.
Those are exactly the kinds of choices that need perturbation.
The First Test: One-at-a-Time Weight Changes
Start simply.
Increase one evidence family while holding the others proportionally adjusted.
Then decrease it.
Ask when the regional ranking changes.
| Evidence family | Stress question |
|---|---|
| Codicology | Does Padua still lead if physical-production evidence receives less weight? |
| Palaeography | Does a transalpine-looking scribal signal overturn the result when given more weight? |
| Image ancestry | Does the result depend heavily on herbal or diagram resemblance? |
| Documentary provenance | How much would a future secure ownership bridge alter the ranking? |
| Astronomical parameters | Can one local parameter dominate the entire model? |
| Expected-evidence penalty | Does Padua survive stronger penalties for predictions that fail? |
| Path complexity | Does a multi-region Padua route survive higher handoff costs? |
The Second Test: Joint Perturbation
One-at-a-time changes can miss interactions.
Suppose codicology weight falls slightly while expected-evidence penalties rise and the null threshold becomes stricter.
Each change alone may leave Padua first.
Together they may return null.
So the stronger test samples combinations of defensible weights rather than moving one slider at a time.
The output should be a distribution of outcomes:
- Padua first;
- Vienna first;
- Germany first;
- another region first;
- null;
- mixed path.
The percentage of reasonable weight sets producing each outcome is more informative than one chosen score.
Robustness Is Not the Same as Certainty
Suppose Padua wins under 80% of plausible weight combinations.
That does not prove Padua.
It tells us that the preference is not an artefact of one narrow weighting scheme.
Likewise, if Padua wins under 51% and Vienna under 45%, the result is unstable even if Padua technically ranks first in the baseline run.
A robust preference is one that survives reasonable uncertainty in how the evidence is combined.
The Null Coordinate Must Participate in Sensitivity Analysis
Do not force every perturbation to produce a winner.
If reasonable weight changes frequently push the result below the assignment threshold, that is important.
The correct sensitivity output may be:
Padua leads the candidate ranking, but the physical-production coordinate is unstable and frequently returns null under plausible weighting.
That is a stronger statement than pretending one baseline weight table settled the question.
Weight Ranges Must Be Justified Before Seeing the Voynich Result
Do not choose a huge range for a feature that hurts Padua and a tiny range for a feature that helps it.
The Preregistration Freeze must include the allowed sensitivity ranges.
Those ranges should come from:
- known-manuscript calibration;
- measurement uncertainty;
- inter-analyst disagreement;
- historical uncertainty in catalogue attributions;
- reasonable alternative modelling choices.
The stress test then uses the frozen ranges.
The Weight Cliff Is a Warning Signal
A useful diagnostic is the point at which the winner changes.
Suppose Padua wins unless codicology weight falls below 27%, at which point Vienna wins sharply.
That 27% boundary is a weight cliff.
We then ask:
- Why is 27% the tipping point?
- Is there independent reason to believe codicology deserves more than 27%?
- Which observations are responsible?
- Would a new measurement move the cliff?
The cliff tells us where future evidence has the greatest value.
Sensitivity Can Expose Hidden Double Counting
If tiny changes in image weight dramatically swing the result, we should inspect whether several correlated image features entered separately.
The Evidence Independence Problem may be hiding underneath apparent weight sensitivity.
Weight instability is therefore not only a numerical problem.
It can diagnose structural flaws in the evidence model.
Sensitivity Must Include Path Penalties
The Minimum Path Principle assigns a cost to extra production and transmission stages.
That cost is uncertain too.
If a Padua → Vienna path wins only when handoffs are almost free, the path is fragile.
If it survives substantial handoff penalties because independent object evidence requires both stages, the path is stronger.
Sensitivity Must Include Missing-Evidence Penalties
Expected-but-missing evidence is another adjustable component.
If Padua wins only when the missing ownership bridge, absent local script signal or failed astronomical parameter carries almost no penalty, that should be visible.
The result may still be Padua.
But readers should know what the conclusion is paying to ignore.
A Robustness Envelope Is Better Than One Number
The final report should not say only:
Padua score = 0.78.
It should say something like:
Across the preregistered plausible weight space, Padua ranks first in 72% of runs, Vienna in 18%, other regions in 4%, and the coordinate returns null in 6%. Padua remains first under all one-at-a-time perturbations except when codicological weight falls below the validated lower bound.
Those numbers are illustrative, not current Voynich results.
The structure is the point.
The eduKate Weight-Sensitivity Protocol
- Freeze baseline weights from known-manuscript validation.
- Freeze defensible lower and upper bounds for each evidence family.
- Run one-at-a-time perturbations.
- Run joint perturbations across the plausible weight space.
- Include path costs, dependence discounts, expected-evidence penalties and null threshold.
- Record candidate rankings for every run.
- Identify weight cliffs and ranking reversals.
- Report the robustness envelope, not only the baseline winner.
- Do not narrow the allowed ranges after seeing the Voynich outcome.
- Use instability to identify which new evidence would be most informative.
What Would Count as a Warning?
- the winner changes under tiny plausible weight changes;
- one subjective evidence family controls the result;
- null appears frequently while the baseline report claims certainty;
- the preferred path survives only when handoff penalties are near zero;
- missing-evidence penalties must be minimised for the favourite to win;
- reasonable analysts choosing defensible weights reach different regions.
Research Sources
- JASSS — Which Sensitivity Analysis Method Should I Use for My Agent-Based Model? — robustness to parameter changes and uncertainty.
- JASSS — sensitivity analysis as evaluation of how model inputs affect output variability.
- How Sensitivity Analysis Works | eduKateSG.
- Voynich and Padua — The Preregistration Freeze.
- Voynich and Padua — The Minimum Path Principle.
- Voynich Research Library.
The Final Idea
A localisation result is more trustworthy when it does not depend on us balancing the scales with a jeweller’s precision.
If Padua is truly supported by several independent evidence systems, it should survive reasonable uncertainty in how heavily each one is weighted.