A method that can explain an unknown manuscript after the fact is not yet a localisation method.
It may be a storytelling machine.
The test is harder.
If our localisation framework is genuinely informative, can it recover the known production region of manuscripts whose answers are already independently catalogued?
That is the Calibration Problem.
The principle is borrowed from a much wider scientific habit: a model should be checked against known outcomes before it is trusted in a case where the outcome is hidden. In prediction research, calibration asks whether predictions line up with observed outcomes, and external validation asks whether a model built in one setting continues to perform on independent data.
For the Voynich programme, the translation is simple:
known manuscript → hide the answer operationally → run the localisation rules → compare inferred region with catalogued region.
Why Calibration Comes Before Confidence
The Padua branch has built a sophisticated framework.
- matched controls;
- localising residue;
- coordinate stack;
- receiver fallacy;
- evidence independence;
- expected evidence;
- minimum path;
- null coordinate.
But a framework can be internally elegant and still fail in the world.
Calibration asks whether the framework knows the difference between:
- a manuscript actually produced in Padua;
- a Viennese receiver carrying Paduan-linked medicine;
- a southern-German manuscript containing similar medical capabilities;
- a Bohemian medical parchment object;
- a mixed Padua/southern-German composite;
- a manuscript whose production city should remain null.
If the method calls all six “Padua-like,” it is not localising.
It is recognising a broad medical world.
Our Known-Answer Calibration Set Already Exists
The recent object-level articles give us a natural calibration panel.
| Object | Independent catalogue coordinate | Why useful |
|---|---|---|
| Pal. lat. 1307 | Padua · 1403 | Early local Paduan medical composite. |
| Pal. lat. 1311 | Padua · 1431 | Local surgery/anatomy/pharmacy composite inside the Voynich window. |
| Pal. lat. 1299 | Vienna · 1413–1414 | Near-contemporary rival centre with anatomy, surgery, pharmacy and multilingual retrieval. |
| Pal. lat. 1305 | Southern Germany · 1422 | Medicine, alchemy, body mapping and calendar material outside Padua. |
| Pal. lat. 1291 | Bohemia · 1425 | Parchment, pharmacy, uroscopy and symbolic medical imagery outside Padua. |
| Pal. lat. 1265 | Padua + southern Germany | Tests whether the method can tolerate a mixed answer. |
| Pal. lat. 1284 | No single production city in the catalogue heading | Tests whether the method can return null instead of forcing a favourite. |
Calibration Is Not “Can We Recognise the Easy Cases?”
A weak test would choose obvious manuscripts after we already know why they are obvious.
A stronger calibration set should contain difficult neighbours.
Padua and Vienna both contain learned medicine.
Padua and southern Germany can both contain anatomy, bloodletting and alchemy.
Padua and Bohemia can both contain parchment, pharmacy and medical image systems.
That is useful because a localisation method should succeed through discriminating features, not through caricatures.
Define the Output Before Running the Test
Calibration requires a fixed output vocabulary.
- Correct region: the inferred physical-production coordinate matches the independent catalogue region at the chosen resolution.
- Correct mixed state: the method refuses to compress a genuinely mixed object into one region.
- Correct null: the method leaves the coordinate unresolved when the available evidence does not justify assignment.
- Near miss: the method recovers a broader region but not the documented city.
- False localisation: the method assigns the wrong region with confidence.
The last category matters most.
A system that confidently mistakes Vienna for Padua is more dangerous than one that returns null.
The Calibration Scorecard
| Dimension | Question |
|---|---|
| Discrimination | Can the method distinguish Padua from close rival environments? |
| Calibration | Does confidence track actual correctness, or is the method overconfident? |
| Null discipline | Can it leave genuinely unresolved cases unassigned? |
| Mixed-state discipline | Can it preserve multi-region production when the object requires it? |
| Robustness | Does the answer survive reasonable changes in feature weighting? |
| Transportability | Does the method work on manuscripts not used to design the rules? |
Do Not Calibrate on the Same Cases Used to Invent Every Rule
This is the most important warning.
If we invent the localisation rules while looking at Pal. lat. 1311 and then proudly recover Padua from Pal. lat. 1311, the test is circular.
Prediction science calls this overfitting.
Rules tuned to the development sample often perform worse on independent data.
Therefore the known manuscripts need roles:
- development set;
- calibration set;
- blind test set;
- later out-of-sample set.
No object should drift between roles after its answer becomes inconvenient.
Calibrate the Confidence, Not Only the Label
Suppose the method says:
Pal. lat. 1299: 90% Padua.
But the independent catalogue says Vienna.
That is worse than a cautious:
North-central European medical environment; city unresolved.
The method must therefore be judged on whether certainty is calibrated to actual performance.
A useful system should be confident only where its evidence families have earned confidence.
The Known-Answer Test Must Preserve the Coordinate Stack
A manuscript can have non-local text ancestry and local physical production.
So calibration should not ask only:
Did we name the city?
It should ask:
- Did we correctly separate C2 text origin from C5 physical production?
- Did we mistake a travelling author for a local production signal?
- Did we confuse later ownership with origin?
- Did we preserve mixed and null states?
Otherwise we may appear accurate for the wrong reason.
A Failure Is Valuable Calibration Data
If the method mistakes Vienna for Padua, do not hide the manuscript.
Ask why.
- Did Paduan-linked authors receive too much weight?
- Did medical integration receive too much weight?
- Were codicological features underweighted?
- Was the null threshold too low?
- Were correlated clues counted separately?
The failure tells us which feature weights are not calibrated.
That is much more useful than protecting a perfect-looking method.
The eduKate Calibration Protocol
- Assemble a known-origin manuscript corpus from institutional catalogues.
- Define the regional resolution before scoring.
- Freeze the evidence features and weights.
- Separate development objects from calibration objects.
- Remove explicit shelfmark, city and provenance labels from the working packets.
- Run the Coordinate Stack on each object.
- Record predicted region, confidence, mixed/null state and competing candidates.
- Reveal the independent catalogue answer.
- Measure correct assignments, near misses, false localisations and null performance.
- Revise the method only in a new version, preserving the failed version’s receipt.
- After revision, validate again on fresh objects.
What Counts as Success?
Not 100% accuracy.
Historical localisation is noisy, catalogue attributions vary in precision, and some manuscripts genuinely deserve mixed or uncertain coordinates.
Success means something more useful:
- the system distinguishes close rivals better than generic resemblance;
- confidence decreases when evidence is weak;
- false Padua calls remain acceptably low;
- null and mixed states are used correctly;
- performance survives fresh manuscripts.
Why This Matters Before We Score the Voynich
Beinecke MS 408 has no answer key for us to reveal.
That means we cannot discover whether our localisation framework is biased by testing only on Voynich.
Known manuscripts provide the missing laboratory.
If the method cannot recover known manuscript geographies with disciplined uncertainty, it has not earned the right to assign an unknown one.
Research Sources
- Evaluation of clinical prediction models: development to external validation — calibration and discrimination as distinct validation dimensions.
- NCBI Bookshelf — Prediction Modeling Methodology — cross-validation, data leakage and external validation.
- Prediction models need appropriate internal, internal-external and external validation.
- Voynich and Padua — The Coordinate Stack
- Voynich and Padua — The Null Coordinate
- Voynich Research Library
The Final Idea
The Voynich has no visible answer key.
Known manuscripts do.
Before we trust our compass in the fog, we should test whether it points correctly when the landmarks are known.