A method can achieve many apparent successes simply by saying “Padua” too often.
If the test set is full of medical, astrological and technical manuscripts, Padua is a plausible answer surprisingly often.
So a framework that recognises broad compatibility may look powerful while failing the real task.
The False Positive Problem asks: among manuscripts known not to be Paduan at the target coordinate, how often does our localisation system incorrectly identify them as Padua?
That question is more important than the number of Paduan examples the method can explain.
What a False Positive Means Here
NIST defines a false positive rate in a classification context as the proportion of known non-matches incorrectly determined to be identifications.
Translated into our localisation problem:
known non-Padua object → method says Padua = false positive.
But we need to be precise about the coordinate.
A Viennese manuscript containing Paduan-linked medicine is not a false positive if the question is “does this object contain Paduan intellectual transmission?”
It is a false positive if the question is “was this physical codicological unit produced in Padua?” and the method answers yes.
Compatibility Is Not Identification
This distinction needs to be built into every score.
- Compatible with Padua: nothing observed rules Padua out.
- Padua-like: several features occur in known Paduan material.
- Padua preferred: Padua explains the current evidence better than specified alternatives.
- Padua identified: evidence crosses a pre-defined threshold for the target coordinate.
A system that turns every “compatible” into “identified” will have a catastrophic false positive rate.
Our Hostile Controls Are Designed for This Test
The recent object ladder gives us exactly the kinds of non-Paduan manuscripts that should tempt the method.
| Known control | Why it may trigger a false Padua call |
|---|---|
| Pal. lat. 1299 · Vienna 1413–14 | Anatomy, surgery, pharmacy, diagnostic medicine and multilingual retrieval. |
| Pal. lat. 1305 · Southern Germany 1422 | Surgery, anatomy, alchemical recipes, bloodletting, body mapping and calendrical medicine. |
| Pal. lat. 1291 · Bohemia 1425 | Parchment, pharmacy, uroscopy, learned medicine and symbolic medical imagery. |
| Laubach 1425 | Astronomical, astrological and medical mixture inside the relevant period. |
If these controls repeatedly score as Padua, the problem is not that Europe was secretly Paduan.
The problem is that our features are too generic.
False Positives Reveal Which Features Carry Too Much Weight
A false Padua call should trigger a feature audit.
- Did “medicine + astrology” score too highly?
- Did the presence of a Paduan-linked author leak into physical-production scoring?
- Did generic parchment use receive geographic weight?
- Did body imagery receive too much weight?
- Did several correlated features get counted separately?
- Was the null threshold too permissive?
The error is therefore diagnostic.
It shows where the model is confusing capability with locality.
The Denominator Matters
“We made two false Padua calls” means little without knowing how many non-Paduan objects were tested.
If two out of four non-Paduan controls are called Padua, the method is unusable.
If two out of two hundred close controls are called Padua, the picture is different.
Therefore the core rate is:
False Padua Rate = false Padua assignments ÷ known non-Padua test objects.
The exact statistic can be refined by coordinate and confidence threshold, but the denominator must always be visible.
Confidence-Weighted False Positives Are Worse
A wrong low-confidence suggestion is less damaging than a wrong high-confidence identification.
So record:
- false Padua at low confidence;
- false Padua at medium confidence;
- false Padua above assignment threshold.
The final category should be rare.
If it is not, the assignment threshold is too low or the evidence weights are miscalibrated.
False Positives and the Localising Residue
The Localising Residue removed features widely shared across rival environments.
False-positive testing is how we check whether the residue is actually localising.
If a supposedly high-value residue feature is common in non-Paduan controls, it does not belong in the high-weight class.
It gets downgraded.
The Receiver Fallacy Is a Major Source of False Positives
The Receiver Fallacy can create false Padua assignments whenever a non-Paduan host contains Paduan-linked texts.
Pal. lat. 1268 in Vienna is the obvious test.
If Santasofia, Tossignano and other Paduan-linked authorities overpower the Viennese physical-production evidence, the coordinate weighting is wrong.
C2 is contaminating C5.
False Positives Can Be Symmetric
Padua is not the only label that can be overcalled.
A method may call too many manuscripts German because several scribal traits look transalpine.
It may call too many books Viennese because mathematical astronomy is over-weighted.
It may call too many books north Italian because herbal imagery dominates the score.
Therefore every candidate region needs its own false-positive matrix.
Build a Confusion Matrix, Not a Victory List
A useful validation table should show all outcomes.
| Known region | Predicted Padua | Predicted Vienna | Predicted Germany | Predicted Bohemia | Null / mixed |
|---|---|---|---|---|---|
| Padua | Correct | Miss | Miss | Miss | Conservative miss |
| Vienna | False Padua | Correct | Miss | Miss | Conservative |
| Southern Germany | False Padua | Miss | Correct | Miss | Conservative |
| Bohemia | False Padua | Miss | Miss | Correct | Conservative |
This stops selective reporting.
We see not only when the method wins, but how it fails.
Set an Error Budget Before Voynich
Before applying the framework to Beinecke MS 408, define an acceptable false-positive budget.
For example:
- no high-confidence Padua assignment if known close controls frequently trigger the same threshold;
- raise the threshold until false identification becomes rare;
- prefer null over a high false-positive regime.
The exact numerical threshold should emerge from the calibration corpus rather than be invented rhetorically here.
The principle is fixed first.
The eduKate False-Positive Protocol
- Define the target coordinate and regional classes.
- Assemble known non-Paduan manuscripts close in date and function.
- Run the frozen localisation method blind.
- Record every Padua score, not only the final label.
- Reveal the known catalogue regions.
- Calculate false Padua assignments at each confidence threshold.
- Identify features common to the false positives.
- Reduce or restructure weights for non-discriminating features.
- Re-test only on fresh holdout objects.
- Publish the error matrix with the successful classifications.
What a Good Result Would Look Like
A useful localisation method will not necessarily name every object.
It will know when not to name Padua.
That is the harder achievement.
A Padua hypothesis becomes scientifically credible not when it can recognise Padua everywhere, but when it reliably refuses Padua where Padua is not the answer.
Research Sources
- NIST — False Positive Rate.
- Evaluation of clinical prediction models — discrimination and calibration.
- The Calibration Problem
- The Blind Localisation Test
- The Receiver Fallacy
- Voynich Research Library
The Final Idea
The question is not how many things we can make look Paduan.
The question is how rarely we call something Paduan when it is not.