Voynich creates an uncomfortable question for the age of AI.
What if a bigger model is not automatically a better model of the manuscript?
That sounds almost obvious.
It is still easy to forget.
A model with millions of parameters can learn more patterns.
It can memorise more contexts.
It can generate smoother-looking output.
But historical explanation is not a beauty contest for generated text.
In Yang Ou’s May 2026 Voynich Statistical Turing Test preprint, model complexity did not improve fit monotonically. A GPT-2 model fine-tuned on the manuscript passed only one of the eight benchmark criteria. The strongest model in that experiment was much simpler: a section-stratified interpolated bigram sampler, which passed seven.
That result does not prove “simple models are better” in general.
It proves something more useful.
More parameters do not rescue the wrong representation of the problem.
Quick Read
- A May 2026 preprint benchmarked eighteen Voynich generative-model variants across eight quantitative properties.
- Model families included uniform sampling, unigram models, Cardan-grille simulation, character and word Markov chains, an HMM, LSTM, fine-tuned GPT-2 and section-stratified bigram models.
- The tested GPT-2 variant passed only one of eight benchmark criteria.
- The tested LSTM also passed only one of eight.
- The strongest model was a section-stratified interpolated bigram sampler, which passed seven of eight.
- The paper concludes that increasing parametric complexity did not monotonically improve statistical fidelity in the tested range.
- This does not mean GPT-2 is intrinsically incapable of learning Voynich-like structure.
- It means that under this data, training setup and benchmark, adding model capacity did not substitute for explicit section structure.
- The result suggests that the important complexity may be structural rather than merely parametric.
- A model can imitate local probabilities well while missing manuscript-scale partitions such as sections, Currier regimes, positional vocabulary and other hidden states.
- A model that predicts well is not automatically a historical explanation.
Parametric Complexity and Structural Complexity Are Different
A neural model becomes more powerful by adding parameters, layers, hidden states or attention capacity.
That is parametric complexity.
But the data itself can contain a different kind of complexity.
Section A uses one vocabulary regime.
Section B uses another.
Labels behave differently from prose.
Line beginnings behave differently from line middles.
That is structural complexity.
A small model with the right partition can outperform a large model asked to learn everything from a flattened corpus.
This is not surprising once stated clearly.
If the manuscript has several operating states, a model that knows the state can need fewer parameters than a model forced to infer the state indirectly from every character.
Why GPT-2 Can Lose to a Bigram
A bigram model is simple.
It asks what tends to follow what.
A section-stratified bigram model asks the same question separately inside different manuscript sections.
That one extra structural decision changes the problem dramatically.
GPT-2, by contrast, is designed to infer structure from context through many parameters.
But the Voynich corpus is tiny by modern language-model standards.
Its tokens are uncertain.
Its page order is historically complicated.
Its “words” may not all be words.
A large model can therefore spend capacity modelling transcription quirks, local repetition and accidental corpus boundaries instead of the particular global statistics chosen by the benchmark.
The benchmark result is not “bigram intelligence beats GPT intelligence”.
It is:
explicit knowledge of the manuscript’s partition can matter more than generic capacity.
Small Data Changes Everything
Modern language models are usually trained on enormous corpora.
Voynich gives us one manuscript.
A few tens of thousands of token-like forms.
One object.
One uncertain transcription tradition.
This creates a brutal data-to-parameter ratio.
Large models can fit the corpus easily.
The hard question is whether what they learn generalises to manuscript properties not used in training.
That is why training loss alone is weak evidence.
A model can predict the next glyph well and still fail:
- Zipf slope;
- vocabulary growth;
- Currier replication;
- positional vocabulary divergence;
- edge coupling;
- hapax rate;
- directional dissociation.
Prediction Is Not Imitation
A model can be excellent at predicting the next character in the real Voynich corpus.
That does not guarantee that long generated samples have the same global statistics.
This distinction is familiar in machine learning.
Teacher-forced prediction sees the correct history.
Free generation sees its own previous outputs.
Small errors accumulate.
The model drifts into states underrepresented in training.
Therefore:
good next-glyph perplexity and good manuscript-scale imitation are different achievements.
Imitation Is Not Explanation
Now go one step further.
Suppose GPT-2 eventually generated perfect Voynich-like text.
Every statistic matches.
Every section looks right.
Every edge constraint survives.
Would that explain the fifteenth-century manuscript?
No.
It would prove that modern neural computation can imitate the observed distribution.
Historical explanation needs a mechanism available to the makers.
The distinction is fundamental:
- prediction: guess what comes next in observed text;
- imitation: generate new text with similar statistics;
- interpolation: produce plausible combinations inside learned regions;
- historical explanation: identify the actual process that created the surviving object.
AI can be outstanding at the first three and tell us nothing directly about the fourth.
Why the Section Model Is So Informative
The section-stratified model’s strong benchmark performance suggests the manuscript contains large-scale state differences that matter statistically.
That does not mean conventional visual sections are the final hidden states.
They may proxy:
- Currier regime;
- scribe;
- topic;
- document role;
- source exemplar;
- production chronology.
The result therefore supports a research strategy more than a historical conclusion.
Do not flatten Voynich before modelling it.
More Capacity Can Hide the Wrong Question
Large models are forgiving.
If the representation is poor, they can often compensate partially.
If token boundaries are wrong, attention can learn across them.
If several regimes are mixed, embeddings can separate some of them implicitly.
This can create impressive predictive performance while leaving the scientific representation uncorrected.
A simpler model fails more visibly.
That can be an advantage.
The failure tells us which structural feature must be represented explicitly.
In an undeciphered manuscript, interpretability of failure can be more valuable than flexibility of fit.
The Overfitting Problem
Voynich is a small corpus with many measurable quirks.
A sufficiently flexible model can learn those quirks.
That does not mean it has learned the mechanism.
Overfitting can happen at several levels:
- memorising specific sequences;
- memorising section vocabulary;
- memorising transcription conventions;
- tuning hyperparameters against the same benchmark repeatedly;
- choosing evaluation metrics after seeing the outputs.
The antidote is not merely regularisation.
It is independent evidence.
New Edge and Unit-Scale results are valuable partly because models developed before those papers could not have been tuned specifically to pass them.
The Historical Mechanism Should Be Compressible
A fifteenth-century production process had finite human complexity.
Scribes could learn sophisticated systems.
Ciphers could be elaborate.
Abbreviation traditions could be dense.
But the real process had to be executable by real people with real tools.
This creates a strong prior:
if a model needs millions of invisible degrees of freedom to imitate Voynich, it may be a good emulator and a poor historical explanation.
The historical mechanism should probably admit a much more compact description.
But Simplicity Is Not Automatically Truth
Occam’s razor is often abused.
A simpler model is not better if it fails real evidence.
The section-stratified bigram model’s seven-of-eight performance makes it a strong control.
It does not make it the historical generator.
A real mechanism can be more complex than the minimum model required to reproduce a benchmark.
The useful principle is:
use the least complexity needed to explain the independent evidence, then stop adding freedom unless a new observation demands it.
A Better AI Experiment for Voynich
Instead of asking which model generates the prettiest Voynich-like text, design a hierarchy.
- Start with unigram and bigram controls.
- Add section state explicitly.
- Add Currier state explicitly.
- Add line-position state.
- Add graded boundaries.
- Add learned multi-symbol units.
- Only then increase neural capacity.
At every step, measure which independent constraints improve.
This tells us what kind of complexity the manuscript actually demands.
If a transformer improves only after being given the same structural variables explicitly, the variables—not the transformer—are the scientific discovery.
The Model Should Be Asked to Fail Out of Sample
Train on some quires.
Test on others.
Train on Currier A.
Predict what changes in B.
Train on ordinary prose.
Test labels and circular loci.
A model that only performs when trained on the exact distribution it is asked to imitate tells us little about transfer.
A historical mechanism should generalise because the same makers and conventions operated across more than one local passage.
What the Model Complexity Problem Does Not Prove
- It does not prove GPT-2 is generally worse than bigrams.
- It does not prove neural networks cannot model Voynich.
- It does not prove simpler mechanisms created the manuscript.
- It does not prove the May 2026 benchmark is complete.
- It does not prove section stratification is the true historical variable.
- It does not equate parameter count with historical implausibility automatically.
What It Can Give Us
- A warning against treating model size as evidentiary strength.
- A distinction between parametric and structural complexity.
- A reason to model known manuscript states explicitly.
- A framework for separating prediction, imitation and historical explanation.
- A preference for interpretable failures and held-out validation.
Reader Checklist
- How large is the training corpus?
- How many parameters does the model have?
- What structure was given explicitly?
- What structure had to be inferred?
- Is evaluation teacher-forced prediction or free generation?
- Which metrics are independent of training?
- Does the model generalise across quires?
- Does it reproduce section and Currier structure?
- Does increased complexity actually improve the full joint benchmark?
- Is statistical imitation being confused with a historically executable mechanism?
Research Foundations
- Yang Ou — A Statistical Turing Test for the Voynich Manuscript (May 2026 preprint).
- The Joint Benchmark Problem.
- The Directional Dissociation Problem.
- The Unit-Scale Problem.
The Final Idea
Voynich does not care how impressive the model is.
It cares whether the model reproduces what the manuscript actually does.
A huge model can hide a bad representation.
A small model can expose the structural variable we forgot to include.
The best model of Voynich may not be the one with the most intelligence inside it. It may be the one that asks the manuscript the right question with the least unnecessary freedom.