eduKateSG Learning Node Series · 0075

How Knowledge Tracing Works | Estimate What a Learner Knows From the Pattern of Their Attempts

A learner answers a fraction question correctly.

Do they know fractions?

Not necessarily. The question may have been easy. The learner may have guessed. The method may still depend on a recent example. One correct answer is evidence, not a complete state description.

Then another response arrives. And another. Some are correct, some wrong, some fast, some delayed, some attached to related skills. The sequence begins to tell a story.

Knowledge tracing is the attempt to estimate a learner’s changing knowledge state from the history of their interactions with learning tasks.

The 50-Second Read

• Knowledge tracing does not directly observe knowledge; it infers a hidden state from behaviour.
• Classic Bayesian Knowledge Tracing models whether a skill is learned while allowing for guesses and slips.
• Modern models can incorporate richer sequences, relationships among skills, forgetting and learner differences.
• Deep Knowledge Tracing introduced recurrent neural networks to predict future responses from interaction sequences.
• More predictive models are not automatically more interpretable or educationally useful.
• Good tracing depends on good skill labels, task design and data quality.
• A probability of success is not the same thing as a diagnosis of misconception.
• The strongest use is decision support: what to review, what to practise, what evidence to gather next and when a human should look more closely.

Canonical Owner Boundary

This page owns knowledge tracing: computational inference about changing learner knowledge from sequential interaction data. How Progress Tracking Works owns human-readable longitudinal evidence of learner change. How Mastery Learning Works owns readiness decisions about progression. How Learning Dependencies Work owns prerequisite structure. Knowledge tracing asks a narrower machine-model question: what hidden learning state best explains the response sequence we can observe?

1. Knowledge Is Hidden

A teacher can see answers, working, hesitation, explanations and questions. The teacher cannot directly see “knowledge” itself.

Knowledge tracing formalises this problem. The system observes interaction data and estimates a latent state underneath it.

That distinction prevents a common error: treating the model output as though it were the learner. It is an estimate about the learner.

2. The Sequence Matters

Suppose a student answers the pattern wrong, wrong, correct, correct, correct. That sequence suggests something different from correct, correct, wrong, wrong, wrong.

The final raw percentage could be similar while the likely direction of learning differs. Knowledge tracing uses order because learning is a process through time.

3. Bayesian Knowledge Tracing Made the Hidden State Explicit

Bayesian Knowledge Tracing, associated with early intelligent tutoring systems, models a skill using a hidden learned-or-not-learned state and updates the probability of mastery as new responses arrive.

The classic model commonly includes four ideas: an initial probability that the learner already knows the skill, a probability of learning after an opportunity, a slip probability for getting an item wrong despite knowing, and a guess probability for getting it right despite not knowing.

A major 2023 survey traces this family from the original Bayesian approach through factor models and deep-learning methods. Source: Abdelrahman, Wang & Nunes, Knowledge Tracing: A Survey.

4. Guesses and Slips Protect the Model From One-Answer Thinking

A student can know a skill and still make a careless error. A student can lack the skill and still select the correct option.

By modelling slips and guesses, knowledge tracing accepts that observed correctness is noisy. That is a useful educational idea even outside software: one answer should rarely carry the full weight of a learning judgement.

5. Skill Labels Are Part of the Model

A tracing system needs to know what knowledge component each task exercises. If the labels are wrong, the inference can be wrong even if the algorithm is sophisticated.

A word problem may require arithmetic, reading comprehension, representation and unit reasoning. Tagging it only “percentage” can hide the real source of failure.

6. The Q-Matrix Problem

Many educational models use a mapping between questions and the skills or knowledge components they require. This mapping is often called a Q-matrix.

It looks administrative, but it is epistemic infrastructure. If one item quietly depends on three skills and the map records one, the model may attribute failure to the wrong place.

7. Deep Knowledge Tracing Changed the Modelling Style

In 2015, researchers introduced Deep Knowledge Tracing using recurrent neural networks to model sequences of student interactions without requiring the same explicit hand-designed state representation as classic Bayesian models.

The model demonstrated strong predictive performance on several datasets and triggered a large research wave around neural sequence models for learner state estimation.

Source: Piech et al., Deep Knowledge Tracing.

8. Better Prediction Does Not Automatically Mean Better Explanation

A model can predict the next answer accurately while remaining difficult to interpret. A teacher may still need to know why the learner is likely to fail and what intervention is appropriate.

Research following Deep Knowledge Tracing showed that more flexible shallow models can capture important regularities and sometimes approach deep-model performance while retaining clearer interpretation.

Source: Khajah, Lindsey & Mozer, How Deep Is Knowledge Tracing?.

9. Forgetting Complicates the State

Classic formulations often assumed that once a skill was learned, it remained learned. Real learners forget.

Time since practice, interference, retrieval history and later use can change accessibility. Modern tracing models therefore often incorporate temporal features or explicit forgetting mechanisms.

10. Performance Can Improve Without the Model Seeing the Cause

A student may study with a tutor, read a book, sleep better or receive an explanation outside the platform. The next online response improves, but the tracing system sees only the changed answer.

This is one reason digital learner models should be treated as partial observers of a larger learning ecology.

11. Response Time Can Add Signal

Two correct answers can mean different things if one takes eight seconds and the other five minutes with repeated hints. Timing, hint use, retries and confidence can enrich the learner model when interpreted carefully.

But more data also create more ways to overread noise. A slow answer may reflect distraction, not weak knowledge.

12. Knowledge Tracing Can Route Practice

Once a system estimates learner state, it can use that estimate to choose a next task, revisit a prerequisite, recommend a resource or decide whether more evidence is needed.

This is where tracing becomes operational. The model is valuable not because it produces a probability, but because the probability changes a useful decision.

13. Knowledge Tracing and Adaptive Testing Are Neighbours, Not Twins

Adaptive testing typically focuses on efficient measurement of a latent ability using calibrated items. Knowledge tracing focuses on evolving knowledge over a sequence of learning interactions.

One asks, “What question measures this learner efficiently now?” The other asks, “How is this learner’s knowledge state changing across practice?” Systems can combine both, but the questions are different.

14. The Danger of the Mastery Number

A dashboard may show “Skill mastery: 0.83.” The visual precision can create false confidence.

The number depends on model assumptions, item labels, training data, response history and parameter estimation. It should be read as modelled evidence, not as 83% of a physical substance called knowledge.

15. Cross-Domain Comparison: Radar Tracking and Medical Monitoring

Radar does not know an aircraft’s future position with certainty; it updates a state estimate from noisy observations. Medical monitoring similarly combines repeated measurements to infer whether a patient is stable, improving or deteriorating.

Knowledge tracing has the same abstract architecture: hidden state, noisy observations, updating over time and decisions made under uncertainty.

The analogy has limits, but it reminds us that one data point is not the object being measured.

16. Missing-Node Scan: Where Knowledge Tracing Breaks

• Questions are tagged to the wrong skills.
• Skills are defined too broadly to be educationally actionable.
• The model assumes learning where repeated exposure is only producing familiarity.
• Outside learning is invisible to the platform.
• Wrong answers from careless slips are interpreted as conceptual loss.
• Correct guesses inflate estimated knowledge.
• Forgetting is ignored.
• Prediction accuracy is optimised without checking educational usefulness.
• Model probabilities are shown without uncertainty or explanation.
• Automated routing replaces teacher judgement instead of supporting it.

17. A Human-Compatible Knowledge-Tracing Loop

1. Define a small, meaningful skill map.
2. Collect several varied observations per important skill.
3. Update the estimated state.
4. Flag surprising patterns rather than hiding them.
5. Use a discriminating question when uncertainty matters.
6. Route practice or teaching based on the likely weak node.
7. Retest after delay and in a changed form.
8. Allow human review to override the model when context explains the data better.

18. Privacy and Governance Matter

Knowledge tracing requires behavioural data. That raises questions about retention, consent, access, model bias, portability and whether a learner can challenge an incorrect automated inference.

The more consequential the decision, the stronger the governance should be. A recommendation for another practice question is different from an automated placement decision that closes educational options.

19. Evidence and Limits

Knowledge tracing is a mature research problem with decades of work and a large modern literature. The evidence supports its usefulness for modelling learner interactions and predicting future performance in computer-supported environments.

But prediction is not understanding. Benchmark gains can depend on dataset structure, model flexibility and hidden regularities. Generalising across curricula, schools and populations remains difficult. Interpretability, fairness and data quality are continuing research problems.

The safest educational stance is to use knowledge tracing as a probabilistic instrument inside a larger teaching system, not as a replacement for the learner’s work, explanation, context or teacher observation.

20. The Return Path

The learner answers one fraction question correctly.

We still do not know enough.

So the system waits for another observation. Then another. It updates. It remains uncertain. It looks for a pattern.

Knowledge tracing works when a sequence of imperfect observations becomes a useful, revisable estimate of learning—without anyone forgetting that the estimate is not the learner.