eduKateSG Learning Node Series · 0090

How Data-Informed Instruction Works | Turn Classroom Evidence Into the Next Teaching Move Without Letting the Dashboard Teach the Lesson

A teacher asks a class ten questions. Seven students miss Question 6. The obvious response is to reteach the topic.

But Question 6 may be badly worded. The seven students may share one prerequisite gap rather than misunderstanding the whole topic. They may know the concept but misread the diagram. The error may come from one procedural step. Or the item may have exposed something important that the previous five questions never tested.

Data-informed instruction begins at that fork. It is not the habit of looking at numbers. It is the discipline of collecting evidence for a teaching question, judging the quality and meaning of that evidence, choosing a response, and then checking whether the response changed learning.

Data is useful when it changes the next instructional decision more accurately than intuition alone would have changed it.

The 50-Second Read

• Start with an instructional question, not with whatever data happens to be available.
• Collect the smallest useful evidence that can discriminate between plausible explanations.
• Scores, student work, observations, talk, attendance and self-reports answer different questions.
• Data quality includes validity, reliability, completeness, timeliness and interpretability.
• A pattern can reveal where to look without proving the cause.
• Disaggregate carefully: subgroup patterns can matter, but tiny samples create false precision.
• Use data to generate and test instructional hypotheses rather than to label learners.
• Choose a teaching response that matches the diagnosed mechanism.
• Monitor whether the response worked; otherwise data use becomes reporting rather than learning.
• Do not measure so much that measurement steals the time needed for teaching.
• A healthy data culture permits people to question the metric, not merely explain the score.

Canonical Owner Boundary

This Learning Node owns the instructional cycle that turns multiple forms of classroom data into the next teaching decision. How Student Work Analysis Works owns close reading of learner artefacts. How Teacher Noticing Works owns professional attention to meaningful evidence during instruction. How Adaptive Testing Works owns algorithmic selection of subsequent test items. How Knowledge Tracing Works owns model-based estimation of changing knowledge state. How Assessment Works remains the broad assessment owner. This page owns the human instructional loop from question to evidence to changed teaching.

1. Begin With a Teaching Question

Data collection becomes useful when it is attached to a decision. “Which students understand equivalent fractions?” “Is the problem vocabulary or ratio structure?” “Did the new modelling routine improve independent performance?” “Who is ready to move on?”

Without a question, teachers can drown in dashboards while learning almost nothing actionable.

2. Decide What Evidence Would Discriminate Between Explanations

If students fail a word problem, possible causes include vocabulary, arithmetic, representation, strategy selection or working-memory overload. A second broad test may only confirm failure. A better probe isolates the competing explanations.

Good instructional data is often discriminating rather than abundant.

3. Use the Smallest Useful Measurement

A two-question exit ticket can be more instructionally useful than a forty-question benchmark if the two questions target the exact misconception that matters tomorrow.

Measurement has a cost. Every minute spent testing, entering, cleaning and discussing data is a minute not spent elsewhere. The aim is not maximum data. It is sufficient evidence for a better decision.

4. Know What Each Data Source Can See

• Selected-response items can sample knowledge efficiently but may hide reasoning.
• Constructed responses expose process and explanation but take longer to score.
• Student work reveals what learners produced under authentic task conditions.
• Observation reveals behaviours, strategies and interaction that tests may miss.
• Student talk can expose conceptual models before they become written answers.
• Attendance and participation reveal opportunity to learn, not mastery by themselves.
• Student self-report reveals experience and confidence but can diverge from performance.

5. Validity Comes Before Visualisation

A beautiful dashboard cannot repair a weak measure. If the question is whether students can transfer a principle to a new situation, a quiz composed entirely of rehearsed examples may not provide valid evidence.

Before asking what a chart shows, ask whether the underlying task measured the capability you care about.

6. Reliability Matters When Small Differences Drive Decisions

Scores contain measurement error. A learner who scores 67 today and 71 next week may not have meaningfully changed. The shorter and noisier the measure, the more careful the interpretation must be.

Use repeated evidence when the decision is consequential. One result can trigger a closer look; it should not always trigger a label.

7. Timeliness Can Matter More Than Perfection

A precise report delivered six weeks after a unit may be less useful instructionally than a rough but trustworthy signal available before tomorrow’s lesson.

Instructional data lives inside a decision window. If the decision has already passed, the data becomes historical rather than adaptive.

8. Completeness Is Not the Same as Representativeness

A dataset can contain every response from students who completed an online task and still systematically miss students who were absent, disengaged or unable to access the task.

Missingness can itself be instructional information. Ask who is absent from the data before drawing conclusions about the group.

9. Look at Distributions, Not Only Means

A class average of 65 can describe thirty students clustered around 65 or two groups centred around 40 and 90. Those classrooms require different teaching responses.

Means compress. Sometimes compression is useful. Sometimes it hides the very structure instruction needs.

10. Item Patterns Can Be More Useful Than Total Scores

A total score tells how much was correct. Item patterns can reveal which concepts, representations or steps produced failure.

When several items share the same underlying demand, a repeated error becomes a stronger diagnostic signal than one isolated mistake.

11. Error Categories Turn Marks Into Mechanisms

Instead of recording only “wrong,” classify errors when useful: concept error, procedure error, reading error, representation error, incomplete reasoning, careless transcription, weak checking.

The category should lead to a different response. If every error category produces the same worksheet, the diagnosis is decorative.

12. Student Work Adds Texture to the Number

Two students can receive the same mark for completely different reasons. One may have strong reasoning but weak arithmetic. Another may have guessed several answers correctly while lacking the underlying model.

That is why aggregate data should often hand off to student work analysis before an instructional decision is finalised.

13. Observation Captures Process

A student may eventually reach the correct answer while using an inefficient or fragile strategy. Observation can reveal whether independence is growing, whether prompts are fading, whether learners check their work and whether productive struggle is occurring.

Process data helps distinguish a correct performance from a stable capability.

14. Student Talk Can Reveal the Hidden Model

Ask, “Why?” “What made you choose that?” “Would this still work if the numbers changed?” A short explanation can reveal whether an answer came from memorised surface features or deeper structure.

Data does not have to be numeric to be systematic.

15. Generate an Instructional Hypothesis

After reading the evidence, state the hypothesis in a testable form: “Students are treating the denominator as a second whole-number count rather than as the size of the partition.”

This is stronger than “They are weak at fractions.” A mechanism-shaped hypothesis suggests a mechanism-shaped intervention.

16. Choose the Response That Matches the Hypothesis

If the problem is vocabulary, teach the language. If it is representation, change the representation. If it is a prerequisite, repair the prerequisite. If it is insufficient retrieval, increase retrieval. If it is failure to transfer, vary examples and conditions.

Data-informed instruction fails when diagnosis changes but treatment remains generic.

17. Group Flexibly Around the Current Need

Data can support temporary grouping around a specific misconception or readiness state. The danger is allowing temporary evidence to harden into permanent identity.

Groups should move when the evidence moves.

18. Do Not Confuse Prediction With Explanation

A model may predict which students are at risk without explaining why. Prediction can help allocate attention, but intervention requires additional diagnosis.

“Likely to struggle” is not an instructional mechanism.

19. Disaggregate With Humility

Subgroup data can reveal patterns hidden in averages. It can also become unstable when groups are tiny. Treat small-number differences as prompts for investigation rather than precise estimates.

Always ask whether categories are educationally meaningful for the question being asked.

20. Protect Against Data Labels

“Red student.” “Low group.” “At risk.” Labels can simplify coordination, but they can also compress a learner into one metric and change adult expectations.

Use data to describe a current state or risk, not to declare a fixed identity.

21. Give Students Access to Useful Data

The U.S. Institute of Education Sciences practice guide Using Student Achievement Data to Support Instructional Decision Making includes a recommendation to teach students to examine their own data and set learning goals.

This works best when the data is interpretable and connected to action. “You are at 62” is weaker than “You can identify the relationship, but multi-step justification is still inconsistent; here is the next practice target.”

22. Build Data Literacy, Not Dashboard Dependency

The 2024 Forum Guide to Data Literacy from the National Forum on Education Statistics emphasises the skills stakeholders need to understand and use data appropriately.

A mature school does not merely train staff where to click. It trains them to ask where the number came from, what it represents, what uncertainty it contains and what decision it can legitimately support.

23. Data Culture Determines Whether Bad News Survives

If every low score becomes blame, people learn to defend the number. If evidence can trigger inquiry without humiliation, people are more likely to surface problems early.

IES resources on cultivating cultures of data use describe data culture as collective behaviours and beliefs around using data to improve decisions. Culture matters because technically perfect data can still be organisationally unusable.

24. Separate Accountability Data From Instructional Data

Large-scale assessments are often designed for comparability and system monitoring. Classroom formative evidence is designed for immediacy and adaptation. One dataset should not automatically be forced to perform the other’s job.

A useful rule is: use the measure at the level and timescale it was designed to support.

25. Avoid the Assessment Load Trap

Schools can respond to uncertainty by testing more. Eventually the system collects enough data to know that students have had less time to learn.

Every recurring assessment should answer three questions: which decision uses it, who acts on it, and what would stop us collecting it?

26. Practical Measures Can Support Improvement

IES’s Practical Measurement for Continuous Improvement in the Classroom focuses on short surveys, checklists, rubrics, exit tickets and quizzes that teams can use to judge whether a change is being implemented and whether intended outcomes are moving.

The important distinction is that practical measures are designed for learning and improvement, not automatically for high-stakes accountability.

27. Recheck After the Teaching Move

Data-informed instruction is incomplete until the response is tested. If you reteach the misconception, collect a fresh probe. If you regroup students, check whether the new grouping helped. If you change explanation, look for improved independent performance.

Otherwise the school has data-informed diagnosis and intuition-informed treatment.

28. Monitor Transfer, Not Only Rehearsal

After targeted teaching, learners often improve on tasks that closely resemble practice. The deeper question is whether the learning survives altered wording, representation, spacing or context.

Add at least one transfer probe before declaring the problem repaired.

29. Cross-Domain Comparison: Clinical Monitoring

Clinicians do not measure every possible biological variable at every visit. They choose observations relevant to the condition, treatment and risk, then update the plan when the patient’s state changes.

The educational transfer is selective monitoring: measure what can change the action.

30. Cross-Domain Comparison: Manufacturing Process Control

Manufacturing distinguishes ordinary variation from signals that the process has changed. The lesson for classrooms is not to turn children into factory output. It is to respect variation and avoid overreacting to every single data point.

Look for patterns across repeated measures before making large changes.

31. Cross-Domain Comparison: Software Observability

Complex software systems combine logs, metrics and traces because no single signal explains a failure. Engineers move from symptom to component to mechanism.

A classroom also benefits from multiple evidence layers: score, work, observation, talk and context. The point is not to collect everything. It is to retain enough observability to diagnose the learning process.

32. Example: Algebra Errors

A class performs poorly on solving equations. Total scores suggest a broad weakness. Student work shows something narrower: most learners can isolate a variable when the variable appears once, but errors rise when brackets require expansion first.

The next lesson therefore targets operation order and structural reading rather than reteaching all equation-solving. A short follow-up probe confirms improvement. Data reduced the size of the intervention by improving the precision of the diagnosis.

33. Example: Reading Comprehension

Students miss inference questions. The teacher initially plans more inference worksheets. A closer look shows that most errors occur when pronoun reference is ambiguous. Students cannot reliably identify who or what a sentence refers to, so the inference collapses downstream.

The intervention shifts from “practise inference” to “repair reference tracking, then retest inference.” Data has found the first weak link.

34. Failure Mode: Dashboard First

Teachers open a dashboard and search for something to improve. The available metrics define the agenda.

Reverse the order. Start with the learning question, then ask which evidence can answer it.

35. Failure Mode: One Score, One Story

A single low score becomes a complete explanation of a learner. The school moves directly from result to intervention without checking the task, error pattern or context.

One score should usually open a diagnostic branch, not close it.

36. Failure Mode: Data Meetings Without Changed Teaching

Teams spend an hour discussing percentages, colours and trends. Nobody leaves with a changed explanation, grouping, example, sequence or practice task.

A data meeting that does not alter action is reporting, not instructional learning.

37. Failure Mode: Measuring the Intervention Instead of the Learning

Schools sometimes measure whether teachers used the new routine and forget to measure whether learners improved. Fidelity matters, but delivery is not the final outcome.

Track both: did we implement the mechanism, and did the intended learning move?

38. A Practical Data-Informed Instruction Loop

• Question: define the instructional decision.
• Discriminate: list plausible explanations and what evidence would separate them.
• Collect: gather the smallest useful evidence.
• Check quality: inspect validity, reliability, completeness and timeliness.
• Pattern: examine distributions, items, error categories and subgroups.
• Triangulate: compare scores with work, observation and student explanation.
• Hypothesise: state the likely learning mechanism.
• Act: choose instruction matched to that mechanism.
• Recheck: collect fresh evidence after the teaching move.
• Transfer: test whether learning survives changed conditions.
• Record: preserve what was tried and what happened.
• Update: adapt, fade support or escalate diagnosis.

39. What the Research and Guidance Base Adds

The IES What Works Clearinghouse practice guide Using Student Achievement Data to Support Instructional Decision Making recommends making data part of an ongoing cycle of instructional improvement, teaching students to use their own data, establishing a clear vision for schoolwide data use and building organisational supports for a data-driven culture. The guide is older, and some of its specific evidence ratings reflect the research available in 2009, but its core distinction between data collection and an improvement cycle remains useful.

More recent IES resources extend the infrastructure around that idea. The 2024 Forum Guide to Data Literacy focuses on the knowledge stakeholders need to understand and use data, while REL resources on data culture emphasise collective behaviours and beliefs. Together they point toward a mature position: good data use is technical, instructional and organisational at the same time.

40. Missing-Node Scan: Questions Before You Change Instruction

• What instructional decision will this evidence inform?
• Does the measure actually represent the capability we care about?
• Could the pattern be measurement error or ordinary variation?
• Who is missing from the dataset?
• What does the item-level pattern show?
• What does the student work show?
• What would the learner say is happening?
• Which competing explanations are still plausible?
• What teaching response matches the mechanism?
• How quickly can we collect evidence after the response?
• Are we measuring learning or merely compliance with the new routine?
• What would make us abandon the current hypothesis?

41. The Return Path

The dashboard says seven students are below threshold.

The teacher does not stop there. She opens the work. Four students share one misconception. Two have correct reasoning but incomplete explanations. One student missed the lesson and has almost no usable evidence. The “seven” becomes three different instructional problems.

The next lesson changes. Four students receive a targeted representation. Two practise answer construction. One gets the missed prerequisite taught. Two days later the teacher checks again, this time with a new task rather than the same worksheet.

The number did not teach the lesson. It opened the investigation. The teacher taught the lesson.

Data-informed instruction works when evidence sharpens professional judgement—and professional judgement remains responsible for turning evidence into teaching.