How Test Information Works | Measure Where a Test Is Precise Instead of Hiding Behind One Reliability Number

eduKateSG Learning Node Series · 0172

A test can be reliable overall and still measure one part of the proficiency range much better than another.

A paper built from mostly medium-difficulty questions may distinguish students near the middle well while telling us relatively little about the strongest or weakest learners. A certification test can deliberately concentrate precision near a pass standard. An adaptive test can choose the next question because the last answer changed where information is most valuable.

Test information works by describing how much statistical precision an item-response model provides at different levels of the latent trait, instead of compressing all measurement precision into one global reliability number.

The 50-Second Read

• In item response theory, measurement precision can vary across proficiency.
• An item information function shows where one item contributes most information.
• A test information function combines information from the items under the model.
• More information means smaller conditional measurement error.
• The relationship is commonly expressed as standard error approximately equal to the reciprocal square root of information.
• Items near a person’s proficiency often provide more information than items that are almost certainly right or wrong.
• Higher discrimination can increase information under common IRT models, but only if model fit and assumptions are defensible.
• A pass/fail test may need especially high information around the cut score.
• A broad growth test may need useful information across a wider range.
• Adaptive testing uses information to choose informative next items.
• Local dependence, multidimensionality and model misspecification can make information look larger than the usable evidence really is.
• Test information is model-based precision, not proof of validity, fairness or good educational decisions.

Canonical Owner Boundary

This node owns conditional measurement precision across the proficiency scale under item-response models. How Standard Errors Work owns sampling precision of estimators broadly. How Generalizability Theory Works owns variance across facets and replication conditions. How Adaptive Testing Works owns item selection from an evolving proficiency estimate. This node asks: where on the proficiency scale does this test provide information, and where does its uncertainty widen?

1. Reliability Is Often Reported as One Number

Classical reliability summaries are useful. They can tell us how consistently a test distinguishes people across a population under a specified framework. But a single coefficient can hide the shape of precision.

If a test is excellent near average proficiency and weak at both extremes, one global number may still look respectable. The test information function makes that unevenness visible.

2. Item Response Theory Moves the Question to the Trait Scale

IRT models the probability of an item response as a function of latent proficiency and item parameters. Instead of saying an item is simply “hard,” the model describes how response probability changes across proficiency.

Information is derived from that response function. It is high where small changes in proficiency produce response changes that help distinguish nearby trait levels, and low where the item response is almost certain regardless of a small proficiency change.

3. The NCME Definition

The National Council on Measurement in Education glossary defines the test information function as a mathematical function relating each level of ability or latent trait to the reciprocal of the corresponding conditional measurement-error variance.

That definition contains the whole idea: information is conditional on proficiency, and it is inversely related to error variance.

4. Information and Standard Error Move in Opposite Directions

A familiar IRT relationship is:

SEM(θ) ≈ 1 / √I(θ)

where I(θ) is the information at proficiency θ. The exact expression depends on the estimation framework, but the conceptual direction is stable: more information means smaller conditional standard error.

The 2026 fifth edition of Educational Measurement, Chapter 10 shows this relationship explicitly when discussing Fisher information and IRT-based standard errors of measurement.

5. An Invented Numerical Example

Suppose a test provides information I = 4 at one proficiency level. The simple reciprocal-square-root relationship gives an approximate conditional standard error of 1/√4 = 0.50.

At another proficiency level, suppose information is 16. The corresponding approximate standard error is 1/√16 = 0.25.

Four times as much information halves the standard error. This illustrates why improving precision becomes increasingly expensive: error shrinks with the square root of information rather than one-for-one.

6. Why Extremely Easy Items Provide Little Information for Strong Learners

If a strong learner has a near-certain probability of answering an item correctly, the response tells us little about whether the learner is moderately strong or exceptionally strong. Most people in that range will answer correctly.

The item may still be useful for content coverage or foundational verification. Low statistical information at one trait level does not make an item educationally worthless.

7. Why Extremely Hard Items Provide Little Information for Weak Learners

The mirror image occurs at the lower end. If nearly everyone at a proficiency level is expected to fail an item, another failure contributes little discrimination among nearby proficiency values.

A hard item becomes informative for learners whose proficiency is closer to the item’s difficulty region.

8. Difficulty Locates the Information Region

In common one- and two-parameter logistic models for dichotomous items, information is greatest near the item difficulty parameter. The exact height also depends on discrimination.

This means a good test for a wide proficiency range usually needs items distributed across that range rather than many near-duplicates at one difficulty level.

9. Discrimination Changes the Height of Information

Under the two-parameter logistic model, a more discriminating item has a steeper response curve and can provide more information near its difficulty location.

But “higher discrimination is always better” is too simple. An extremely high estimated discrimination can signal local dependence, overfitting, narrow content or other model problems. Item content and fit still matter.

10. Guessing Changes the Information Shape

In models that include a lower asymptote for multiple-choice guessing, the relationship becomes more complex. Very low-proficiency respondents can answer correctly by chance, reducing how much a correct response tells us.

Information functions are therefore model-specific. Do not copy a formula from one IRT model into another without checking the parameterisation.

11. Polytomous Items Have Information Too

Items scored across several categories—rubric levels, partial credit or rating categories—can also have information functions. The information depends on how category probabilities change across proficiency.

ETS research by Eiji Muraki adapted information-function reasoning to the generalized partial credit model and showed how category scoring choices affect information.

12. Item Information Adds to Test Information—Under the Model

When the model’s local-independence assumptions hold, item information can be added across items to form a test information function.

This is one of IRT’s practical strengths. A test developer can see which proficiency ranges are already well measured and where the item pool is thin.

13. More Items Do Not Add Equal Information Everywhere

Adding ten very easy items can increase total information for low-to-middle proficiency while doing little at the high end. Adding ten hard items can produce the opposite effect.

Therefore, “longer test” and “more precise test” are incomplete statements. Ask: more precise where?

14. A Test Information Curve Is a Map

Plot information on the vertical axis and proficiency on the horizontal axis. The shape reveals where the test is strongest.

A broad hill suggests useful precision across a wide range. A narrow peak suggests targeted precision. Two peaks can indicate a test built to distinguish two regions. A low tail warns that uncertainty widens there.

15. A Certification Test Can Target the Cut

If the decision is pass/fail at a known standard, the most consequential uncertainty lies near the cut score. Test developers may therefore select items that provide strong information around that region.

ETS’s nonmathematical introduction to IRT by Samuel Livingston explains this design logic directly: when a test serves a pass/fail decision, high information near the cut can be especially valuable.

16. A Growth Test Needs Wider Coverage

A test used to measure growth across several years may need useful information across a wider proficiency distribution. If the test is precise only near one grade-level average, students far below or above that region can have unstable estimates.

Growth claims require not just comparable scales but adequate precision at both measurement occasions and across the relevant proficiency range.

17. Adaptive Testing Uses Information Operationally

Computerised adaptive testing can select an item because it is expected to provide high information near the current proficiency estimate while satisfying content and exposure constraints.

After the response, the estimate updates. The most informative next item may change. The test is not simply becoming harder; it is trying to reduce uncertainty efficiently.

18. Stopping Rules Can Use Conditional Standard Error

An adaptive test can stop when the estimated standard error becomes sufficiently small for the intended purpose, provided other blueprint and minimum-length rules are satisfied.

This is why overestimated information is dangerous. The system can become falsely confident and stop too early.

19. Local Dependence Can Inflate Information

If several items share a passage or earlier answer, they may contain overlapping evidence. A model that assumes conditional independence can count that shared signal too generously.

ETS research by Wainer and Wang found substantial overestimation of test information in historical TOEFL testlets when local dependence was ignored.

20. Correction Methods Exist Because Information Can Be Overstated

Feifei Li developed an information-correction approach combining perspectives from IRT and generalizability theory for testlet-based data.

The existence of such methods reinforces a key principle: information is not an intrinsic quantity printed on an item forever. It is model-dependent and sensitive to assumptions about response dependence.

21. Multidimensionality Changes the Geometry

If the test measures several latent dimensions, information becomes a matrix rather than one scalar curve. Precision depends on direction in the multidimensional trait space.

A one-dimensional information curve can therefore be misleading when the construct is substantially multidimensional.

22. Posterior Information Is Not Identical to Test Information

Bayesian estimation can combine item-response information with a prior or population distribution. The resulting posterior precision contains more than item information alone.

The 2026 Educational Measurement chapter distinguishes contributions from item information and the population distribution. Reports should therefore specify what their standard error represents.

23. Information Does Not Prove Content Coverage

A test could achieve an impressive information curve by selecting many statistically powerful items from one narrow content area.

The score might be precise for the wrong construct representation. Blueprint coverage and validity remain separate requirements.

24. Information Does Not Prove Fairness

An item can be highly informative while functioning differently across comparable groups. High statistical discrimination does not override DIF, accessibility or opportunity-to-learn concerns.

Measurement precision is one dimension of quality, not a universal quality score.

25. Information Does Not Prove the Model Is True

Information is computed inside a model. If item parameters are biased, local independence fails or the dimensional structure is wrong, the information curve can be confidently precise about a poor representation.

Fit checks, residual analysis and sensitivity work belong before strong interpretations.

26. Item Banks Can Be Audited by Information Gaps

Suppose a mathematics item bank contains hundreds of medium items but very few strong high-difficulty reasoning items. The total item count looks healthy. The information map reveals a high-proficiency gap.

That gap can guide item-development priorities alongside content blueprint needs.

27. A Practical Test-Information Workflow

1. Define the score use and proficiency range.
2. Specify an appropriate IRT model.
3. Calibrate items with suitable data.
4. Check item and model fit.
5. Inspect item information functions.
6. Sum to the test information function where independence assumptions permit.
7. Convert information into conditional error for interpretation.
8. Compare the precision map with the decision map. Where do cuts, growth claims or reporting levels sit?
9. Inspect content coverage, fairness and dependence separately.
10. Revise the item pool or form where precision is misplaced.
11. Validate the resulting decisions, not only the curve.

28. Failure Mode: One Reliability Number Is Enough

The technical manual reports reliability = .90, so the test is assumed equally precise for every learner.

Repair: inspect conditional precision across the proficiency range relevant to the score use.

29. Failure Mode: Build Every Item at the Cut

A certification programme learns that information near the cut matters and fills the test with near-cut items.

Repair: preserve content coverage, minimum breadth and any secondary score uses. Statistical targeting must remain inside the construct blueprint.

30. Failure Mode: Trust Overly High Discrimination

An item has an enormous discrimination estimate, so it is celebrated as exceptionally informative.

Repair: inspect local dependence, exposure, item similarity and model fit. Implausibly strong information can be a warning.

31. Failure Mode: Ignore the Low-Information Tails

A growth dashboard reports precise-looking scores for students far above the range the test was designed to measure.

Repair: display conditional uncertainty and use an instrument or adaptive pool that actually contains information in that region.

32. Cross-Domain Comparison: A Camera Lens

A lens can be sharp in one focal plane and blurred elsewhere. One global statement—“this is a sharp lens”—hides where precision actually occurs.

Test information asks for the measurement equivalent: where is the instrument sharp?

33. Cross-Domain Comparison: Radar Coverage

A radar system may detect targets reliably in one distance band and poorly in another. Average performance does not show the coverage holes.

A test information curve is a coverage map for latent proficiency under the model.

34. A Classroom Translation Without IRT Software

A teacher does not need to fit an IRT model to use the conceptual lesson. Ask whether the question set distinguishes the students you need to understand.

If everyone gets every item right, the quiz verifies a floor but tells little about differences above it. If everyone gets every item wrong, the quiz verifies that the ceiling is too high but tells little about intermediate learning.

Use a spread of questions aligned to the curriculum and the decision. That is not equivalent to formal item information, but it carries the same design instinct: precision depends on where the tasks sit relative to learner capability.

35. Missing-Node Scan

The missing node may be test information when a reliability coefficient looks strong but high achievers all hit the ceiling; pass/fail decisions remain unstable near the cut; an adaptive test keeps asking questions that seem too easy or too hard; an item bank has thousands of questions but few in a crucial proficiency region; or the standard error changes sharply across the reported scale.

36. Evidence and Limits

Information functions are foundational to item response theory and modern test design. NCME defines test information directly in relation to conditional measurement-error variance, and current Educational Measurement treatments derive the relationship to standard error. ETS research and large-scale assessment practice use information for item analysis, test assembly and adaptive measurement.

But information remains conditional on the model. Local dependence can inflate it. Multidimensionality changes it. Poor content representation can make a precise test invalid. A strong information curve is therefore evidence of statistical precision within a measurement system, not a substitute for the system’s wider validity argument.

37. The Return Path

Return to the paper filled with medium-difficulty items.

Its overall reliability may be high. The information curve shows that it is especially precise near the middle and much less precise at the extremes.

Now the test developer can ask the right next question: is that shape appropriate for the decisions the test is supposed to support?

Test information matters because measurement precision has a location. A trustworthy test knows not only how precise it is, but where that precision lives.

Research and Further Reading

• NCME Glossary — Test Information Function
• Educational Measurement, Fifth Edition — Item Response Theory
• Livingston — Basic Concepts of Item Response Theory
• Muraki — Information Functions of the Generalized Partial Credit Model
• Wainer & Wang — Testlets and Overestimated Test Information
• Li — Information Correction for Testlet-Based Analysis

eduKateSG Learning Node Series · 0172 · Previous: 0171 — How Classification Consistency Works.