How Cognitive Demand Works in Assessment | Why a Difficult Question Is Not Always a Deep Question

A question can be hard for the wrong reason.

A student sees an unfamiliar term, misreads a dense paragraph, forgets an obscure fact or makes one awkward calculation error. Most of the class gets the item wrong. The item statistics later show that the question was difficult.

But what kind of thinking did it actually require?

Now consider another question. Almost every fact needed is familiar. The arithmetic is simple. Yet the learner must compare two explanations, identify which evidence matters, coordinate several relationships and justify a conclusion that is not stated anywhere.

That question may demand much deeper thinking even if more students answer it correctly.

This distinction sits at the centre of good assessment design:

difficulty is not the same thing as cognitive demand.

Difficulty tells us how hard an item proved to be for a particular population under particular conditions. Cognitive demand concerns the kind and depth of mental processing the task requires.

OECD’s 2026 PISA assessment framework makes this distinction explicitly. An item can be difficult because the knowledge is unfamiliar while requiring little more than recall. Another can use familiar knowledge yet require strategic, relational or evaluative reasoning.

If we confuse the two, education begins to reward accidental hardness.

The goal should not be to make questions harder.

The goal should be to ask questions whose thinking demand matches the learning we claim to value.

The 50-second route

If you only have a minute, keep this:

Item difficulty is an empirical outcome: how successfully a group of learners performs on a question.
Cognitive demand is a task property: the type and depth of thinking needed to answer it.
Hard vocabulary, tricky wording, obscure content and heavy arithmetic can raise difficulty without raising meaningful cognitive demand.
Familiar content can support high cognitive demand when students must interpret, connect, compare, generalise, evaluate or justify.
Verbs such as “analyse” and “evaluate” do not automatically create deep thinking. The actual information structure of the task matters.
A good assessment contains a planned range of demand appropriate to the curriculum and purpose.
Teachers should review both what students know and what the task made them do with that knowledge.
Cognitive demand should be raised by deeper reasoning, not by hiding the route behind avoidable confusion.

Why the difference matters

Schools routinely use words such as easy, medium and hard.

Those labels are useful operationally. They are not enough educationally.

Suppose two mathematics questions are classified as “hard”.

Question A requires students to calculate a percentage using awkward decimal values and several steps.

Question B uses friendly numbers but asks students to decide which of three percentage methods will still work when the starting quantity changes, then justify the choice.

Both may take time.

Question A may be difficult because execution is cumbersome.

Question B may demand deeper relational reasoning.

If a curriculum says students should “reason mathematically” but the hard questions are merely longer calculations, the assessment is misrepresenting the intended competence.

This matters because assessment does not only measure learning. It signals what learning is worth.

Students study toward the demand they repeatedly encounter.

Stage 1: Separate the observed outcome from the task analysis

After a test, a teacher notices that only 25% of students answered an item correctly.

The first statement is descriptive:

“This item was difficult for this group.”

Do not immediately conclude:

“This item required high-level thinking.”

Several explanations are possible.

The content may not have been taught adequately. The wording may have been confusing. The item may contain an unusual convention. A diagram may be hard to read. The distractors may exploit a trivial misconception. The calculation may be long. The question may genuinely require complex reasoning. Or several of these may interact.

Cognitive demand must be analysed from the task itself.

Ask:

How many ideas must be coordinated?
Is the route obvious or must it be selected?
Must the learner interpret a representation?
Is evidence weighed?
Are alternatives compared?
Must a general rule be constructed?
Is justification required?
Can the answer be reached by a memorised procedure?

The correctness rate cannot answer those questions by itself.

Stage 2: Identify the knowledge load

Thinking does not happen without knowledge.

A task may require:

factual knowledge;
conceptual knowledge;
procedural knowledge;
strategic knowledge;
disciplinary conventions.

This creates an important complication.

A question can look cognitively sophisticated to a novice because the underlying knowledge is weak.

For an expert, the same task may be routine.

Consider: “Explain why the graph has a turning point at this value.”

A learner who does not know how gradient relates to a turning point faces both a knowledge problem and a reasoning problem.

A learner with well-organised calculus knowledge may recognise the relationship quickly.

Cognitive demand is therefore partly relative to prior learning, even though the task has analysable structural features.

The practical lesson is not that demand is unknowable.

It is that teachers should distinguish: the complexity built into the task from the difficulty produced by the learner’s current knowledge state.

Stage 3: Look beyond command verbs

Teachers often use taxonomies to plan assessment.

“State” sounds low. “Explain” sounds higher. “Evaluate” sounds high.

But verbs are unreliable shortcuts.

“Evaluate (3+4)” uses the verb evaluate and is simple arithmetic.

“State the most defensible conclusion from these conflicting sources and justify the limits of that conclusion” begins with state but may demand substantial reasoning.

Likewise, “explain” can mean: repeat a memorised sentence, or construct a causal account from new evidence.

The cognitive demand sits in the whole task: the knowledge available, the relationships to coordinate, the amount of transformation required, the openness of the route and the standard of justification.

Use verbs as signals, not as proof.

Stage 4: Distinguish routine execution from strategic selection

One of the strongest markers of higher demand is whether the method is given.

Compare:

A. Solve (3x+5=20).

The route is familiar and signalled.

B. A phone plan charges a fixed monthly amount plus a cost per gigabyte. Two bills are shown. Determine a model that could generate the bills and use it to predict the cost for another usage level.

The algebra may still be elementary.

The demand rises because the learner must:

identify relevant quantities;
infer a relationship;
choose a representation;
construct an equation;
use it;
check whether it makes sense.

The arithmetic is not harder.

The decision structure is.

This is why high cognitive demand is often about method selection, not computational decoration.

Stage 5: Look for coordination

Deep tasks frequently require several pieces of information to be held in relationship.

In reading: a student may need to compare the narrator’s claim with later evidence.

In science: a student may coordinate a graph, an experimental condition and a causal model.

In history: a student may compare provenance, chronology and corroborating sources.

In mathematics: a student may connect a diagram, equation and constraint.

The key is not the number of facts.

Ten unrelated facts can be recalled one by one with low relational demand.

Three facts may create high demand if the learner must determine how they interact.

A useful question is:

Does the task ask the learner to accumulate information or organise it?

Organisation usually tells us more about cognitive demand.

Stage 6: Examine the degree of novelty

Novelty is useful and dangerous.

A familiar question may be solved by pattern recognition.

A changed surface can reveal whether the learner understands the structure.

For example, a student who has practised: “Find 20% of $80” may succeed automatically.

A more novel item asks: “A price after a 20% discount is $80. What was the original price?”

The mathematical ideas are familiar but the direction changes.

That novelty can raise meaningful demand because students must reconstruct the relationship.

But novelty can also become irrelevant surprise.

If a primary mathematics question is wrapped in unfamiliar specialist knowledge about marine insurance, performance may be affected by context knowledge rather than mathematics.

Useful novelty changes the reasoning demand.

Bad novelty changes the access conditions.

Stage 7: Ask what has been scaffolded away

A task’s demand changes when support changes.

Consider a science question.

Version 1: “Use the graph to identify the temperature at which the reaction rate is highest.”

Version 2: “Describe the relationship between temperature and reaction rate.”

Version 3: “Two students propose different explanations for the change in reaction rate. Use the graph and your scientific knowledge to evaluate both explanations.”

The stimulus may be identical.

The scaffolding differs.

In Version 1, the operation is named. In Version 2, the student must identify the relationship. In Version 3, the student must coordinate evidence and explanation.

This is why task demand cannot be judged from topic or appearance.

The prompt architecture matters.

Stage 8: Make low-demand items legitimate

There is a strange consequence of conversations about “higher-order thinking”: simple questions begin to sound inferior.

They are not.

If a learner needs to know multiplication facts, vocabulary, formulae, spelling patterns or scientific definitions, direct recall can be exactly the right assessment demand.

A test made entirely of extended reasoning can be inefficient and unbalanced.

Low-demand items can:

sample breadth;
diagnose missing foundations;
check automatic knowledge;
reduce time;
provide access to later tasks.

The problem is not low demand.

The problem is demand mismatch.

If a curriculum values explanation and the assessment asks only recall, mismatch. If a foundational quiz intends to check facts and every item becomes an elaborate puzzle, mismatch.

Assessment quality comes from alignment, not maximal difficulty.

A concrete example: science knowledge versus scientific reasoning

Question 1: “Name the organelle where photosynthesis occurs.”

This is a direct knowledge check.

Question 2: “A variegated leaf is partly covered, exposed to light and then tested for starch. Predict which regions will contain starch and explain what the pattern shows.”

This requires knowledge plus prediction and explanation.

Question 3: “Two experiments produce different starch patterns. One changed light exposure and the other changed carbon dioxide availability. Compare what each experiment can and cannot show about the requirements for photosynthesis.”

Now the student must distinguish evidential roles.

All three can belong in a science assessment.

Calling Question 1 “bad” because it is recall would be silly. Calling Question 3 “hard” without analysing the reasoning would be incomplete.

The useful description is the demand profile.

A concrete example: history questions with the same content

Topic: causes of a political revolution.

Low demand: “State two causes.”

Moderate demand: “Explain how an economic crisis contributed to the revolution.”

Higher demand: “Two historians disagree about whether economic crisis or political legitimacy was the more important cause. Using the evidence provided and your own knowledge, evaluate which interpretation better explains the timing of the revolution.”

The topic did not change.

The relationship between knowledge, evidence and judgement changed.

Notice also that the last question is not automatically better for every purpose.

If students do not know the causes at all, asking for sophisticated evaluation may produce weak reasoning built on weak knowledge.

Demand must be sequenced with learning.

A concrete example: English reading

A passage describes a character entering an unfamiliar house.

Question A: “Where does the character go?”

Question B: “Which phrase suggests the character is uneasy?”

Question C: “How does the writer make the house feel threatening before any danger is confirmed?”

Question D: “Another reader argues that the house is not threatening; the character is projecting fear onto a neutral setting. How far does the passage support that interpretation?”

The progression is not simply longer answers.

The later items require more coordination of language, evidence and alternative interpretations.

That is a meaningful demand progression.

A concrete example: mathematics where difficulty misleads

Question A: Compute: [ \frac{8473 \times 0.037}{1.9} ]

Without a calculator, many students may find this unpleasant.

Question B: A student says: “If a quantity decreases by 20% and then increases by 20%, it returns to its starting value.” Decide whether the claim is always true and justify your conclusion.

Question B can use tiny numbers.

Yet it asks the student to examine changing bases, generate or use a counterexample and justify a general claim.

If Question A has a lower success rate, it is empirically harder for the tested group.

Question B can still have greater cognitive demand.

Those statements can coexist.

Cognitive demand and accessibility

An accessible item is not necessarily an easy item.

This is an important design principle.

Clear language can make a cognitively demanding task more valid because it removes irrelevant barriers.

A complex science reasoning item does not become intellectually weaker when the instructions are concise.

A history source question does not become less rigorous when the document is legible.

A mathematics problem does not become “too easy” because unnecessary narrative is removed.

Sometimes assessment writers fear that clarity gives away the answer.

But opacity is not depth.

The learner should struggle with the intended idea, not with preventable confusion about what the task is asking.

Cognitive demand and time pressure

Time can change observed difficulty.

A task that most students can solve in six minutes may become difficult if the examination gives three.

That does not automatically increase cognitive demand.

It increases execution pressure.

Sometimes speed is legitimately part of the construct. Fluency matters in some settings. Rapid recognition can be valuable.

But if the intended construct is deep reasoning, excessive time pressure can distort measurement by rewarding fast processing rather than the reasoning itself.

Assessment designers should therefore ask:

Is speed part of what we mean to assess?

If not, difficulty created by time shortage is noise.

Cognitive demand and artificial intelligence

Generative AI adds an interesting new reason to think carefully about demand.

A task that once required substantial student production may now be easy to outsource if the prompt is generic.

“Write 500 words explaining photosynthesis” appears substantial but may require little student decision-making if a tool can generate a competent response instantly.

A stronger task might require:

interpretation of class-specific data;
explanation of why one model changed;
defence of a reasoning step;
comparison with the learner’s earlier prediction;
an in-person transfer question.

The goal is not to create AI-proof puzzles.

It is to design evidence of thinking that corresponds to the intended learning.

Surface length has never been a reliable proxy for cognitive demand. AI merely makes that problem more obvious.

What the current evidence and frameworks signal

OECD’s PISA 2025 Assessment and Analytical Framework, published in May 2026, treats cognitive demand as an explicit assessment feature. It warns against confusing empirical item difficulty with the mental processing a question requires and notes that an item may be difficult because knowledge is unfamiliar while requiring only recall, or demanding because it requires coordination of familiar knowledge.

This is a powerful design signal because PISA is not merely ranking items from easy to hard. The framework asks what types of thinking the assessment is meant to elicit.

OECD’s August 2026 work on Latvia’s assessment system likewise reinforces the value of clear national assessment frameworks and coherent instruments. When assessment purposes and constructs are explicit, cognitive demand can be designed rather than left to chance.

The broader OECD work on high-quality teaching also matters. Cognitive engagement is not synonymous with keeping students busy or giving them difficult work. It involves sustained effort on challenging ideas, multiple approaches, meaningful contexts and metacognitive control.

Assessment should preserve that distinction.

Caveat: taxonomies simplify cognition

Educators often classify tasks into levels.

That is useful.

It is also imperfect.

Human cognition does not divide neatly into four boxes. The same task can feel different depending on prior knowledge. A procedure that is cognitively demanding during initial learning can become automatic with practice. An expert sees patterns that a novice must construct slowly.

So do not treat a demand taxonomy as a machine that produces objective truth.

Use it as a design conversation.

Two teachers may classify one item differently. Their disagreement can be productive if they explain:

what knowledge is assumed;
what decisions are required;
what relationships must be coordinated;
what counts as justification.

The value lies in inspecting the thinking.

Common failure mode 1: adding more steps to make a question “higher level”

A routine ten-step calculation is still routine.

Length can increase working-memory load and error opportunity without increasing reasoning depth.

If you want deeper demand, change the decision structure, not merely the number of operations.

Common failure mode 2: using obscure vocabulary as rigor

Specialist vocabulary is appropriate when the vocabulary itself belongs to the subject.

Unnecessary rare wording is not rigor.

If students fail because they cannot decode a decorative word unrelated to the construct, the item has become harder without becoming better.

Common failure mode 3: calling every unfamiliar question “application”

Unfamiliarity is not automatically transfer.

A question may look new but still signal the same memorised procedure.

Conversely, a familiar context can require genuine reasoning if the relationship changes.

Analyse what students must recognise and transform.

Common failure mode 4: putting a command verb at the top of a shallow task

“Analyse.” “Evaluate.” “Critically assess.”

The verb cannot manufacture intellectual depth.

If all relevant information points obviously to one memorised conclusion, the task may still be low demand.

Demand comes from the structure of the reasoning space.

Common failure mode 5: assuming a low success rate proves the question was excellent

A very difficult item may be poorly taught, ambiguously worded, miskeyed, inaccessible or outside the intended curriculum.

Celebrate evidence, not suffering.

Low success should trigger investigation.

Common failure mode 6: making every question high demand

A paper can become exhausting and narrow.

High-demand items often take longer, which reduces content sampling.

A balanced assessment uses different demands strategically.

Breadth and depth have to coexist.

Common failure mode 7: confusing scaffolding with lowering standards

A diagram, glossary or structured response space can reduce irrelevant load while preserving the intended reasoning.

The question is: What support removes noise, and what support removes the thinking we intended to assess?

If a scaffold tells students which reasoning step to take, it may reduce demand. If it merely makes the information legible, it may improve validity.

Common failure mode 8: never reviewing demand after students learn more

A question that was demanding in September may be routine in March.

That is success.

As knowledge becomes organised and procedures become fluent, instruction should create new opportunities for selection, connection and transfer.

Cognitive demand is not a permanent property of a worksheet label.

The learner route: diagnose what made a question hard

After a difficult question, do not write only:

“Hard question.”

Ask:

Was I missing knowledge? Did I know the ideas but not know which method to use? Did I misread a representation? Did I fail to connect two pieces of information? Did I choose a route but execute it badly? Did I understand the answer but struggle to justify it? Did time pressure change my performance?

These diagnoses lead to different repairs.

More facts will not fix method selection. More routine practice will not necessarily fix evidence evaluation. More deep reasoning tasks will not fix missing multiplication fluency.

Name the demand that broke.

The parent route: stop using “hard” as one category

When a child says the paper was hard, ask:

“What kind of hard?”

Unfamiliar? Long? Tricky wording? Too many calculations? Needed a method you did not recognise? Needed explanation? Ran out of time? Knew the idea but could not connect the evidence?

This conversation helps parents avoid two unhelpful responses:

“You just need more practice.” or “The school made the paper unfair.”

Sometimes more practice is needed. Sometimes the assessment is poorly designed. Sometimes the learner needs a different kind of thinking practice.

The distinction matters.

The teacher route: audit five questions

Choose five items from your next assessment.

For each, write:

Knowledge needed
Representation used
Decision required
Reasoning relationship
Expected justification
Likely irrelevant barriers
Estimated empirical difficulty

Then compare your predicted difficulty with the demand analysis.

If every “hard” item is merely longer, redesign one.

If every high-demand item requires unusually heavy reading, reduce the confound.

If no item requires students to select a method, ask whether method selection belongs in your curriculum.

Small audits can improve whole papers.

The curriculum route: plan a demand progression across years

A strong curriculum does not only add more content.

It changes what learners are expected to do with content.

Early: identify, recall, represent, execute.

Then: connect, select, compare, interpret.

Later: justify, generalise, evaluate, model, critique.

These are not rigid age stages. Young children can reason deeply, and older students still need factual fluency.

The point is that the curriculum should create repeated movement from supported knowledge use toward more independent coordination.

Assessment should make that progression visible.

Demand can also be hidden in the response standard

Two questions can have identical prompts and different cognitive demand because the expected quality of response changes.

Consider:

Why did the population fall?

For a young learner, a valid answer may identify one direct cause.

For an advanced geography student, the same surface question may require:

distinguishing immediate and structural causes;
using demographic data;
weighing migration against natural change;
acknowledging uncertainty;
judging whether one explanation is sufficient.

The prompt alone does not contain the whole demand.

Marking criteria, exemplars and classroom expectations also shape what counts as a successful response.

This is why assessment review should compare three things: the question, the intended answer and the scoring rule.

A prompt may look demanding, but a mark scheme may reward only a memorised phrase. Another prompt may look simple, while the rubric expects complex evidence coordination.

If those layers are not aligned, students receive mixed signals.

Demand profiles matter more than one label

It is tempting to classify each item as Level 1, 2, 3 or 4 and finish.

But many useful tasks have mixed profiles.

A science investigation question may require:

low-demand recall of a formula;
routine calculation;
interpretation of a graph;
higher-demand evaluation of whether the evidence supports a causal claim.

A history essay may combine straightforward factual retrieval with demanding organisation and judgement.

Instead of forcing the whole task into one category, teachers can sometimes annotate where the demand changes.

This is especially helpful for diagnosing student performance.

A learner may complete the factual setup correctly and fail when the task shifts into comparison. Another may identify the right relationship but lose marks in execution.

The item is then not simply “too hard”. It contains a demand transition.

Seeing those transitions can improve teaching because the repair can target the exact point where cognition changed jobs.

Beware of demand inflation in classroom language

Teachers sometimes raise the status of an ordinary task by describing it as “critical thinking”.

Students highlight a paragraph. They complete a comparison table. They answer an open question. The lesson plan labels the activity analysis.

But cognitive demand should be inferred from what learners actually have to decide and construct.

Highlighting can be demanding if students must determine which evidence is relevant to a contested claim.

It can be nearly mechanical if the relevant sentence is already signposted.

A comparison table can require deep relational thinking if the categories must be generated and justified.

It can be clerical if students merely copy features from two boxes.

The activity format does not determine the demand.

This matters for schools auditing “higher-order thinking”. Counting how many lessons contain debates, projects or open-ended tasks is not enough. The better question is what intellectual work those formats required.

Demand and feedback should be connected

Feedback after a high-demand task often collapses back into correctness.

“Good.” “Wrong evidence.” “Need more detail.”

That misses the reasoning opportunity.

Feedback can name the cognitive move:

“You identified both explanations but did not compare their evidential strength.”

“You used the correct procedure, but the task required you to choose why that procedure fit.”

“You described the graph accurately; the next step is to connect the pattern to the scientific model.”

Now the learner sees not only the missing answer but the missing kind of thinking.

Over time, this vocabulary helps students recognise demand independently. They begin to notice when a question is asking for recall, method selection, interpretation or evaluation.

That recognition itself becomes part of examination and learning competence.

Frequently asked questions

Is cognitive demand the same as Bloom’s taxonomy?

No. Bloom-related taxonomies are one family of tools for discussing cognitive processes. Cognitive demand is the broader idea of the mental processing required by a task. Other frameworks, including depth-of-knowledge approaches, emphasise different dimensions.

Is a difficult question always high demand?

No. Difficulty is observed performance. High demand concerns the reasoning structure. A question can be difficult for trivial or irrelevant reasons.

Can an easy question be high demand?

It can be cognitively rich for the intended population even if many learners answer correctly. Strong prior learning can make deep reasoning successful.

Should examination papers contain a fixed percentage of high-demand items?

That depends on the curriculum, purpose, age and certification requirements. There is no universal ideal percentage.

Does more writing mean more demand?

No. A learner can write a long memorised response. A short counterexample can demonstrate sophisticated reasoning.

Does multiple choice mean low demand?

No. Carefully designed selected-response items can require substantial analysis. But some constructs, such as extended explanation, need constructed evidence.

How can teachers know the actual cognitive demand?

Analyse the task, test it with students, examine solution strategies and review item performance. No single indicator is enough.

What if students find a supposedly high-demand item easy?

That may show excellent learning, prior familiarity or lower demand than expected. Investigate rather than assuming one explanation.

Is time pressure a form of cognitive demand?

Usually it is better treated separately as an execution constraint unless speed is explicitly part of the construct.

The deeper lesson: rigor is about the quality of thinking, not the amount of pain

Schools sometimes perform rigor.

Longer papers. Bigger numbers. Denser texts. More steps. Less time. More obscure wording.

Those features can make an assessment harder.

They do not automatically make it more intellectually serious.

Real cognitive demand asks the learner to do something worthwhile with knowledge: connect it, select with it, interpret through it, test it, generalise it, evaluate it, or use it to justify a conclusion.

That kind of difficulty is educational because it reveals the organisation of understanding.

A fair assessment should not remove challenge.

It should put the challenge in the right place.

The best question is not the one that defeats the most students.

It is the one that makes the intended thinking visible.

Sources

OECD, PISA 2025 Assessment and Analytical Framework (29 May 2026): https://www.oecd.org/en/publications/pisa-2025-assessment-and-analytical-framework_86c36975-en.html
OECD, Strengthening Latvia’s student assessment system (27 August 2026): https://www.oecd.org/en/publications/strengthening-latvia-s-student-assessment-system_56534ec0-en.html
OECD, Unlocking High-Quality Teaching — Ensuring cognitive engagement (2025): https://www.oecd.org/en/publications/unlocking-high-quality-teaching_f5b82176-en/full-report/ensuring-cognitive-engagement_998c3147.html
OECD, The Theory and Practice of Upper Secondary Certification (28 January 2026): https://www.oecd.org/en/publications/the-theory-and-practice-of-upper-secondary-certification_b3fea5ba-en.html