The gold standard of Evaluation is a defensible judgement about how well a learning experience, programme, product or service serves a stated purpose. It makes criteria visible, chooses suitable evidence, checks the comparison and explains who benefits, what remains uncertain and what decision should follow. A number does not evaluate itself.
This root guide connects measurement, verification, scientific research, school assessment, service design, organisational decisions and fair access across the eduKateSG ecosystem. The fictional worked cases show why a polished dashboard may still leave the most important question unanswered.
Begin with Verification for checking requirements, Judgement for the reasoning behind a conclusion, or Education for the larger learning purpose. The chapters offer educational methods rather than professional certification, legal advice or a universal scoring formula.
Find the part you need
Start with the decision, build the comparison, challenge the evidence or use a worked example.
Chapter index
Clarify the decision
Build the evaluation
Challenge the evidence
- Let quantitative and qualitative evidence work together
- Look for selection bias and missing participants
- Check who benefits and who is left behind
- Triangulate sources without double-counting agreement
- Report uncertainty in terms a decision maker can use
- Separate ethical priorities from empirical findings
Connect to practical life
Work through cases
Choose your next route
CHAPTER 1 OF 23 · Clarify the decision
1. What makes evaluation different from measurement
Evaluation asks how well a programme, product, system or learning experience serves a stated purpose. Measurement supplies observations; verification checks defined requirements; evaluation connects those results to criteria and a consequential judgement. A count does not evaluate itself. A confident recommendation needs a visible bridge from facts to the decision.
Imagine a school reporting that 90 per cent of children attended an enrichment programme. Attendance establishes participation, not independent learning, equity of access or value for time. A defensible evaluation asks what the programme was supposed to change and who is able to act on that finding.
Compare Verification, Measurement and Judgement. They overlap, but confusing them makes weak reports sound stronger than their evidence permits.
CHAPTER 2 OF 23 · Clarify the decision
2. Choose the decision before collecting favourable evidence
Name the person or body authorised to decide, the available alternatives and the timing. An evaluation that arrives after a decision can no longer be influenced may still support later learning, but it has not served its intended immediate purpose.
For a pilot reading programme, alternatives may include expanding it, redesigning it, protecting one successful component or using the time for another intervention. Each choice asks slightly different questions about outcomes, costs, staffing and who benefits.
Write the realistic options before studying results so the evaluation is not simply a search for support for a preferred announcement. Use Decision Making when comparing trade-offs and uncertain future outcomes.
Success criteria convert aspirations into conditions that can be checked. A service might need to be accurate, safe, understandable, accessible and reliable. Speed and cost can matter without automatically overriding those minimum requirements.
State whose needs shape the criteria. A library kiosk that works for confident smartphone users may still fail a patron who needs accessible assistance. The relevant outcome is whether people can complete the intended task, not whether an attractive interface was installed.
Separate mandatory conditions from preferences and explain their authority. If criteria change after the scores are known, disclose why. Quality and Governance help distinguish a genuine duty from an optional improvement.
CHAPTER 4 OF 23 · Build the evaluation
4. Describe the mechanism from activity to outcome
An activity is something performed; an output is what it produces; an outcome is the important effect. A teacher’s lesson is an activity, a corrected worksheet may be an output, and improved independent reasoning is the learning outcome we hoped would follow.
Draw the assumed causal path. Explicit explanation may repair a misconception; appropriate practice may stabilise the method; unfamiliar problems may reveal whether understanding transferred. Each link is a hypothesis that can be examined, not an automatic guarantee.
The Management and Learning guides distinguish doing the work from establishing the result. A good evaluation measures enough of the chain to find where it may have broken.
CHAPTER 5 OF 23 · Build the evaluation
5. Use comparisons that answer the actual claim
A difference needs a relevant reference. It might be an agreed minimum, previous performance, a comparable group or a feasible alternative. Explain why that comparison is suitable for the question and what conditions make it fair.
When examination scores rise after a new programme, the improvement is real as a description. Attributing all of it to the programme requires additional attention to maturation, practice effects, question difficulty, other teaching and selection into the intervention.
A randomised comparison is not appropriate or necessary for every decision. Direct acceptance testing can be enough for a product requirement, while claims about causation often need stronger designs. Be exact about what the chosen comparison proves.
CHAPTER 6 OF 23 · Build the evaluation
6. Choose measures that represent valued outcomes
Convenient metrics can be misleading. Clicks are not comprehension; complaint counts are not necessarily service quality; a low average waiting time does not mean every visitor received a usable service. Define the concept before choosing the number.
State the unit, denominator, population and measurement period. An improvement of 15 percentage points is not the same as a 15 per cent relative increase. A student average does not describe the learners whose foundations remain weak.
Include direct checks where possible: can the user finish the task, can the learner explain the method, does the device work under intended conditions? Read Measurement and Engineering Quality Assurance for deeper methods.
CHAPTER 7 OF 23 · Build the evaluation
7. Make weights and unacceptable trade-offs transparent
A scorecard often combines safety, value, access, cost and efficiency. Those categories involve human choices. Hiding their weights inside a single total does not make the result objective.
Some failures must not be compensated by high scores elsewhere. A service that violates an essential safety condition cannot automatically become acceptable because it is inexpensive and popular. Separate pass-or-fail constraints from desirable advantages.
Test whether rankings change under plausible weights and explain why the selected weights fit the purpose. A scoring rubric can make criteria more consistent without removing judgement from their design.
CHAPTER 8 OF 23 · Challenge the evidence
8. Let quantitative and qualitative evidence work together
Numbers reveal patterns and magnitudes; carefully collected observations and interviews can explain processes and user experience. Both require appropriate sampling, clear definitions and fair interpretation.
A new lesson sequence may raise short-test averages while student explanations reveal reliance on memorised steps. A shorter online queue may conceal callers who never reached the booking screen. The additional evidence does not cancel the first result; it changes the scope of the conclusion.
Use Research and Evidence-Based Thinking to examine source quality. Do not present one compelling quotation as the prevalence of an experience or dismiss a serious access failure because the mean improved.
CHAPTER 9 OF 23 · Challenge the evidence
9. Look for selection bias and missing participants
Who enters the data matters. Volunteers to a phone-based survey may already be comfortable using smartphones. Students who finish a programme may differ from those who leave. A pilot recruited from one neighbourhood may not represent an entire city.
Show recruitment rules and missing observations. Ask whether missing records might be related to failure, difficulty or exclusion. Reporting results only for successful completers can make an intervention appear stronger than it is.
When making causal claims, examine plausible confounders: third factors that affect both exposure and outcome. A credible evaluation welcomes these questions rather than treating them as disloyal objections.
CHAPTER 10 OF 23 · Challenge the evidence
10. Check who benefits and who is left behind
An average may improve while access worsens for a subgroup. A classroom programme may suit students who already know the prerequisites while making slower progress among pupils with gaps. A digital redesign may disadvantage people without reliable devices.
Evaluate relevant differences without publicising private individual data. Design suitable alternatives where required, and distinguish equal treatment from equitable opportunity to obtain the intended service.
The Accessibility Programme Management guide explores this responsibility in service systems. The Clementi Mathematics reference shows why an individual’s unstable reasoning step matters even when a class average looks healthy.
CHAPTER 11 OF 23 · Challenge the evidence
11. Triangulate sources without double-counting agreement
Two sources are not independent merely because they appear on different websites. One may copy the other’s dataset, definition or mistaken figure. Track provenance before treating agreement as confirmation.
When results conflict, compare collection dates, denominators, instruments, question wording and populations. A discrepancy may reveal a legitimate difference in what was measured rather than deception or incompetent analysis.
Use Verification for traceable testing and Investigation when a material conflict requires reconstructing the source history. The question is not how many figures support a story, but how strongly the evidence supports the claim.
CHAPTER 12 OF 23 · Challenge the evidence
12. Report uncertainty in terms a decision maker can use
An estimate may depend on sampling, measurement variation, confounding or assumptions about future demand. Present uncertainties that could alter the decision. Do not manufacture a precise percentage of confidence when no justified method supports it.
Distinguish practical importance from statistical detectability. A difference can be real but too small to matter, while an uncertain difference may still justify further attention if the potential harm is severe.
Sensitivity analysis asks whether a recommendation survives plausible changes to uncertain inputs. Robust conclusions should withstand reasonable challenges; fragile conclusions should name the extra evidence that would be valuable.
CHAPTER 13 OF 23 · Challenge the evidence
13. Separate ethical priorities from empirical findings
Facts help inform decisions, but criteria such as dignity, affordability and opportunity are also value choices. Explain who set the priorities and provide affected people an appropriate chance to correct factual errors.
Disclose relevant interests, funding and access limitations. A review conducted for an organisation can be useful, but independence should not be claimed unless the arrangement justifies it. Protect confidential and personal information.
The Integrity and Governance roots connect honesty in evidence with responsibility for the final choice. Transparency means the reasoning can be examined, not that every personal record must be public.
CHAPTER 14 OF 23 · Connect to practical life
14. Evaluate learning through transfer and explanation
A learner may answer a rehearsed algebra question quickly yet be unable to explain why the same operation must be applied to both sides of an equation. A high familiar-question score can hide fragile understanding.
Check whether the student can retrieve the method later, explain it in their own words, recognise a changed problem and repair an error. Use mistakes diagnostically rather than only as marks to deduct.
This is the deeper principle demonstrated by the Clementi Mathematics guide: teach the underlying structure, observe the exact point where thinking breaks and verify independent use. Teacher Education and Feedback develop the instructional consequences.
CHAPTER 15 OF 23 · Connect to practical life
15. Evaluate services from the user’s complete journey
A process may meet its internal milestones while failing the person it serves. Opening a new appointment portal is an output; successful access to suitable appointments is a user outcome. Reducing handling time matters only alongside accuracy, accessibility and safety.
Test the complete journey under realistic conditions. Include users who need assistance and check downstream handoffs. A promising pilot may still require correction, staff training and an observation period before wider rollout.
Compare Operations Management, Workflow Design, Coordination and Management. Each examines a different part of converting a designed process into dependable service.
CHAPTER 16 OF 23 · Work through cases
16. Worked case: scores rise but algebraic reasoning remains fragile
A fictional class completes one month of intensive drills. Average algebra quiz scores rise from 68 to 84. The school considers replacing several explanation lessons with more drills because the graph appears convincing.
When numbers and question forms change, some students cannot justify their operations. The evidence supports better performance on the rehearsed items; it does not yet establish confident transfer.
Repair the evaluation using unfamiliar questions, spoken explanations, delayed practice and error logs. Retain the score improvement as a finding without using it to support a stronger claim than it can bear. The resulting teaching decision may combine structured explanation with useful practice.
CHAPTER 17 OF 23 · Work through cases
17. Worked case: the booking dashboard excludes those who fail
A fictional clinic launches online booking. Completed online bookings now average three minutes instead of nine. Managers propose removing the assisted telephone route.
The dashboard observes only successful online users. It omits people who abandoned the website or could not start using it. Interviews and call records show that some patients still need accessible help.
Revise the denominator to include the intended user population. Measure completion, abandonment, timeliness, accuracy and availability of an effective alternative. A faster digital service can be an improvement without proving that every other access path is obsolete.
CHAPTER 18 OF 23 · Work through cases
18. Worked case: apparent energy savings have another explanation
A fictional school adds reflective window film and consumes 12 per cent less electricity the following month. The supplier attributes the entire saving to the film.
The evaluation has not yet considered temperature, occupancy, timetable changes, other repairs or metering boundaries. A cooler month may explain part of the difference.
Keep the observed saving separate from estimated causal contribution. Use a justified comparison or weather adjustment if feasible, and state the limits. The conclusion can be promising without assigning a false level of certainty.
CHAPTER 19 OF 23 · Work through cases
19. Worked case: pilot volunteers do not represent the community
A fictional city pilots a public-information app with eighty volunteers recruited through a smartphone advertisement. Ninety-three per cent say that the service is easy to use, and officials propose replacing printed notices city-wide.
The recruitment method excludes many of the people for whom an alternative channel may be essential. The reported satisfaction result applies to the participants who responded, not automatically to the whole population.
Test the intended service with more representative users and investigate accessibility, languages and completion of real tasks. Preserve the positive pilot result but disclose its limited reach before recommending wide adoption.
CHAPTER 20 OF 23 · Choose your next route
20. Eight questions before accepting an evaluation
What decision does this report support? Which valued outcome is being judged? Who chose the criteria? Are the measures valid? Is the comparison fair? Whose evidence is missing? What uncertainty might change the conclusion? What harm or unintended effect could the recommendation create?
Mark each answer as established, provisional or unresolved. A weak answer to a critical question should not be hidden inside a composite score.
Separate facts, assumptions, selected values and recommendations in the report. If a decision needs further evidence, state exactly what additional check would be useful. The goal is proportional confidence, not a theatrical certainty.
CHAPTER 21 OF 23 · Choose your next route
21. The Evaluation specialist reading library
Use the routes below according to the problem: verifying a requirement, choosing an outcome measure, evaluating a learning programme, studying causal claims or delivering service improvement. These are connected eduKateSG learning resources, not universal certifications.
Begin with Verification, Judgement, Decision Making, Research or Education and follow the deeper subject branches.
Criteria, measurement and checking
- Verification
- Quality
- Quality Control
- Measurement
- Engineering Quality Assurance
- Process Mapping
- Workflow Design
Reasoning, causal evidence and decisions
- Research
- Science
- Mathematics
- Evidence-Based Thinking
- Reasoning
- Critical Thinking
- Judgement
- Decision Making
- Investigation
- Problem Solving
Learning and human performance
- Learning
- Feedback
- Teacher Education
- Higher Education
- Clementi Mathematics
- Collaboration
- Leadership
- Accountants
- Nurses
- Pilots
Delivery, access and governance
- Management
- Coordination
- Governance
- Operations Management
- Accessibility Programme Management
- Root Cause Analysis
- Integrity
- Time Estimation
- Education
Further reading: Scoring Rubric Validation · Evidence-Informed Decisions · The Gold Standard directory.
Is evaluation the same as testing? Testing can provide evidence; evaluation interprets that evidence against criteria and a purpose. A passed test may not settle broader user outcomes.
Must every evaluation use an experiment? No. The design should be appropriate to the claim. Causal claims generally require more care than straightforward checks against requirements.
Can qualitative evidence be rigorous? Yes, when collection and interpretation are systematic and limitations are disclosed. Its purpose and inferential reach differ from a representative numerical estimate.
Does a rubric make judgement objective? It makes the criteria easier to inspect, but choosing criteria and weights still involves judgement.
What if an average improves while some people lose access? Report both effects and examine the relevant obligations and alternatives rather than concealing the difference in a mean.
When is evaluation complete? When the authorised decision has sufficiently reliable evidence for its stakes, important gaps are visible and the next action can be justified. Some impacts need continuing review.
CHAPTER 23 OF 23 · Choose your next route
23. Apply the method to one modest decision
Choose a real but low-stakes decision in a class, organisation or household. State the purpose, two feasible alternatives, three criteria and one condition that must not fail. Write what evidence would distinguish the options before collecting it.
Collect only information you are entitled to use. Keep units, time periods and source definitions visible, and invite another reader to question your comparison. If the data cannot settle the choice, specify the missing evidence instead of inventing a winner.
A useful evaluation improves the next decision. The Clementi Mathematics reference shows this at the level of a single learner’s reasoning; the Education root carries it into careers, professional practice and civic life.