An innovative nation can spend more on research, award more grants and celebrate a rising technology ranking without knowing whether its programmes actually caused additional useful innovation. Excellent policy begins with a genuine problem, not a favourable announcement.
The Gold Standard of Innovation Policy asks whether public action changed research, adoption or social outcomes compared with a reasonable alternative. It follows scientific capability into testing, implementation, diffusion and accountability, rather than confusing programme spending with public value.
This guide includes 26 subject-specific chapters, two worked fictional calculations, reader checks, evaluation limitations and links to primary international measurement guidance.
Chapter index
Foundations, instruments and additionality · Chapters 1–9
- Innovation policy has to change what happens, not only announce ambition
- Distinguish invention, research, adoption and innovation
- Write the underlying problem in testable form
- Identify the market or coordination failure
- Map the innovation system, not just the laboratory
- Define missions with boundaries and exit conditions
- Choose instruments that fit the cause
- Keep research and development statistics distinct
- Worked example: estimate additionality rather than gross growth
Selection, infrastructure and diffusion · Chapters 10–18
- Evaluate who applies and who is missing
- Use fair selection and conflict management
- Support infrastructure and standards as public goods
- Design responsible experimentation and sandboxes
- Make public procurement an evidence pathway
- Measure adoption and diffusion beyond the funded group
- Account for failure, learning and portfolio risk
- Worked example: divide cost by the right adoption denominator
- Separate output, outcome and public value
Evaluation, fairness and connected disciplines · Chapters 19–26
- Use credible comparison groups and uncertainty
- Consider distribution, displacement and opportunity cost
- Avoid interpreting rankings as causal evaluations
- Prevent gaming through balanced measures
- Structure governance across agencies and partners
- Apply an independent readiness and evaluation checklist
- Frequently asked questions about innovation policy
- Connect to the Gold Standard subject ecosystem
1. Innovation policy has to change what happens, not only announce ambition
Innovation policy concerns public choices that influence the creation, adoption and diffusion of useful new products, services and processes. Governments may support research, skills, infrastructure, competition, procurement, standards or financing because market incentives alone do not always deliver socially valuable innovation. The question is what additional capability or benefit public action creates compared with a credible alternative.
A gold-standard policy states a problem, a theory of change, an instrument, an affected population, a cost, measurable outcomes and a realistic way to learn when the intervention does not work. High technology spending, publicity or a favourable ranking are not by themselves evidence of impact.
2. Distinguish invention, research, adoption and innovation
A new scientific finding may be important research without yet being implemented as an innovation. A patent may represent a legal claim without demonstrating a product’s use. A firm adopting an established technology for the first time may improve productivity in a way that matters greatly to its customers and workers. Policy should not collapse all these activities into the same score.
The OECD/Eurostat Oslo Manual 2018 distinguishes innovation from invention through implementation and offers guidance for measurement. Use its definitions consistently when comparing firms or programme years. If eligibility changes, separate the change in measurement from genuine growth in innovative activity.
3. Write the underlying problem in testable form
Start with a clear friction: small manufacturers cannot test new processes cheaply; researchers cannot access specialist instrumentation; promising clinical tools lack evidence needed for adoption; or smaller firms face difficulty obtaining long-horizon capital. Different problems demand different public instruments, even when all are described as ‘supporting innovation’.
Gather baseline evidence and competing explanations. Interview users, suppliers, researchers and non-participants, but do not treat testimony as a substitute for observed constraints. A subsidy cannot solve a missing professional standard if firms still cannot integrate the resulting products.
4. Identify the market or coordination failure
Knowledge spillovers may prevent inventors from capturing the full social value of research. Capital markets may hesitate to finance uncertain long-term projects. Network technologies may suffer from incompatible standards or require many organisations to move together. Public purchasing might demand evidence that no individual startup can afford to produce.
Each diagnosis has a counterargument. Sometimes poor adoption reflects a weak product, not a market failure. Sometimes a regulatory burden protects safety rather than merely slowing innovation. Record what evidence would disprove the diagnosis before selecting the policy instrument.
5. Map the innovation system, not just the laboratory
Discovery, prototyping, manufacturing, testing, regulation, procurement, distribution, skills and after-sales service form a system. A scientific breakthrough can stall at scale-up because suppliers, technicians or standards are missing. Product adoption may depend more on organisational practice than on a new algorithm.
Draw the path from knowledge to practical use, recording bottlenecks and responsibilities. Identify where universities, firms, consumers, public buyers, investors and communities interact. A systems map is valuable only when it changes the intervention chosen; decorative arrows do not measure a functioning ecosystem.
6. Define missions with boundaries and exit conditions
A mission can organise resources around a concrete public challenge, such as reducing the energy required for particular industrial processes while preserving output. Specify the affected process, baseline, time horizon, acceptable trade-offs and evidence threshold. Avoid a mission so vague that almost any project can claim it contributes.
Some missions require research, while others require procurement reform, interoperable infrastructure or workforce skills. Set review points and an exit route. Long-term ambition should not mean that a programme continues forever without a test of progress or an opportunity to redirect spending.
7. Choose instruments that fit the cause
Research grants, collaborative centres, demonstration facilities, standards support, targeted procurement, prizes, tax incentives, loans and regulatory sandboxes solve different frictions. A grant may support exploratory knowledge with uncertain payoff; procurement can create early demand for a proven solution; shared facilities reduce duplicated fixed costs.
State why an instrument is more appropriate than an alternative, its eligibility rules and plausible unintended effects. A tax incentive may reach established firms with taxable income more readily than young businesses; a grant might favour applicants who write better proposals rather than those with the highest expected social return.
8. Keep research and development statistics distinct
Research and development encompasses basic research, applied research and experimental development under the methodology of the OECD Frascati Manual 2015. Not every purchase of software is R&D, and not every innovation activity fits the formal R&D definition. A programme may appropriately fund adoption or training, but should report that separately.
Record whether a figure is spending, participants, researchers, outputs or outcomes. Convert currencies and periods only with disclosed assumptions. Spending intensity can show commitment; it cannot alone prove additional innovation, productivity or welfare.
9. Worked example: estimate additionality rather than gross growth
In a fictional programme, 100 supported firms increased their total validated prototypes from 40 before support to 56 afterwards: a rise of 16. A comparable group of non-supported firms increased from 42 to 50: a rise of 8. The simple difference-in-differences estimate is (56 − 40) − (50 − 42) = 8 extra validated prototypes for the supported group over that period.
This estimate is not automatically causal. It depends on the comparison group representing what would have happened without support, comparable measurement and a credible parallel-trends argument. Selection bias, different sector shocks and changes in the meaning of ‘validated prototype’ can invalidate the estimate. Report uncertainty and pre-program trends before attributing eight additional results to the policy.
10. Evaluate who applies and who is missing
Open calls often reward organisations with grant-writing teams, established networks and spare capacity. Eligible smaller firms may never apply because paperwork, matching-fund demands or uncertainty create barriers. A programme with many good funded proposals can still exclude the population that needed the support most.
Track the funnel from awareness to application, eligibility, assessment, award, delivery and subsequent outcome. Segment by relevant firm size, sector or region without revealing confidential applicant details. Publish reasons for rejection in an appropriate aggregate form and test whether outreach changes representation.
11. Use fair selection and conflict management
Independent evaluation requires explicit scoring criteria, written conflict declarations and decision records. A panel should not discover after a bid is opened that a reviewer supervises the principal investigator or stands to benefit from the supplier chosen. Scores may aid consistency, but they do not eliminate hidden judgment.
Document scientific merit, feasibility, societal value, team capability and opportunity cost separately. Avoid demanding precise commercial forecasts from genuinely exploratory research. Appropriate review standards differ between a basic-science programme and an industrial scale-up award.
12. Support infrastructure and standards as public goods
Shared testbeds, metrology, computing capacity, pilot production lines and interoperable standards can enable many users rather than one award recipient. Their value often appears through lower verification costs or faster adoption across organisations. Capacity without access rules may simply subsidise a small incumbent group.
Measure utilisation, queue time, accessibility, reproducibility of tests, technical support quality and actual projects that reach deployment. Retain a maintenance and staffing budget. A state-of-the-art laboratory that cannot calibrate its equipment or schedule outside users is not delivering the promised infrastructure benefit.
13. Design responsible experimentation and sandboxes
Regulatory experimentation can help authorities observe new products under controlled conditions, but ‘sandbox’ should not mean absence of protections. Specify who is exposed to risk, which safeguards remain, what is monitored, when trials stop and how findings will inform wider rules. Safety and privacy rights are part of the experiment design.
Compare sandbox participants with the wider market before claiming general applicability. An intervention that works for carefully selected small-scale pilots may fail when many users, different systems or new incentives enter. Document limits and responsibilities for harm before recruitment begins.
14. Make public procurement an evidence pathway
Public purchasers can define a real service problem, publish performance requirements and permit multiple approaches to compete. They may help create an early market for useful innovation, but should avoid writing specifications that merely repeat one vendor’s product features. Independent acceptance tests should matter more than promotional claims.
Specify lifecycle cost, interoperability, accessibility, maintenance, security, failure response and termination options. A low initial bid can become expensive when the system cannot export data or a supplier controls all future integration. Procurement evaluation needs evidence of delivered use, not only a signed contract.
15. Measure adoption and diffusion beyond the funded group
A prototype that works in the original research team has not necessarily diffused to users. Follow adoption by other firms, sectors or regions and ask whether support created tools, practices or know-how that continue after the grant ends. Distinguish new adopters from returning users and nominal accounts with no sustained use.
Describe diffusion barriers: worker training, switching costs, complementary investments, trust and local conditions. Evidence of benefit should include user outcomes where practical. Policymakers should not reward one highly visible demonstration more than a quieter process improvement that many organisations adopt safely.
16. Account for failure, learning and portfolio risk
Innovative work includes genuine uncertainty. A robust portfolio can contain experiments that do not reach deployment; the issue is whether failure was bounded, informative and openly evaluated. Punishing every unsuccessful project can encourage inflated claims and selection of only predictable work.
Set milestone evidence suited to maturity. Early research may produce reproducible methods or negative findings; a pilot should test usability and integration; a mature service should demonstrate sustained performance and manageable cost. Stop or redirect projects when evidence contradicts the theory of change.
17. Worked example: divide cost by the right adoption denominator
Imagine a fictional demonstration programme that spends S$2.4 million, registers 1,000 expressions of interest and records 600 verified organisations actively using the improved process. The observed spending per verified adopter is S$2,400,000 ÷ 600 = S$4,000. Dividing by 1,000 would give S$2,400 per interested organisation, a different and less relevant quantity.
Even the S$4,000 ratio is not cost-effectiveness evidence without a counterfactual, adoption duration, the net benefits and any private contributions. Some of the 600 might have adopted without support. Report direct programme spending separately from capital supplied by firms, and define ‘active use’ before outcomes are known.
18. Separate output, outcome and public value
Inputs include funding and staff time. Outputs include published studies, tested prototypes and training sessions. Outcomes include improved productivity, safer services, lower emissions or wider access that can plausibly be connected to the intervention. Public value asks how those benefits compare with costs, distributional effects and alternatives.
Select indicators before programme launch where feasible. Measure maintenance burdens and harms as well as success, particularly for technologies that affect safety, employment or privacy. Avoid counting patents, startups and public headlines as interchangeable units of social welfare.
19. Use credible comparison groups and uncertainty
Random assignment may be practical for oversubscribed small grants or staged invitations, but many policies require quasi-experimental methods such as matched comparisons, regression discontinuity near eligibility cutoffs or time-series analyses. Every method has assumptions and failure modes.
Publish sufficient methodological detail for a reviewer to test those assumptions. Include missing-data handling, confidence intervals where appropriate, sensitivity analyses and evidence of spillovers between funded and unfunded groups. An elegant model cannot correct an invented or unrepresentative baseline.
20. Consider distribution, displacement and opportunity cost
Innovation benefits can be uneven. A subsidised firm may recruit workers from neighbouring employers rather than expanding total skilled employment. New processes may improve output while creating transition costs for workers or excluding small suppliers. Aggregate national growth may conceal vulnerable groups bearing concentrated losses.
Assess who gains, who pays and which complementary supports are needed. Include counterfactual public spending: could the same funds have improved basic infrastructure, school laboratory access or health service reliability? This does not make innovation unimportant; it makes investment decisions more accountable.
21. Avoid interpreting rankings as causal evaluations
Composite indices can support international comparison and reveal questions worth investigating. The WIPO Global Innovation Index reports a basket of innovation-related indicators. A change in rank can also reflect other economies moving, indicator definitions and data availability.
Use underlying indicator values and comparable time series rather than presenting rank change as proof that a specific grant succeeded. A country’s innovation ecosystem has many determinants outside one agency’s control. Ratings are diagnostic context, not a programme-level counterfactual.
22. Prevent gaming through balanced measures
If funding renewals depend exclusively on patent counts, teams may pursue narrow, weakly used patents. If every project is judged by jobs reported at completion, managers may count temporary contract roles as durable employment. Incentives shape reporting, so performance systems must anticipate predictable adaptations.
Combine documented technical progress, independent validation, adoption evidence and responsible conduct. Audit data definitions and a sample of cases. Penalise deliberate manipulation while permitting honest reporting of experimental failure and uncertain outcomes.
23. Structure governance across agencies and partners
Innovation initiatives often span research funders, education, finance, procurement and sector regulators. Assign a lead owner for the outcome and delineate each partner’s authority over grants, standards, safety and evaluation. Cross-agency committees are useful only when they resolve trade-offs and maintain clear accountability.
Publish decisions, conflict safeguards, programme review dates and the process for external challenge. Avoid creating overlapping incentives where firms can be reimbursed multiple times for the same eligible spending without disclosure. Keep administrative burden proportional to the award and risk.
24. Apply an independent readiness and evaluation checklist
Before approving a programme, require a problem statement, baseline, target group, causal logic, instrument rationale, alternatives, eligibility and selection criteria, risk assessment, funded budget, evaluation design, decision rights and review schedule. Verify what data can actually be collected rather than choosing indicators for convenience.
During delivery, inspect a sample of reported outputs and user outcomes. At the decision gate, compare evidence with the original predictions, analyse adverse effects and decide whether to scale, redesign or stop. A gold standard includes a defensible record of decisions not to continue.
25. Frequently asked questions about innovation policy
Does more R&D spending guarantee greater innovation? No. It may strengthen capability but outcomes also depend on institutions, markets, talent, demand and use. Are patents always the best metric? No. Many valuable improvements are not patented, and patents differ widely in significance. Does a successful pilot prove readiness to scale? No; scale changes the operating context.
Can government make innovation happen through grants alone? Rarely. Standards, skills, infrastructure, procurement and competition often matter. Is a failed research grant necessarily waste? Not if its risk was justified and its findings were credible, documented and useful. Should every project use a randomised trial? No; evaluation design must fit ethics, implementation and feasibility.
26. Connect to the Gold Standard subject ecosystem
For economic incentives and social returns, continue to Economics, Finance and Industrial Organisation. For methods, follow Experimental Design, Statistics and Education.
Primary reading routes: the Oslo Manual, Frascati Manual and WIPO Global Innovation Index. All numerical examples here are fictional teaching cases. A credible policy identifies a problem, makes a defensible bet, measures what changed and openly revises the course.