The Core Aim of Vocabulary Mastery | AI System Documentation Vocabulary

THE CORE AIM OF VOCABULARY MASTERY · AI SYSTEM DOCUMENTATION · PURPOSE → ARCHITECTURE → EVIDENCE → OPERATIONS → CHANGE

What should an organisation document before letting an AI assistant answer questions, search private resources or take action through connected software? AI system documentation vocabulary gives people a way to answer without hiding behind technical jargon. Words such as system inventory, intended use, architecture diagram, data flow, trust boundary, dependency, model card, runbook, decision record and change log describe different parts of one operational picture.

The core aim of vocabulary mastery for AI system documentation is intelligibility and recoverability. A new teacher, software engineer, reviewer or service owner should be able to understand what an AI system does, where its information comes from, who can authorise actions, what evidence supports its performance and how to correct or disable a component when something changes.

This page is the AI System Documentation Vocabulary owner in the eduKateSG Vocabulary Hub. For closely related owner pages, see AI Model Cards Vocabulary, AI Data Provenance Vocabulary and AI Audit Trail Vocabulary. Each explains one important component; this guide connects the entire system.

Central proposition: Good AI documentation explains not just a model, but the complete service around it: data, prompts, tools, permissions, people, evaluation, monitoring and repair.

The 60-second AI system documentation router

Why it exists: purpose, user, intended use, excluded use.
What it contains: system boundary, component, interface, dependency.
What it knows: data source, provenance, data flow, retention.
What it can do: tool permission, actor, approval, external action.
Why to trust a result: evaluation, limitation, safety case, evidence.
How to keep it working: runbook, owner, incident process, change log.

The AI system documentation architecture

Document or term	What it should describe	Question it answers
System inventory	A register of deployed AI use cases, owners, locations and status.	Which AI systems actually exist?
System boundary	The components and services included in the design under review.	What is inside and outside scope?
Architecture diagram	Major software, data and user components and their connections.	How does the system fit together?
Data flow diagram	How data enters, moves through and leaves the system.	Where does information travel?
Trust boundary	A point where the authority or trust level of information changes.	Where must untrusted content be handled carefully?
Model card	The model’s task, evaluation, limitations and intended uses.	What evidence applies to the model?
Runbook	Procedures for operating, troubleshooting and recovering a service.	What should an operator do?
Decision record	A record of a material design choice and its rationale.	Why was this choice made?
Change log	A record of relevant changes between released versions.	What is different now?
Responsibility matrix	Named responsibility for development, decisions, approvals and maintenance.	Who is accountable?

A model card is not the entire system specification

A model card describes a trained model: its intended task, evaluation and limitations. But imagine a study assistant that includes search, a vector database, an external model, tool calls and a parent-facing interface. The model card does not by itself specify which documents may be retrieved, whether the assistant can send messages, what gets logged or which person owns an incident.

That wider picture belongs in system documentation. The two documents should link to one another rather than duplicate every paragraph. If a model changes, the system overview can identify the affected interface or workflow while the updated model card explains changes in evaluation evidence.

Worked example: an education assistant and its five boundaries

Suppose a tuition site introduces an AI learning guide. A parent asks how a Primary 6 pupil can improve in mathematics. The assistant searches verified site articles, constructs an explanation and proposes a practice plan. The parent then requests a consultation. The system looks simple from the outside, but documenting its boundaries prevents several kinds of confusion.

Boundary	Documented behaviour	Reason it matters
User to application	Receives a question and optional learner-level context.	Clarifies consent, expectations and data minimisation.
Application to knowledge store	Retrieves approved, versioned educational content.	Distinguishes factual references from arbitrary webpages.
Knowledge store to model	Passages enter the model context as data, not as privileged instructions.	Preserves the instruction/data trust boundary.
Model to user	Generates an answer with source references and uncertainty where warranted.	Supports verifiability and appropriate educational advice.
Application to consultation workflow	A request may be prepared, but any external action requires the appropriate authority.	Clarifies tool permissions and approval.

A useful system diagram labels both the information and the permission boundary at each stage. It does not simply connect five boxes with arrows and assume that readers understand what those arrows permit.

System inventory versus component inventory

A system inventory lists operational AI use cases: who owns each one, what it serves and whether it is experimental or live. A component inventory lists technical parts and dependencies: models, runtime libraries, data sources, retrieval indexes, external APIs and credentials managed by the hosting environment. The former answers ‘Which services use AI?’; the latter answers ‘What does each service depend on?’

An AI model or data source may be shared across multiple services. An inventory is most useful when it records those relationships, because retiring a shared component without checking dependencies can break several downstream applications at once.

Requirements, assumptions and acceptance criteria

Functional requirements

These describe what the system must do. For example, it may answer syllabus-related questions using an approved source collection and show a route for users to verify or request help.

Nonfunctional requirements

These describe qualities such as response time, availability, accessibility, reliability, data retention and operational limits. A useful requirement is testable; ‘must be fast’ is less useful than a measured performance target suited to the workflow.

Assumptions

An assumption is a condition the design relies on but has not necessarily proven permanently. A system may assume sources are up to date or that a connected API returns stable formats. Naming assumptions supports change detection and review.

Acceptance criteria

These specify evidence needed before considering a release acceptable. They may cover correct use of approved content, robust behaviour on ambiguous questions, appropriate access controls and successful operation of human approval boundaries.

Data provenance and rights belong in the documentation

A dataset’s presence in a database does not demonstrate that it can be used for every purpose. For each material source, relevant metadata may include publisher, version, licence, collection method, limitations, authority level and review date. A source registry is therefore more useful than a list of links with no explanation of provenance or permitted use.

Document changes also matter. If a school’s published assessment structure is updated, the old source may remain in a retrieval index after the authoritative page has changed. Versioning, review dates and an update process help teams recognise what must be rechecked.

Architecture diagrams: three questions every arrow must answer

What moves? A query, document excerpt, model output, metadata record or action request.
Who has authority? The user, application, administrator, external source or service account.
What happens if it fails? Is the outcome recoverable, logged, retried, rejected or escalated?

These questions turn a decorative diagram into an operational map. They also keep people from confusing a source’s factual relevance with its authority to instruct the system.

Why the runbook matters after deployment

An operational runbook documents how to check the service, respond to specific alerts, safely disable an integration, verify a rollback and contact responsible operators. It should be short enough to use during an actual incident and specific enough that another authorised person can follow it.

For a study assistant, the runbook might describe how to remove an outdated syllabus source from retrieval, verify indexing state and suspend answer categories that are temporarily unsupported. It should not require staff to understand every detail of model training before taking necessary containment action.

Decision records: preserving the reason, not just the outcome

A design might choose a link-only external library rather than storing full third-party texts. Another might require human approval for sending messages. If the reasoning is never recorded, future maintainers may undo those decisions accidentally. A decision record captures the chosen option, alternatives considered, constraints and consequences.

Documentation needs owners and review triggers

Good documentation is not a one-off publication. Name the owner of each important section and the events that should trigger review: new models, changed provider terms, new tools, a data-source retirement, an incident, a new target audience or an assessment update. A stale system specification can be more misleading than a visibly incomplete one.

A practical one-page documentation exercise

Name the AI system, its owner and its intended users.
Write a six-word purpose statement, then expand it into a short description.
Draw the major components and mark all external services.
Trace one request from the user through data retrieval to its outcome.
Mark each trust boundary and any permission to change external state.
Name the model version, key sources and relevant evaluation evidence.
Write one failure scenario, the containment action and the responsible contact.
Add a review date, change log and links to supporting documents.

Common AI system documentation mistakes

Only documenting the model: describe the complete application and its external dependencies.
Diagram without permissions: mark authority and trust boundaries, not only data movement.
Every source is interchangeable: preserve origin, licence and review status.
A fixed screenshot is the system: link documentation to versioned configuration.
Evaluation results equal operational readiness: include human procedures, alerts and recovery.
Anyone can update the document: assign accountable owners and appropriate review controls.

Frequently asked questions

What is AI system documentation?

It is structured information describing an AI-enabled service’s purpose, architecture, data, model, permissions, evidence, operation and maintenance.

How is a system specification different from a model card?

A specification describes the complete service and its interfaces; a model card focuses on the characteristics, evaluation and limitations of a trained model.

What is a trust boundary?

It is an interface across which the authority or trust level of information or actions changes, requiring explicit handling rules.

What is the difference between a runbook and a design document?

A design document explains how and why the system is structured; a runbook gives operators practical procedures for keeping it working or handling failure.

How often should system documentation be reviewed?

Review it when material dependencies, permissions, models, data sources, risks or operating conditions change, as well as at a cadence appropriate to its importance.

Authoritative reference and related learning

The NIST AI Risk Management Framework treats governance, context, measurement and management as ongoing elements of trustworthy AI work. The associated NIST AI RMF Playbook offers voluntary, context-dependent actions and examples rather than a mandatory universal documentation template.

Where this fits in the eduKateSG vocabulary ecosystem

The AI system documentation vocabulary standard

Mastery means that a reader can explain the system boundary, owner, data and source lineage, model evidence, trust and permission boundaries, incident procedure and change history. The result should be understandable to someone who did not build the system—and usable when the system has to be fixed.