THE CORE AIM OF VOCABULARY MASTERY · FOUNDATION MODELS VOCABULARY · PRETRAIN → REPRESENT → ADAPT → PROMPT → APPLY
Foundation models vocabulary is the language used to describe large pretrained models that learn broad representations from substantial datasets and can then be adapted to many downstream tasks. Terms such as pretraining, foundation model, downstream task, adaptation, fine-tuning, prompting, transfer, multimodal and emergent capability matter because one model can become the starting point for many different applications.
The core aim of vocabulary mastery for foundation models vocabulary is reuse-and-adaptation clarity. Learners should be able to explain what broad capability was learned during pretraining, how a downstream task differs from the original training objective, and which adaptation method turns a general model into a useful application.
This page is the Foundation Models Vocabulary owner inside the eduKateSG Vocabulary hub. For model transfer, use Transfer Learning Vocabulary. For generative systems, use Generative AI Vocabulary.
Central proposition: Foundation-model vocabulary is mastered when the learner can separate broad pretraining from downstream adaptation and deployment.
The 60-Second Foundation Models Vocabulary Router
- Pretraining: corpus, objective, self-supervision, scale.
- Representation: embedding, parameter, feature, latent space.
- Adaptation: prompt, fine-tuning, instruction tuning, adapter.
- Task: downstream task, classification, generation, retrieval.
- Modality: text, image, audio, multimodal.
- Evaluation: benchmark, transfer, robustness, limitation.
Foundation Model and Application Are Different
A foundation model is a reusable pretrained base. An application combines that model with prompts, data, tools, interfaces, policies and task-specific logic. The model is an important component, but it is not the whole product.
A Worked Example: Pretraining
Pretraining exposes a model to broad data and an objective that encourages reusable representations or predictive capability. The result can later be adapted rather than trained from scratch for every new task.
A Worked Example: Downstream Task
A downstream task is a later use such as classification, summarisation, retrieval or image understanding. Performance on downstream tasks shows how effectively broad learned representations transfer to practical problems.
Adaptation Methods
Adaptation can include prompting, fine-tuning, instruction tuning, adapters or retrieval. These approaches differ in how much model behaviour is changed and how much new task-specific data or computation is required.
Scale and Capability
Foundation models are often associated with scale in data, parameters and computation, but size alone does not guarantee usefulness. Vocabulary should connect scale to capability, evaluation, cost and limitations rather than treating “larger” as a synonym for “better.”
How to Learn Foundation Models Vocabulary
- Separate pretraining from downstream use.
- Compare prompting and fine-tuning.
- Trace one model into several applications.
- Identify the modality involved.
- Study transfer-learning examples.
- Evaluate capabilities and limitations separately.
- Ask what application layers sit around the base model.
Common Foundation Models Vocabulary Mistakes
Confusing model and product
Repair: identify the surrounding application stack.
Treating every large model as interchangeable
Repair: compare training objective, modality, context and evaluation.
Assuming pretraining solves the downstream task automatically
Repair: examine adaptation and task-specific evidence.
Frequently Asked Questions
What is a foundation model?
It is a broadly pretrained model intended to serve as a reusable base for multiple downstream tasks or applications.
What is a downstream task?
It is a later task that uses or adapts capability learned during pretraining.
How can I learn foundation-model vocabulary?
Trace one pretrained model from training objective through adaptation to several real applications.
The Foundation Models Vocabulary Standard
Mastery means explaining what was learned broadly, what downstream task is required, how the model is adapted and how success is evaluated.
That is the standard: model language precise enough to distinguish reusable capability from finished application.