eduKate Singapore

VIEW THIS AS

Auto mode follows the Route Engine until you choose a viewpoint.

YOU ARE HERE

ROUTE CHECK

CONNECTED TO

WHAT NEXT

Use the canonical route for this room, or HELP if you are unsure.

Next useful routeCanonical hubHELP

How AI Works | Catastrophic Forgetting — Sequential Training, Replay and Protected Weights

—

by

eduKate Asia

Did you know? A neural network can become better at a new task while suddenly losing accuracy on a task it previously performed well. This is catastrophic forgetting, also called catastrophic interference, in sequential or continual learning. It happens when updates needed for new examples change shared parameters that supported earlier skills. Better performance on today’s data is not proof that yesterday’s capability survived.

The practical repair is to test old and new capabilities separately after every important training stage, preserve representative earlier information when lawful and feasible, or constrain changes to parameters known to matter for old tasks. Replay, regularisation methods such as elastic weight consolidation, distillation and architectural separation offer different trade-offs. None guarantees retention under every domain shift; trustworthy continual learning requires a matrix of task-specific evaluations, not just a combined final score.

This article derives the mechanisms, solves numerical retention and regularisation examples, compares original research and demonstrates transfer to a changing workplace classifier. All training outcomes and operational figures used for illustration are fictional. The subject is current machine learning and should not be mistaken for a claim that artificial superintelligence exists.

Contents

Why training a second skill can damage the first

A neural network stores useful distinctions across many learned parameters. An early image classifier might use features that separate handwritten digits, while a later task rewards different combinations of features to distinguish symbols or visual textures. When the same shared parameters are updated for the new task, the older decision boundary can move. The first task’s training examples may no longer be present to push back against destructive changes.

This differs from an ordinary temporary fluctuation in validation accuracy. Catastrophic forgetting refers to a substantial loss of previously learned performance associated with sequential adaptation. In some settings the loss is dramatic; in others it is limited or offset by beneficial transfer. The honest diagnosis uses actual held-out performance on earlier tasks before and after learning the new task.

Think of two customer-support categories that share words. A classifier first learns to identify billing complaints and later is fine-tuned only on subscription cancellations. If the new updates treat every mention of a payment as evidence of cancellation, billing-class distinctions may deteriorate. That example describes a possible mechanism, not an unavoidable property of every fine-tuned model.

The new task can also be genuinely incompatible with the old one, or labels can change meaning. If a category’s definition changes, retaining the old rule may be undesirable. This is why ‘avoid all forgetting’ is not a complete objective. First define which old capability remains required, which knowledge has become obsolete, and which data may still be used.

The 2017 original research Overcoming catastrophic forgetting in neural networks by Kirkpatrick and colleagues investigated sequential tasks and a method for protecting important weights. It is evidence that interference and stability-plasticity trade-offs can be studied experimentally, not evidence that one technique cures every form of forgetting in modern models.

Sequential, joint and changing-distribution training are different questions

Under joint training, examples from all relevant tasks can be mixed together during optimisation. Each update receives information about old and new requirements, though not necessarily in equal proportions. Under sequential training, the model may first receive task A data and then only task B data. The second process can remove the corrective signal from A while B updates change shared weights. Comparing these protocols without naming the available data is misleading.

Online continual learning adds further constraints: data arrive in a stream, perhaps only once, and task boundaries may not be labelled. A company may not know that a new ticket type has begun to dominate its incoming messages. A system that knows the task ID at inference has a different problem from one that must infer the task from the input. Task-incremental and class-incremental evaluations should not be treated as equivalent.

Distribution shift matters but is not identical to forgetting. A deployed model can perform worse because the world changed even if its parameters remain frozen. Catastrophic forgetting concerns a performance decline after training updates, measured against appropriate earlier-task tests. To separate these explanations, evaluate the old frozen model and the updated model on a fixed old-task test set, while also examining current data drift.

A change in label mapping can masquerade as forgetting. If one dataset encodes class A as integer zero and another encodes it as integer one, the final evaluation can look disastrous for a data-integration reason. Verify dataset versions, preprocessing, tokenisers, output heads and evaluation labels before concluding that old representations have been erased.

Another confounder is simply using the wrong test set. If the final score covers only task B, it tells you nothing about retained A performance. If it combines tasks with heavily unequal sample counts, a small old-task subgroup can be hidden inside a high aggregate score. The evaluation design should expose the question rather than making it easy to overlook.

The stability-plasticity dilemma in parameter space

Learning requires changing parameters so that the model can represent new evidence. Retention requires preserving enough parameter relationships to continue handling old examples. Those goals can conflict, but they do not always conflict. Two tasks may share useful features so the new training improves the first one; other tasks may demand changes to precisely the weights the first task relied upon.

Imagine a model with only two parameters, theta-one and theta-two. Task A achieves good performance near (1,1). Task B’s loss would decrease if both parameters moved toward (2,0). The direct line from the first optimum toward the second changes each parameter. If task A depends strongly on theta-one but only weakly on theta-two, a method that protects both equally may waste flexibility, while a method that selectively resists theta-one movement can preserve more of task A.

This is the intuition behind importance-weighted regularisation. A model estimates how consequential each old parameter was to previous behaviour, then adds a penalty for moving away from its earlier value. The penalty does not know every possible future input or guarantee old decisions remain correct; it biases optimisation toward solutions that satisfy the new objective without needlessly disturbing highly valued old parameters.

Representation reuse complicates the story. A large network can sometimes fit new tasks in directions that minimally affect old ones, while a small model might have insufficient capacity to satisfy both. Architectural growth, task-specific adapters and routing offer alternative compromises, but these approaches introduce memory, inference and task-identification costs.

The useful engineering question is not ‘Can we freeze everything?’ Freezing all weights prevents catastrophic forgetting from updates but also prevents learning the new task. The question is how to distribute limited plasticity, new evidence, stored old evidence and model capacity under a clear retention target.

Worked case: a capability seems strong until old tests are rerun

Consider a fictional document classifier. After completing training on task A, it correctly classifies 460 of 500 held-out A examples. Its A accuracy is 460/500 = 92%. The team then fine-tunes it only on task B. On a separate B test set it correctly classifies 440 of 500 examples, or 88%. These are distinct denominators: 500 old-task cases and 500 new-task cases.

After learning task B, the team reruns the same fixed A test. The updated model now gets 315 of 500 correct, or 63%. The decline from the previously recorded 92% to 63% is 29 percentage points. A relative decline from the original accuracy is (92−63)/92 ≈ 31.52%, which is a different denominator and should not be described as a 29% relative loss. Name the metric before drawing a conclusion.

Suppose a dashboard shows only the final B accuracy of 88%. It might proclaim success despite the severe loss of A capability. Even an equally weighted average across final A and B accuracies gives (63%+88%)/2=75.5%. That average hides the temporal decline because it does not show the 92% earlier benchmark. Track the full before-and-after task matrix and retention deltas.

Now imagine a replay-based training variant achieves 425/500 = 85% on A and 430/500 = 86% on B. It trades two percentage points of B accuracy against the first model’s 88%, but retains twenty-two additional percentage points on A compared with 63%. Whether that trade is preferable depends on the two tasks’ importance and operating risk. There is no universally correct average that decides for every organisation.

These invented outcomes illustrate the need for explicit retention criteria. The real training methods in the source papers were evaluated on specified research benchmarks; this example is not a reproduction and must not be cited as observed evidence for a particular replay method.

Build an evaluation matrix that makes forgetting measurable

For multiple tasks, construct a matrix whose row records the training stage and whose column records the task evaluated. After learning A, report A accuracy. After learning B, report both A and B. After learning C, report A, B and C. Do not leave blank cells interpreted as success. A missing old-task evaluation is missing evidence.

A useful forgetting measure for an earlier task compares its best earlier performance with its current performance. For task A, if the best previous test accuracy was 92% and the final was 63%, the observed decline is 29 percentage points. A related transfer metric compares final performance with performance at the time an earlier task was learned. Define your convention because research papers use several related metrics.

Sample independence matters. Reusing the exact evaluation set at every iteration for monitoring can be necessary to compare stages, but repeated decisions based on it can eventually overfit the evaluation process. Maintain a separate final test or audit set when feasible, and document which data were used to select replay ratios and regularisation strengths.

Class imbalance also changes interpretation. If the old task had a common majority class, 92% overall accuracy may conceal important minority failures. Report per-class precision, recall, confusion patterns or relevant decision costs. Forgetting may be highly concentrated in rare categories even when the average moves little.

The matrix should include changes to the task ontology. If A’s label definition was legitimately retired, a deliberate loss on that obsolete test is not necessarily an operational failure. Annotate which capabilities must persist, which have been superseded and why, without silently dropping a hard old test simply because it damages the headline number.

Replay: keep a controlled sample of the past in the learning stream

Experience replay stores some earlier examples and mixes them with current-task training. The old examples provide gradients that remind the model which behaviours must remain stable. In a simple classifier, replay can counteract new-data updates that would otherwise move old decision boundaries. The method is conceptually straightforward but has important sampling, storage and privacy choices.

A naive buffer might keep the first examples observed. That can be unrepresentative if data arrive in chronological clusters. Reservoir sampling, stratification or class-aware selection may produce better coverage under particular assumptions. Yet any chosen buffer can overfit, miss rare edge cases or preserve outdated patterns. The goal is to retain the task-relevant distribution, not merely to store the largest number of records.

Memory size is a measurable cost. If the learner receives 1,200 training instances per epoch and a policy reserves 8% of updates for replayed older examples, that is 0.08 × 1,200 = 96 replayed example uses and 1,104 new-example uses under this illustrative scheduling rule. It does not mean the buffer holds exactly 96 unique records; the same stored record may be sampled repeatedly. Distinguish replay frequency, unique buffer capacity and old-task coverage.

A replay buffer can introduce data-governance obligations. Old emails, personal health records or licensed documents cannot automatically be retained forever merely because they would improve a model. Consent, retention periods, deletion requests and contractual restrictions can rule out storage. Synthetic or generated replay may reduce some storage constraints but can introduce errors and privacy leakage of its own.

To evaluate replay, compare a no-replay baseline with controlled replay ratios and representative old and new tests. Too much replay can slow adaptation; too little may allow forgetting. The optimal trade-off depends on memory, compute, classes, learning stage and the changing value of each task.

Elastic weight consolidation protects parameters unequally

Elastic weight consolidation, abbreviated EWC, adds a regularisation term that resists moving parameters thought to matter for earlier tasks. In the classical formulation, the penalty is one-half times lambda times a sum across parameters of importance weights F_i multiplied by the squared difference between current parameter theta_i and its stored earlier value theta_i-star. In shorthand: L_total = L_new + (lambda/2) Σ F_i (theta_i−theta_i-star)^2.

The original Kirkpatrick and colleagues paper connects this idea to approximate Bayesian reasoning and uses information related to the Fisher information matrix to estimate weight importance. It is a practical approximation, not an exact guarantee that all old behaviours are protected. The diagonal approximation neglects many parameter interactions and can be weak when old tasks are complex or strongly conflicting.

Use a fictional two-parameter example. Let lambda=10, importance weights F=(4,1), and proposed displacements from the old optimum be (0.1,0.3). The weighted squared sum is 4×0.1² +1×0.3² =4×0.01 +0.09 =0.13. The penalty contribution is (10/2)×0.13 =5×0.13 =0.65 loss units. This result assumes the stated conventional factor one-half.

If both importance weights were one, the analogous penalty would be 5×(0.01+0.09)=0.50. EWC makes moving the first parameter more expensive because its estimated importance is higher. This does not mean that the model will move the second parameter by exactly three times as much; all gradients and model interactions determine the final update. The numerical case isolates the added objective term.

A larger lambda raises retention pressure but can reduce new-task learning. The right value must be selected on suitable validation tasks. If importance estimates are noisy, the method can protect the wrong parameters or fail to protect truly important interactions. Document how F_i is estimated and whether the old optimum is stored accurately.

Synaptic intelligence and other importance estimates

Synaptic intelligence is another regularisation approach that estimates parameter importance from accumulated training dynamics rather than relying on exactly the same importance calculation as EWC. The original Continual Learning Through Synaptic Intelligence paper by Zenke, Poole and Ganguli, published at ICML 2017, studied ways of protecting task-relevant parameter changes while maintaining computational practicality.

The shared engineering idea is to preserve important old behaviour with a soft constraint. The differences include how importance is accumulated, what information must be stored and what assumptions make the estimate meaningful. Two algorithms with similar quadratic penalties need not behave identically when their importance terms represent different histories.

These methods are especially attractive when keeping old training data is costly or prohibited, but they do not create an old-task test set out of thin air. An organisation still needs permitted evidence to check retention. If evaluation examples are also unavailable, it must be honest about how much of the old skill can actually be verified after updating.

Importance-weighted methods can struggle with long task sequences. Importance values from early tasks may accumulate, eventually limiting plasticity. Simplistic accumulation can produce either excessive rigidity or insufficient protection. Methods that combine or decay penalties need to justify what information they are allowed to forget and how that choice affects earlier capabilities.

Do not compare method names as brands. The right experiment measures retention, new-task learning, memory use, training cost, privacy constraints and robustness to changed contexts on the actual task family. An elegant penalty equation is useful only if the resulting model meets operational requirements.

Knowledge distillation can preserve old model behaviour

Another approach preserves an earlier model’s predictions as targets while learning the new task. Instead of storing every old raw record, the learner may use old-model outputs on available new examples, sometimes with softened probability distributions. This can encourage continuity in old functions while adding new capabilities. The original Learning without Forgetting work by Li and Hoiem explored this approach for visual recognition when old task data were unavailable.

The crucial question is whether the inputs available during new-task training are informative about old-task behaviour. If they never resemble old examples, matching the old model’s outputs on them may be insufficient to protect old performance. A learned teacher can also carry its own errors. Distillation preserves a behavioural approximation, not an independent guarantee of correctness.

Temperature and loss weighting matter. A softened teacher distribution may convey relationships among categories, but a very flat distribution can hide distinctions. This interacts with label smoothing, whose original targets are intentionally softened. The companion label smoothing and knowledge distillation guide explains why an accurate teacher is not necessarily the best teacher for compression.

Teacher-model retention still requires versioning. Record the teacher checkpoint, preprocessing, output mapping and training method. If the old teacher is silently updated during a student run, the reference behaviour moves. An independent immutable old-model checkpoint provides a clearer baseline against which to test whether new learning changed behaviour.

Distillation can be useful with privacy constraints, but it is not guaranteed privacy preserving. Model outputs may reflect or leak information about training data, depending on access and attack models. Governance should evaluate both raw-data retention and generated-target risks rather than assuming one is automatically compliant.

Gradient Episodic Memory uses old examples as a constraint

Gradient Episodic Memory, or GEM, is a continual-learning method proposed by David Lopez-Paz and Marc’Aurelio Ranzato. The original NeurIPS 2017 paper frames learning a sequence of tasks and uses episodic memory to avoid harmful changes to earlier tasks while allowing beneficial transfer.

The geometric intuition is that an unconstrained gradient for task B may point in a direction that increases loss on retained A examples. GEM seeks an update direction satisfying selected constraints derived from the memory. This changes the optimisation step rather than only blending training datasets or penalising parameter displacement from an earlier optimum.

Consider two gradient vectors as a toy illustration: g_new=(1,−1) and g_old=(1,1). Their dot product is 1×1 + (−1)×1 =0. Under the local first-order approximation, that particular new gradient is orthogonal to the old gradient. A different new gradient g_new=(−1,−1) gives dot product −2, signalling a potentially opposing direction under the chosen sign convention. The actual method’s constrained optimisation must follow its precise derivation; this toy vector arithmetic is only an intuition.

Constraints based on a small memory are approximations. The method can protect performance on the stored examples while failing on unrepresented old cases. The gradient geometry may also become more complex with many tasks and high-dimensional models. Evaluation must include independent old-task tests beyond the memory itself.

Use the original method as a research reference, not as an implicit claim that a production model will improve. What matters is whether the retained memory, constraint computations and task sequence match the application’s requirements and whether the final test matrix confirms preservation.

Adapters, expansion and parameter isolation

Instead of requiring one fixed set of weights to accommodate every task, a system can reserve some parameters for each task or add a small new module. If earlier weights remain frozen, those stored functions may be more stable. Adapters, separate heads and progressive architectures are examples of ways to separate plasticity while sharing parts of a representation.

The cost is capacity and routing. A new adapter consumes memory; a growing collection of task-specific components can complicate deployment. More importantly, a model may need to know which adapter or head to use when presented with an unknown input. If task identity is unavailable, selecting the correct component becomes a separate prediction problem with its own errors.

Freezing old weights also does not guarantee old system behaviour if shared preprocessing, tokenisation, retrieval or external tools change. A classifier with an immutable head can still fail if its input vocabulary remaps IDs. Lifecycle control must version the whole pipeline, not just a set of neural parameters.

Architectural isolation can be sensible when tasks are legally separated or have very different safety requirements. It can also reduce beneficial transfer from newer data to old tasks if the designs are completely isolated. Choose separation based on desired capability reuse and risk, not simply because it looks safer.

An engineering comparison should report storage growth per task, latency, inference routing accuracy, joint-task performance and ease of rollback. Without those measures, a method that appears to solve forgetting by adding unlimited task modules may merely move the complexity elsewhere.

Design a fair continual-learning experiment

Start with a clearly defined task sequence, A then B then C, and record exactly what data can be retained between stages. Hold task-specific test sets aside at the beginning and protect them from training choices. Establish the joint-training upper reference when legally and technically possible, but label it as a reference rather than a fair competitor if sequential methods are not allowed access to the same old examples.

Train an ordinary sequential fine-tuning baseline. After each stage evaluate every still-required earlier task, not only the newest one. Record accuracy, key subgroup errors, probabilities where relevant and costs. Then compare one intervention at a time: replay, EWC, distillation or architectural isolation. Keep model capacity, optimiser, training budget and evaluation standards comparable whenever feasible.

If a method uses extra memory or a frozen teacher, include that expense in the comparison. A replay model storing millions of examples is not resource-equivalent to a small regularisation-only model. A complex model that takes substantially longer to update may not meet real operational windows despite higher average accuracy.

Use multiple task orders. A model that learns A then B successfully might forget substantially when trained B then A because the tasks differ in difficulty or representation. A single favourable order can exaggerate generality. Randomise or report the sequence carefully and avoid attributing all differences to algorithm superiority.

Publish failure cases. A retention method might protect common labels while losing rare ones, or it might prevent useful adaptation to a genuine policy change. Distinguish unavoidable conflict, removable implementation error and poor intervention tuning. The correct result may be to keep separate models, decline an update or collect more authorised old-task evidence.

A practical fault tree for apparent forgetting

Failure one: performance is compared across changed preprocessing or label mappings. Re-run an unchanged old-task test through both old and updated pipeline versions. Failure two: only new-task metrics are reported. Restore the before-and-after matrix. Failure three: old test examples were accidentally used for tuning. Create a genuinely untouched audit protocol and disclose the limitation of prior results.

Failure four: the replay buffer is skewed toward recent or frequent categories. Inspect its counts, sampling probability and rare-case coverage. Failure five: old records were retained without permission. Stop the data use, apply the proper governance and choose a compliant method rather than hiding a privacy violation behind an accuracy benefit.

Failure six: EWC penalty strength overwhelms all adaptation. Inspect the regularisation term’s magnitude relative to the new-task loss and validate the importance estimate. Failure seven: the importance estimate protects the wrong parameters. Revisit its estimation and evaluate whether a different method is needed. Failure eight: an old teacher’s predictions are inconsistent because its checkpoint or tokenizer changed. Version and validate the teacher pipeline.

Failure nine: the organisation has intentionally changed the desired definition of the old task. Update the capability specification rather than treating an obsolete label as a sacred target. Failure ten: a final dashboard combines tasks so unevenly that a minority capability disappears numerically. Report task-level outcomes and weighted measures only with explicit denominators and decision costs.

Each repair begins by locating the first evidence boundary that failed. Forgetting is not a magical property that excuses missing tests. A model update can be analysed as a controlled change with before-and-after observations and a clear rollback condition.

Changed-context transfer: from image classes to new support policies

Transfer the reasoning to a fictional helpdesk model. In January it learns to identify refund requests and shipping complaints. In April the company introduces a new subscription cancellation category, while earlier categories remain in use. The model is trained on cancellation examples but old raw tickets cannot be retained beyond a contractual period. This removes the simplest full replay option.

Define the durable capability first. The updated system must still identify the old permitted categories on a legally usable test set and must also classify cancellations. If authorised old evaluation data are unavailable, leaders cannot truthfully certify retention from model confidence alone. They may need separate controlled testing, synthetic cases vetted against real policy or human review during a transition.

Suppose the earlier classifier had 180 correct old-task predictions out of 200, or 90%. After new-task fine-tuning it has 142/200 = 71%. The drop is 19 percentage points. An EWC variant obtains 169/200 = 84.5% old-task accuracy and 170/200 = 85% on a 200-case new-task set. This is a fictional result that shows how retention and adaptation should be reported separately; it does not prove that EWC would achieve those numbers on a real helpdesk.

Now change a policy detail: refund eligibility actually changes after April. Some January labels are no longer correct for new cases. Preserving the old model’s exact answer on every new record would be wrong. The evaluation should distinguish genuinely still-valid old concepts, historically labelled examples and deliberately updated policy semantics.

The transfer lesson is to protect required skill, not simply every old output. Apply sequential-learning safeguards only after a clear decision about which behaviours must persist, which may evolve and which old information can lawfully be kept.

What continual-learning methods cannot promise

Catastrophic forgetting is a measurable performance phenomenon under a task and evaluation definition. It is not proof that a neural network has a human-like memory, subjective experience or an intention to discard information. Human learning analogies can be useful for explanation but should not substitute for experiments about model parameters and task behaviour.

Each mitigation has an information budget. Replay depends on stored or generated old examples; importance penalties depend on estimates and stored old parameters; distillation depends on a preserved teacher; architectural separation adds capacity and may require task routing. None makes the stability-plasticity conflict disappear under arbitrary tasks, privacy constraints and compute limits.

Old data can contain outdated instructions or harmful stereotypes. Retaining them solely to protect a score may be undesirable. A policy change might require removing an obsolete capability. Deliberate forgetting and unavoidable accidental forgetting should be distinguished, documented and evaluated differently.

Reported laboratory benchmarks are not operational guarantees. Task sequences, class balance, update budgets and model families affect comparisons. Modern foundation models may involve pretraining, adapters, retrieval layers and tool use beyond the settings studied in older continual-learning experiments. Transfer the mechanism with care rather than importing a benchmark percentage as a promise.

An agent that can adapt continuously still requires change control, authoritative evidence and oversight. Preventing old-task accuracy from collapsing does not establish that the model is safe, truthful or authorised to act independently. Those properties need their own tests and controls.

Frequently asked questions

What is catastrophic forgetting? It is a substantial loss of previously learned task performance after a neural network is trained on new data or tasks, typically because parameter updates interfere with earlier representations.

Is all accuracy decline catastrophic forgetting? No. Data drift, changing labels, evaluation bugs and a different deployment environment can reduce accuracy without destructive sequential parameter updates. Compare unchanged tests before and after training.

Does mixing old and new data solve it? Replay can reduce forgetting, but its success depends on representative old data, memory size, task conflict and lawful retention. It is not a universal guarantee.

What does EWC do? Elastic weight consolidation adds a parameter-importance-weighted penalty for moving away from earlier values, encouraging retention while permitting some learning.

Can distillation work without old raw data? Sometimes. It can preserve old teacher behaviour using suitable available inputs, but old-task coverage and teacher quality limit the result, and model outputs can carry privacy risks.

Why not freeze all model weights? That preserves the stored weights but prevents those weights from learning the new task. Other modules may be added, at the cost of capacity, complexity and routing decisions.

How do I measure forgetting? Evaluate earlier required tasks after each new training stage, compare with the earlier best or baseline under a clearly defined convention, and report denominators and task-level errors.

Is preventing all forgetting always desirable? No. Obsolete labels or harmful behaviours may need correction or removal. Protect capabilities that remain valid and required, not all historical outputs.

Reader pathways and original sources

For learners, compute an old-task score before and after fine-tuning, then compare two interventions using identical denominators. Reproduce the EWC two-parameter penalty and explain why the importance weights change which parameter movement is penalised. Ask which evidence demonstrates that an old skill truly survived.

For engineers, build a task-by-stage performance matrix, control preprocessing and label changes, measure replay memory and privacy obligations, then evaluate at least one simple baseline before adding more complicated algorithms. The early stopping guide addresses a related but different choice about when to end training on one objective; it does not alone solve retention across task sequences.

Primary research: Kirkpatrick and colleagues, Overcoming catastrophic forgetting in neural networks (PNAS 2017); Zenke, Poole and Ganguli, Continual Learning Through Synaptic Intelligence (ICML 2017); Li and Hoiem, Learning without Forgetting; and Lopez-Paz and Ranzato, Gradient Episodic Memory (NeurIPS 2017).

For the wider framework, visit How AI Works and the education-to-transfer standard. The numerical cases are openly fictional and no model was trained during their preparation.

Discover more from eduKate Singapore

Subscribe now to keep reading and get access to the full archive.

Continue reading