Lesson 8.5 · 23 min

Continual Learning: Training Without Forgetting the Past

If we teach a trained model one new thing, why might it suddenly get worse at the things it already knew?

In short: Continual learning is training a model on a stream of new data or tasks over time without losing what it learned before. Neural networks tend to suffer catastrophic forgetting: new training overwrites shared weights that old skills depended on. Practical fixes include replaying old data, penalising changes to important weights, giving new tasks their own parameters, and keeping fast-changing facts outside the model.

What is continual learning?

Continual learning (also called lifelong or incremental learning) means a model keeps learning from new data or new tasks that arrive over time, while keeping what it learned earlier. The data arrives as a stream: task A, then task B, then task C, and we usually cannot retrain from scratch on everything each time.

Humans do this naturally. Learning to ride a scooter does not make us forget how to ride a bike. Standard neural networks, however, are trained on one fixed dataset and are not built for this. When we keep training them on new data only, they often lose old abilities.

Think of it like a shared whiteboard One whiteboard holds all of a team’s notes. A new project arrives and someone writes its notes over the old ones because that is the quickest space. The new project looks great; the old notes are half erased. A neural network stores all skills in the same shared weights, so new training can “write over” old skills in the same way.

Running example: our bike-rental support model was fine-tuned on rental questions last year. This year we add e-scooters, with new safety rules and pricing. We want the model to learn scooters without becoming worse at bikes.

Why we need continual learning in LLMs

An LLM is a snapshot. Its knowledge stops at its training cutoff (the date of its latest training data). After that, the world changes: new products, laws, software versions, medical guidelines, and words. Pretraining a large model from scratch costs a great deal of compute, so we want ways to add to a model instead of rebuilding it.

Stay current: add recent knowledge, such as a new version of a programming library.
Specialise: adapt to a new domain (legal, medical, our company) without breaking general skills.
Add skills step by step: first instruction following, then tool use, then a new language.
Learn from users: improve from feedback collected after deployment.

In LLMs this shows up at several stages: continual pretraining (more raw text in a new domain or time period), continual instruction tuning (new tasks in sequence), and continual alignment (updating preferences and safety behaviour over time).

The big problem: catastrophic forgetting

Catastrophic forgetting (first described as “catastrophic interference” by McCloskey and Cohen in 1989) is the sharp loss of earlier skills when a network is trained on new data. The word catastrophic is apt: performance on old tasks can drop a lot after only a little new training.

Why does it happen? Gradient descent changes weights to reduce the loss on the current data only. Nothing in that loss mentions the old task. Because the same weights serve every skill, a step that helps task B can quietly move weights that task A relied on.

This is the stability–plasticity dilemma: a model that is very plastic (changes easily) learns new tasks fast but forgets; a very stable model remembers but struggles to learn. Every continual learning method picks a point between the two.

Let us see forgetting happen. Our tiny model has fixed random features and one shared trainable weight vector. Task A lives in one region of input space, task B in another, with different target rules. A single weight vector can fit both reasonably, but only if training keeps A in mind.

forgetting_demo.py

import numpy as np
rng = np.random.default_rng(1)
# A tiny "model": fixed random ReLU features + one shared trainable weight vector
R = rng.normal(size=(2, 60))
feats = lambda X: np.maximum(X @ R, 0)
# Task A lives around (+1, +1), task B around (-1, -1); different target rules
XA = rng.normal(loc=+1, scale=0.7, size=(200, 2)); yA = np.sin(2 * XA[:, 0])
XB = rng.normal(loc=-1, scale=0.7, size=(200, 2)); yB = XB[:, 1] ** 2
FA, FB = feats(XA), feats(XB)
def mse(w, F, y): return float(np.mean((F @ w - y) ** 2))
def train(w, F, y, steps=3000, lr=0.003, anchor=None, lam=0.0):
w = w.copy()
for _ in range(steps):
g = 2 * F.T @ (F @ w - y) / len(y)
if anchor is not None:
g += 2 * lam * (w - anchor)    # pull back toward the old weights
w -= lr * g
return w
w0 = train(np.zeros(60), FA, yA)           # 1) learn task A
print(f"after task A:          A err {mse(w0, FA, yA):.3f}  B err {mse(w0, FB, yB):.3f}")
w_naive = train(w0, FB, yB)                # 2) then fine-tune on B only
print(f"naive fine-tune on B:  A err {mse(w_naive, FA, yA):.3f}  B err {mse(w_naive, FB, yB):.3f}")
idx = rng.choice(200, 40, replace=False)   # 3) mix 20% of old A data back in
w_replay = train(w0, np.vstack([FB, FA[idx]]), np.concatenate([yB, yA[idx]]))
print(f"B + 20% replay of A:   A err {mse(w_replay, FA, yA):.3f}  B err {mse(w_replay, FB, yB):.3f}")
w_reg = train(w0, FB, yB, anchor=w0, lam=0.5)  # 4) L2 anchor (EWC-like idea)
print(f"B + L2 anchor:         A err {mse(w_reg, FA, yA):.3f}  B err {mse(w_reg, FB, yB):.3f}")

Output:

after task A:          A err 0.273  B err 8.687
naive fine-tune on B:  A err 1.097  B err 0.360
B + 20% replay of A:   A err 0.316  B err 0.374
B + L2 anchor:         A err 0.408  B err 0.378

Pause and think: In the demo, why did naive fine-tuning on B make task A worse even though we never showed it any wrong A examples?

Because the loss during B training only measures B. Gradient steps move the shared weights wherever B wants, and nothing pushes back to protect A. Forgetting comes from interference in shared weights, not from bad data.

Approaches to continual learning in LLMs

Methods fall into a few families. Real systems often combine them.

Replay (rehearsal): keep a sample of old data and mix it into new training. For LLMs this often means mixing some general pretraining or instruction data into domain training. A variant, generative replay, uses a model to generate pseudo-examples of old tasks when the original data cannot be stored.
Regularisation: add a penalty that discourages changing weights that matter for old tasks. EWC (Elastic Weight Consolidation, Kirkpatrick et al., 2017) estimates each weight’s importance with the Fisher information and penalises important weights more. Learning without Forgetting uses distillation from the old model’s outputs as the penalty.
Parameter isolation (architecture): give new tasks their own parameters. With LLMs this is natural: train a separate LoRA adapter per task or domain while the base stays frozen, then pick or combine adapters.
Careful continual pretraining: when adding a lot of new text, re-warm and then re-decay the learning rate and include replay of older data; published studies have found this simple recipe recovers much of the quality of retraining from scratch.
Knowledge editing: methods such as ROME and MEMIT change specific facts by editing a small set of weights, useful for a few targeted corrections rather than broad learning.
Keep knowledge outside the model: retrieval (RAG), tools and memory stores let a frozen model use fresh information with no forgetting at all.

Challenges in continual learning

Measuring forgetting is hard. LLMs have countless skills; we can only test a sample. Forgetting can hide in skills nobody re-checked.
Scale and cost. Replay data and evaluation suites grow with every update.
Data retention limits. Privacy rules or licences may forbid keeping old data for replay.
Conflicting knowledge. New facts may contradict old ones (“the price is now 15, not 12”). The model can end up mixing both.
Safety drift. Even small fine-tunes on harmless-looking data have been reported to weaken safety behaviour, so alignment must be re-tested after each update.
No universal recipe. The best mix ratio, learning rate and method vary by model, domain and data size.

Common mistake Evaluating only on the new task. A model fine-tuned on scooter questions can score brilliantly on scooters while quietly getting worse at bike refunds, general chat, or safety refusals. Always keep and rerun a fixed regression suite for old abilities.

Pause and think: Our scooter prices change every month. Should we continually fine-tune the model monthly to keep prices right?

Usually not. Fast-changing facts belong in a database or retrieval index that the model reads at answer time. Monthly fine-tunes would be costly, risk forgetting, and could leave old prices mixed in the weights.

Real-world use cases

Adding scooters to our support bot, safely

Split knowledge from skills: Put scooter prices and station lists in the retrieval index. Keep training data for tone and the new safety-check conversation flow.
Freeze base, add an adapter: Train a scooter LoRA adapter, or continue the existing adapter with replay.
Mix replay data: Include a share of past bike conversations and general instructions in every batch.
Run both test suites: Compare bike accuracy before and after; the change should be near zero.
Ship and monitor: Watch real conversations for regressions and log new failure cases for the next round.

Where continual learning matters Code assistants that must learn new library versions; domain models (law, medicine, finance) updated as rules change; enterprise assistants adapting to a company’s new products; model providers releasing updated versions with later knowledge cutoffs; and personal assistants that adapt to a user over time. In practice most teams combine retrieval for facts with periodic, carefully evaluated fine-tunes for skills.

When not to bother: if a task can be solved with retrieval or a better prompt, or if we can afford to retrain on the full combined dataset each time, a dedicated continual learning method may not be needed. Continual learning techniques earn their place when data arrives over time and full retraining is too costly.

Worked example, step by step

We keep saying “check the old tasks”. Here is how to turn that into numbers. Suppose our support model learns three tasks in order: bikes, then scooters, then helmets. After each stage we test it on all three. That gives an accuracy matrix: each row is a moment in time, each column is a test set. The numbers below are illustrative.

Three numbers from one matrix

Final average accuracy: Average the last row: (0.65 + 0.80 + 0.91) / 3 ≈ 0.79. This is how good the model is now, across everything it was taught.
Forgetting per task: For each old task, take its best earlier score minus its final score. Bikes: 0.90 − 0.65 = 0.25. Scooters: 0.88 − 0.80 = 0.08. Average forgetting: (0.25 + 0.08) / 2 ≈ 0.17.
Forward transfer: Look above the diagonal. Before any scooter training the model scored 0.40 on scooters. If a model with no bike training scored, say, 0.30, then bikes helped scooters by 0.10. Earlier learning can help later tasks.
Find the worst step: Bikes fell 0.18 when we added scooters and only 0.07 when we added helmets. The scooter update did most of the damage, so that is the one to redo with replay or an adapter.

Two models can share the same final average and still be very different. One may be steady on every task. The other may be excellent on the newest task and poor on the oldest. That is why we report average accuracy and forgetting together.

There is also a trap in the other direction. A model with zero forgetting and a low score on the new task has not solved the problem either. It was simply too stable to learn. The matrix shows both failures: look at the first column for forgetting and at the diagonal for learning.

Practice: try it yourself

We will shrink the whole stability–plasticity trade-off to two weights. Task A depends heavily on the first weight and hardly at all on the second. Task B wants to move both. We compare three ways of learning B: no protection, the same spring on every weight, and a spring that is stiff only where task A needs it.

practice_weight_importance.py

import numpy as np
# Two weights. Task A needs w1 badly and barely cares about w2.
imp_A = np.array([10.0, 0.1])      # importance of each weight for task A
w_A = np.array([1.0, 1.0])         # weights after learning task A
w_B = np.array([3.0, 3.0])         # the weights task B would like
def loss_A(w):
return float(np.sum(imp_A * (w - w_A) ** 2))
def loss_B(w):
return float(np.sum((w - w_B) ** 2))
def train_on_B(penalty, lam, steps=2000, lr=0.01):
w = w_A.copy()                 # start from the task-A solution
for _ in range(steps):
grad_B = 2 * (w - w_B)                      # pull toward task B
grad_pen = 2 * lam * penalty * (w - w_A)    # spring back to old weights
w -= lr * (grad_B + grad_pen)
return w
methods = [("naive fine-tune", np.zeros(2), 0.0),
("uniform L2", np.ones(2), 1.0),
("importance-weighted", imp_A, 1.0)]
print("method                  w1     w2  loss A  loss B")
for name, penalty, lam in methods:
w = train_on_B(penalty, lam)
print(f"{name:<20} {w[0]:5.2f}  {w[1]:5.2f}  {loss_A(w):6.2f}  {loss_B(w):6.2f}")

Output:

method                  w1     w2  loss A  loss B
naive fine-tune       3.00   3.00   40.40    0.00
uniform L2            2.00   2.00   10.10    2.00
importance-weighted   1.18   2.82    0.66    3.34

The importance-weighted run keeps w1 near 1 (task A needs it) and lets w2 travel almost all the way to 3 (task A does not care). It gives up a little on B and saves almost all of A.

Now change it:

Raise lam for the importance-weighted method on line 24 from 1.0 to 10.0. Predict: which loss goes up and which goes down?
Make task A care about both weights: set imp_A on line 4 to [10.0, 10.0]. Predict: can any method now keep both losses low? What does that say about tasks that truly conflict?
Set w_B on line 6 to [1.0, 3.0], so task B only wants to move the weight A does not need. Predict the loss A of naive fine-tuning before you run it.

Pause and think: Uniform L2 ends with total loss 10.10 + 2.00 = 12.10. The importance-weighted run ends with 0.66 + 3.34 = 4.00. Both use the same strength λ = 1. Where does the difference come from?

Uniform L2 holds both weights back equally, so it stops w2 from moving even though task A barely uses it, and still lets w1 drift to 2.0 where task A is badly hurt. The importance-weighted penalty spends its “stiffness” only on w1. Same budget, placed where it matters. This is the idea behind EWC.

Pause and think: In a real model we cannot hand-write imp_A. If our importance estimates were wrong and swapped the two weights, what would we expect?

The penalty would protect w2 and free w1, the opposite of what task A needs. We would get strong forgetting of A and a worse fit on B than naive fine-tuning. Regularisation methods are only as good as their importance estimates, which is one reason replay is often used alongside them.

Key takeaways

Continual learning means learning from a stream of new data without losing old skills.
Catastrophic forgetting happens because new training changes shared weights with no regard for old tasks.
Replay, regularisation (EWC), and parameter isolation (per-task adapters) are the main protections.
Keep fast-changing facts in retrieval; use training for stable skills and style.
Always rerun old test suites after an update; forgetting is invisible otherwise.

Key terms

Continual learning: Training a model on data or tasks that arrive over time while keeping earlier knowledge.
Catastrophic forgetting: A sharp loss of old abilities after training on new data.
Stability–plasticity dilemma: The trade-off between remembering old knowledge and being able to learn new knowledge.
Replay: Mixing stored or generated old examples into new training.
EWC: Elastic Weight Consolidation: a penalty that protects weights important to earlier tasks.
Continual pretraining: Further pretraining an existing model on new raw text, such as a new domain or time period.

← 8.4 Knowledge Distillation: Compressing Large Models into Small Ones · 8.6 Deep RL from Human Preferences: The Foundational Paper →