Catastrophic forgetting when fine-tuning LLMs

By Ivo Donker — compiled with AI support (Claude & Gemini) · Last updated: 6 August 2026

When you adapt a language model for a specific task through fine-tuning, an invisible problem often arises: the model performs excellently on the new training data, but gradually loses fundamental skills it acquired during initial training. This phenomenon is known in the machine learning literature as catastrophic forgetting (catastrophic forgetting).

The treacherous thing about catastrophic forgetting is that you don't see it directly in the regular training statistics. The loss function (loss curve) on your specific dataset declines steadily, and the evaluation statistics for the target task show excellent results. Only once the model is deployed in a production environment and confronted with more general questions does it become clear that crucial capabilities have been lost. To better understand how this process works at the neural level, it is useful to look at the basic principles of how an AI learns.

What the phenomenon precisely entails

When fine-tuning a large language model, the network's weights are adjusted based on a new dataset. The model tries to capture the patterns in this new data as accurately as possible. During this process, the gradient updates overwrite the existing parameters that were responsible for previously learned knowledge and skills.

The phenomenon is not unique to language models, but it manifests itself very prominently in neural networks in general. Because a neural network does not have separate memory modules like a traditional computer, knowledge is distributed across the entire architecture. When the weights change to optimize task A, the configuration needed to correctly perform task B fades away.

In practice, this means that a model that has been retrained to summarize legal documents can suddenly struggle with simple arithmetic steps, no longer understand the syntax of a programming language, or no longer be able to maintain a structured format such as JSON.

Why it happens at the parameter level

To understand why catastrophic forgetting occurs, you need to look at the internal representation of a transformer model. A large language model stores its knowledge in billions of weights. Together, these weights form a highly high-dimensional landscape in which relationships between concepts, grammar, facts, and reasoning steps are captured.

Unlike a database where each piece of data has its own address, almost all skills in a neural network share the same parameters. Knowledge is literally stored in the mutual relationships between the weights. When you train the model on a new dataset, the weights are adjusted through backpropagation. The direction of this adjustment is determined solely by the error margin on the new examples.

Because the model has no access to the original pre-training data during fine-tuning, the algorithm has no 'awareness' of the importance of specific weight ratios for old tasks. The weights that were essential for the old skills are shifted without restriction toward values that are optimal for the new training examples. The larger the step size (learning rate) and the longer the training takes, the further the weights drift from their original, balanced state.

Why language models are extra vulnerable

Large language models are designed as generic systems. Their strength lies in a broad range of abstract, overarching skills. Think of:

• Understanding and executing instructions (instruction following).
• Multilingualism and translating between different languages.
• Reasoning ability and taking logical steps.
• Safety filters and refusing harmful requests.
• Formatting requirements, such as generating valid HTML, JSON, or Markdown.

These generic skills are not localized in one specific layer or a clearly defined group of neurons; they are broadly distributed across the entire network. When you retrain a model on a specific task — for example, converting customer questions into internal database queries — the model does everything it can to minimize the error on that specific task.

Because the generic skills are spread so widely across the network, virtually every weight change affects these subtle structures. A model that is exposed exclusively to concise, specific answers unlearns the sensitivity needed to understand nuances in language or to maintain safe, ethical boundaries.

The signals of catastrophic forgetting

Recognizing catastrophic forgetting requires active monitoring, because it is not directly apparent from the loss function of your training set. There are several clear signals indicating that a fine-tuned model is losing its broad functionality:

A common scenario arises when retraining on specific code examples. After training, the model may write excellent code in the target language, but turns out to be no longer able to explain in normal Dutch what the code actually does. The answer degenerates into a jumble of syntax and fragmentary statements.

Measures to prevent forgetting

There are various strategies to counter catastrophic forgetting. These range from simple adjustments to hyperparameter settings to more advanced architectural choices and data management. We cover the measures in order of complexity and required effort.

1. Conservative hyperparameters (Lower learning rate and fewer epochs)

The most direct way to limit the overwriting of weights is to adjust the training parameters. When fine-tuning a pre-trained model, the learning rate (learning rate) must be considerably lower than when training from scratch. Typical values for fine-tuning lie between 1e-5 and 5e-6, depending on the size of the model.

In addition, the number of passes through the data (epochs) is crucial. Whereas traditional neural networks sometimes use dozens of epochs, for large language models one to three epochs is often more than sufficient. The more often the model sees the same examples, the stronger the pull to optimize the weights exclusively for that dataset.

2. Freezing layers (Layer freezing)

A step further is explicitly blocking weight changes in part of the network. In this technique, the lower layers of the model are frozen. These lower layers generally contain the most fundamental representations of language, such as syntax, general grammar rules, and basic token relationships.

By making only the top layers of the network adjustable, the model retains its fundamental language understanding, while the higher layers are tuned to the specific task. This reduces the risk of catastrophic forgetting, although it also limits the extent to which the model can adapt to a completely new domain.

3. Using adapters (LoRA / PEFT)

Instead of adjusting the full model (all weights), modern fine-tuning workflows often use Parameter-Efficient Fine-Tuning (PEFT), of which Low-Rank Adaptation (LoRA) is the best-known variant. In this approach, all the original weights of the base model remain completely frozen and unchanged.

Small, additional weight matrices (adapters) are added to specific layers of the network. Only the values in these adapters are trained. Because the original model remains intact, you might assume that catastrophic forgetting is thereby completely solved. This is, however, a misconception, as explained in the next section. If you are considering setting up such a model, see the guide to local fine-tuning with LoRA for a practical manual.

4. Data replay and blending in generic data

The most effective method for keeping the model broadly applicable across the full parameter set is to blend in general training examples. This process is known as data replay or experience replay.

Here, you add a percentage of a general, high-quality instruction dataset to your specific fine-tuning dataset. This general dataset contains diverse material: open-ended questions, multilingual texts, reasoning tasks, and ethical cases. By also exposing the model to these general examples during every training step, the gradient updates are forced to take the broad skills into account.

Why adapters reduce the risk but don't eliminate it

The use of adapters like LoRA is often seen as a guaranteed protection against catastrophic forgetting. After all, because the base weights don't change, the base model cannot become 'damaged.' Yet a model with an activated adapter can, in practice, exhibit exactly the same forgetting behavior.

The reason for this lies in the way the adapter's output is combined with the output of the base model. The adapter acts as a modification layer that steers the network's activations. If the adapter is trained in such a way that it emits dominant, pronounced steering signals, it can effectively suppress or override the useful signals from the base model.

Although the original knowledge is still physically present in the frozen weights, the computation chain can no longer reach it due to the adapter's interference. Once the adapter is active, the combined system exhibits exactly the same symptoms of catastrophic forgetting as a fully fine-tuned model. The main benefit of an adapter is that the process is reversible: disabling the adapter immediately restores the original model behavior. For more background on how these techniques relate to each other, you can read the article on adapters and LoRA.

Evaluating and measuring: you only see it if you look

One of the biggest dangers of catastrophic forgetting is that it goes unnoticed during standard validation. If you measure your model's performance solely on a random sample of your own fine-tuning dataset (the validation loss), you only see positive results. After all, the model keeps getting better at that specific task.

To detect catastrophic forgetting, you need to use an evaluation method that is independent of the target task. This requires a fixed test set (evaluation benchmark) with diverse, general tasks that are run on the model before and after fine-tuning.

A well-thought-out evaluation strategy includes the following steps:

1. Perform a baseline measurement: Test the unmodified base model on a broad set of benchmark tasks (such as general language proficiency, logical reasoning, and instruction following in Dutch).
2. Task-specific testing: Test performance on the intended task before training.
3. Fine-tune: Carry out the desired weight adjustments.
4. Comparative measurement: Afterward, run exactly the same broad benchmark as in step 1, alongside the test on the specific task.

Only when you compare the results of the baseline measurement with the situation after fine-tuning do you see whether there is regression in the general domains. Setting up such systematic checks closely aligns with performing regression testing for language models.

The dangers of sequential retraining

The problem of catastrophic forgetting worsens exponentially when a model is retrained in multiple successive phases. Suppose an organization first retrains a model on task A, saves the new weights, then retrains this model on task B, and later again on task C.

With each training round, the weights drift further from the original, stable point reached during large-scale pre-training. After a few successive steps, the internal structure is distorted to such an extent that the model has lost not only the skills from tasks A and B, but also its general robustness and language proficiency.

It is therefore a 'best practice' in the industry to always return to the clean, original base model for new requirements or additional data. Instead of continuing to train an already adjusted model, you combine the new training data with the old training data and run a single, combined fine-tuning run starting from the base.

When forgetting is acceptable or even desirable

Although catastrophic forgetting is generally regarded as a risk, there are specific applications in which the loss of general skills is not a problem at all — or is even a deliberate choice.

When a language model is deployed for one specific, tightly scoped task in an automated pipeline (for example, converting unstructured medical logs into a structured XML format), the model has no need for general knowledge, creative writing, or multilingualism. In such cases, the loss of those secondary skills is not relevant. After all, the model never needs to have a general conversation with an end user.

In certain cases, limiting the general functionality can even offer advantages. A model that has lost its general conversational skills through fine-tuning is often less susceptible to *jailbreaking* or unwanted deviations from the prescribed task. The model has become purely a specialized processing unit. For a broad consideration of the costs, risks, and alternatives of this process, you can look at the overview of off-the-shelf models versus fine-tuning, or the comparison between fine-tuning, prompting, and RAG is worth consulting.

Conclusion on the balance in fine-tuning

Catastrophic forgetting is a fundamental property of neural networks that stems from the way knowledge is stored in weights. It requires a clear approach throughout the entire process of data selection, training, and evaluation. By setting conservative hyperparameters, applying efficient adapter structures, blending in sufficient generic data, and always running structured regression tests on the base model, you can add the desired specific skills without damaging the foundation of the language model.

Further reading

• LoRA and adapters explained
• Fine-tuning vs. prompting vs. RAG
• How does an AI learn?
• Guide to local fine-tuning with LoRA
• Regression testing for prompts and models
• Fine-tuning vs. using an off-the-shelf model