Data contamination occurs when examples from evaluation sets end up present, totally or partially, in a model's training data. This breaks the separation that should exist between what the model learns and what its performance is measured against. As a consequence, the model does not demonstrate real generalization capability, but has instead "memorized" the correct answers.
Its relevance is critical because it invalidates benchmark results: accuracy figures become inflated and no longer reflect the model's behavior when faced with new data. This makes it difficult to compare systems fairly and can lead to wrong decisions in research or production.
It is an especially common problem in large language models, trained on massive volumes of text scraped from the internet, where the following can slip in:
- Questions and solutions from published academic tests.
- Open evaluation sets such as MMLU or GSM8K.
- Repositories with exercises and their answers.
To mitigate this, decontamination techniques are applied, which detect and remove overlaps before training or evaluation.