Cruxly

Cruxly › AI engineering › Fine-Tuning and Model Adaptation

Training Data: Dedup, Contamination, and Knowing It Actually Worked

near-duplicate detection, benchmark contamination, catastrophic forgetting, and the before/after eval that proves it

Open the interactive lesson

Prompt: a team fine-tunes a model on 50,000 scraped support tickets and reports a 12-point improvement on their internal support-quality eval. Before this ships, you're asked to sign off. List everything you'd check before believing the 12 points, and design the evaluation process that should have existed from the start.

40 minutes first. The 12-point number is real in the sense that it was computed correctly. Whether it means what the team thinks it means is a completely separate question, and three specific things can make it lie.

Near-duplicate detection: why exact-match dedup misses most of it

Exact-match dedup catches almost nothing, because near-duplicates are the actual volume problem. Two support tickets with the same complaint, one paraphrased, differing only in customer name and ticket number, are not exact duplicates and will sail through a hash-based dedup step untouched — yet they contribute almost the same training signal twice. At the scale of a scraped corpus, a large fraction of apparent volume is often this kind of near-duplicate: templated tickets, boilerplate responses, auto-generated confirmations with one field changed.

Why this matters more than it sounds like it should. A training set with heavy near-duplication doesn't just waste compute — it biases the model toward whatever pattern happens to be over-represented, and it makes an eval number that looks like generalization look better than it is, because if a near-duplicate of an eval-adjacent pattern exists many times in training, the model is closer to recalling it than reasoning about it.

The standard fix: MinHash (or a similar locality-sensitive hashing scheme) over n-gram shingles, not exact hashing. Break each document into overlapping n-grams (shingles), compute a MinHash signature that approximates Jaccard similarity cheaply, and cluster documents whose signatures are close. Set a similarity threshold (commonly in the 0.7-0.8 Jaccard range for near-duplicate clusters) and keep one representative per cluster rather than every member — or, less aggressively, downweight cluster members during training so the same underlying pattern doesn't dominate the gradient.

Know what each layer catches, because MinHash does not catch paraphrases.

Run MinHash first (cheap, removes the bulk of templated copies), then semantic dedup on what remains.

State the checkable claim. If someone describes their dedup process as "we removed exact duplicates," that is not a dedup process for a scraped corpus — it is the step before the dedup process. The question worth asking out loud in review: "what fraction of the corpus is near-duplicate under MinHash, and what threshold did you use?" A team that has never run this analysis has not actually checked.

Benchmark contamination: the check that invalidates every number if skipped

Contamination means the training data overlaps with the eval set, and it silently inflates every metric computed against that eval. If even a modest fraction of an eval set's examples — or close paraphrases of them — leaked into the training corpus, the model is being scored partly on recall of what it was shown rather than on the capability the eval claims to measure. This is not a hypothetical: scraped corpora (especially anything pulled from the same general pool a benchmark was built from) contaminate benchmarks routinely, and it is one of the most common reasons a reported gain doesn't reproduce in production.

How to actually check, concretely. Run the same near-duplicate detection used for corpus dedup between the training set and the eval set, not just within the training set — treat the eval set as one more document pool and look for high-similarity matches against training documents. Use both layers here too: n-gram overlap catches copied eval items, and embedding similarity catches eval items that leaked in paraphrased form. For an internal eval like the support-quality set in the prompt, this means directly checking whether any of the 50,000 scraped tickets are near-duplicates of tickets in the eval set, which is entirely possible if both were pulled from the same ticketing system's history without deliberately partitioning by time or ticket ID before either set was assembled.

The most common source of contamination is not malice, it's carelessness in how the split was made. If the eval set was carved out after the training set was already assembled, from the same underlying pool, without a clean time-based partition, contamination is the default outcome, not an edge case. The fix is procedural: freeze the eval set first, from a time window or ID range that the training scrape explicitly excludes, and re-run the contamination check whenever either set changes. Skipping this is the single most common reason a reported gain looks excellent internally and does not reproduce once the model meets real traffic — the eval was never measuring what it claimed to.

Catastrophic forgetting: the improvement that hides a regression

A fine-tune can get measurably better at the target task and measurably worse at everything else, and a target-task-only eval cannot see the second half. This is catastrophic forgetting: training on a narrow distribution (50,000 support tickets, in this scenario) shifts the model's weights toward that distribution's patterns, and capabilities the base model had — general reasoning, instruction-following on tasks unlike support tickets, even basic things like following a formatting instruction the fine-tuning data never exercised — can measurably degrade, without any single training example being individually responsible.

Why the 12-point number in the prompt cannot rule this out. The team reported one number, on one eval, for the target task. That number can be entirely real and the model can simultaneously be worse at everything the support-quality eval doesn't test. If the fine-tuned model replaces the base model for a broader set of duties — even just "the model this team owns," which downstream systems may call for other things — forgetting becomes a production regression nobody measured until a user hits it.

The fix is a held-out general-capability eval, run before and after, every time. Not a second glance at the target task — a genuinely different eval suite exercising capabilities the fine-tuning data was never meant to touch: general instruction-following, basic reasoning, formatting compliance, anything the base model was known to do well. A team that only re-runs the target-task eval after fine-tuning has, by construction, no way to detect forgetting even if it happened — the eval they ran is structurally blind to it.

What a forgetting result actually looks like, so it's recognizable. Target-task eval: +12 points. General-capability suite: -6 points on instruction-following, flat on basic reasoning, -3 points on formatting compliance. That is not a wash — whether it's an acceptable trade depends entirely on how the model gets used downstream, which is a product decision that can only be made if the regression was measured in the first place. Shipping without measuring it isn't accepting the trade, it's making the trade blind.

When forgetting shows up, there are standard mitigations — name them rather than just reporting the regression:

How you actually know the fine-tune helped

Vibes are not evidence, and neither is a single number on a single eval the team already believed in going in. "It reads better," "the demo answers felt right," or "the eval score went up" are each individually weak evidence, and a rigorous sign-off needs all of the following before the 12 points is believable:

  1. A documented near-duplicate analysis of the training corpus, with a stated threshold and the fraction of the corpus that clustered — not "we deduped it," a number.
  2. A documented contamination check between training and eval sets, using the same near-duplicate technique, run in both directions.
  3. A before/after comparison on the target task, which is the number the team already has — necessary but nowhere near sufficient.
  4. A before/after comparison on a held-out general-capability suite the fine-tuning data was never meant to touch, to catch forgetting that the target-task eval is structurally blind to.
  5. The same eval infrastructure run on both the base and fine-tuned model, so the comparison isn't contaminated by an eval harness change that happened to coincide with the fine-tune.
  6. A confidence interval on the gain. "+12 points" on a 200-item eval set can be mostly noise; report the eval size and a paired confidence interval (bootstrap over items) before calling it real.
  7. A judge check, if the eval is LLM-judged. A judge can prefer the fine-tuned model's style — longer, more confident, closer to the judge's own phrasing — independent of correctness. Spot-check with human ratings, or use a judge from a different model family.
  8. A PII review of the training data. 50,000 scraped support tickets contain names, phone numbers and addresses; once trained into the weights they can't be erased (see the DPDP section of the governance module), so redact before training.

State the sign-off decision precisely, because "looks good" is not a decision. "I can't sign off on the 12 points alone. I want the near-duplicate fraction in the training corpus, a contamination check between the 50,000 tickets and the eval set, and a before/after run on our general-capability suite. If the corpus is clean, contamination is negligible, and the general suite doesn't regress, the 12 points is a real result and I'd approve it. If the general suite drops meaningfully, that's not necessarily a blocker — but it's a trade someone with product context needs to accept explicitly, not one that ships because nobody measured it."

Close with the discipline this whole lesson is actually about. Every step here — dedup, contamination, forgetting — is a way of asking the same question from a different angle: is this number measuring what I think it's measuring? A fine-tune that skips all three can still report a genuine, reproducible 12-point gain and still be a bad idea to ship, because "better at the thing we measured" and "better, full stop" are not the same claim, and only the second one is what "the fine-tune helped" is supposed to mean.

How to say it in the interview

"A 12-point gain on one internal eval, measured only on the target task, is not enough to sign off on. I would check three things before believing it: whether the training set has near-duplicates that let the model memorize a handful of patterns rather than generalize, whether any training examples overlap with the eval set itself, and whether general-capability performance regressed even as the target task improved. Without a held-out general eval and a documented dedup and contamination check, a 12-point improvement on the target metric is as likely to reflect leakage or narrow memorization as real capability gain."

Checklist

Practice this interactively
Share this lessonWhatsAppLinkedInX

Related lessons