Synthetic Data for Model Training: Value and Collapse Risk
Generated data can fill a gap or degrade a distribution. Read the two papers behind both claims and a worked decision.
Synthetic data is praised as a solution to data scarcity and blamed for poisoning future training sets. Both claims come from real research, and the difference is how the data is used. Deliberate generation with verification is a tool; uncontrolled recursive replacement is a failure mode.
Direct answer: Synthetic data can improve a model when it is generated for a defined purpose with verified labels — the Textbooks Are All You Need result shows strong performance from small models trained on high-quality generated textbooks. The risk appears in recursive training on generated output, where The Curse of Recursion reports degradation of the distribution's tails. The boundary: the two papers study different setups, and neither supports a blanket claim that synthetic data is good or bad.

Two findings, two setups
Targeted generation. The Textbooks Are All You Need paper trained small models on high-quality, textbook-style synthetic data and reported strong results on code-generation benchmarks. The setup was deliberate: a specific format, a specific purpose, and a controlled generation process.
Recursive replacement. The Curse of Recursion studied what happens when models are trained on data generated by previous models over successive generations. The reported effect is degradation, particularly in the tails of the distribution — the rare cases that make data diverse.
Three differences explain the apparent contradiction:
- Purpose. One generates data to teach a specific skill; the other replaces a corpus with its own output.
- Verification. A labelled, checked dataset is different from unverified generated text.
- Generation count. A single deliberate augmentation is not the same as recursive replacement across generations.
A worked decision example
The following is hypothetical and illustrative.
A team has 10,000 labelled examples of a rare error class and needs 100,000 to train a classifier.
Option A — generate more examples. Use a model to produce 90,000 additional examples from the seed set. Three conditions decide whether this helps: a human checks a sample for realism, the generated examples are labelled by a rule that can be verified, and a held-out test set of real examples remains untouched. If all three hold, the augmented set may improve recall on the rare class.
Option B — leave the imbalance. Train on 10,000 real examples and accept lower recall. This is a baseline that must be measured, because generation is not free.
Three checks decide the outcome:
- Does the generated data match reality? If it is too clean, the model learns a simplified version of the error class and fails on real cases.
- Are labels verified? A generated example with a wrong label is worse than no example, as the data-quality article explains.
- Does the held-out real set improve? That is the only metric that matters. An improvement on generated validation data proves nothing, which the leakage guidance makes clear.
Three conclusions follow:
- Generate to fill a defined gap, not to inflate a dataset.
- Keep real data as the anchor and measure against it.
- Record provenance so future training runs know which examples were generated, as What Is Model Collapse and Why It Matters argues.
When generation helps
Rare classes where a few real examples can seed verified synthetic ones.
Format diversity where the underlying fact is known but many phrasings are needed.
Controlled edge cases where a specific failure mode must be represented deliberately.
When it harms
Uncontrolled replacement of a real corpus with model output.
Unverified labels that teach the wrong association.
Loss of tails — rare but important patterns disappear from successive generations, reducing diversity.
The counterargument — that all training data is now partly synthetic — is fair and makes provenance more important, not less.
Knowing which portion is generated and how it was checked is the only way to reason about the risk.
The role of human review
A verification step is what separates targeted generation from recursive degradation.
Three forms are practical:
Rule-based checks. If a generated example has a verifiable property — a label that can be computed, a format that can be parsed, a value within a known range — check it automatically. This is cheap and scales to large sets.
Human sampling. Review a random sample of generated examples, not the ones that look best.
Performance on real held-out data. The ultimate check is whether the model improves on examples that were never generated. If it does not, the synthetic set has taught a pattern that does not transfer, regardless of how realistic it looks.
These checks cost time, and that cost belongs in the decision. Generation is cheap; verification is the real expense, which is why the method is worth using only when a specific gap needs filling.
A decision test
- State the purpose of the generated data.
- Keep an anchor of real data and measure against a real held-out set.
- Verify labels with a rule or a human sample.
- Record provenance for every generated example.
- Check distributional diversity, not just aggregate quality.
- Avoid recursive replacement without a real-data anchor.
- Re-evaluate after each generation of data or model.
For the failure mode, see Model Collapse; for data practices, see Data Curation.
References & Further Reading
- Suriya Gunasekar et al., 2023. Textbooks Are All You Need — The result that training on high-quality synthetic textbook-style data can produce strong models at small scale.
- Ilia Shumailov et al., 2023. The Curse of Recursion: Training on Generated Data Makes Models Forget — The model-collapse result: training recursively on model-generated data degrades the tails of the distribution.