The Emerging Risk of AI Inbreeding: Why Boards Should Care About the Integrity of AI Training Data
Executive Summary
Most organisations are currently focused on how to adopt artificial intelligence safely, responsibly and at scale. Considerable attention is rightly being given to governance, regulation, security, ethics and workforce adoption.
Far less attention is being paid to a more fundamental issue.
The quality of AI systems is ultimately determined by the quality of the data used to train them.
Recent reporting by New Scientist, supported by academic research published in Nature and analysis from the University of Oxford, suggests that this data ecosystem is beginning to deteriorate. Workers employed to generate human training data are increasingly using AI systems to complete those tasks themselves, introducing synthetic content into datasets that future models assume were created by humans. At the same time, the internet itself is becoming saturated with AI-generated content, increasing the likelihood that future foundation models are trained on the outputs of previous models rather than original human knowledge.
Researchers refer to the resulting phenomenon as model collapse, although it is increasingly described as AI inbreedingor AI cannibalism.
While this is unlikely to result in a sudden failure of AI systems, the evidence suggests it could gradually reduce model capability, diminish creativity, amplify existing biases and weaken the quality of future decision-making.
For boards, this represents more than a technical concern. It raises important questions about supplier assurance, AI governance, operational resilience and long-term strategic dependency on third-party AI platforms.
The Emerging Problem
The first generation of modern large language models was primarily trained using publicly available human-created content gathered from books, websites, academic publications and online discussions.
As these datasets have become exhausted, AI developers have increasingly relied upon human contractors to generate new examples of conversations, problem solving, reasoning and evaluation that can be used to improve future models.
However, an investigation published by New Scientist (27 June 2026) suggests this process is becoming compromised.
Multiple workers involved in AI training admitted using systems such as ChatGPT to generate the very training data intended to improve future AI models.
One worker stated:
“It’s very widespread… I don’t think they can stop it.”
Another admitted:
“It just became easier to run everything through LLMs.”
Rather than producing genuinely human reasoning, workers described using one AI system to generate scenarios before asking another AI system to create supporting material.
The consequence is straightforward.
Models intended to learn from authentic human judgement are instead learning from previous AI outputs.
This creates a feedback loop in which synthetic data continually reinforces itself.
The Academic Evidence
This concern is supported by significant academic research.
In 2024, researchers published “The Curse of Recursion: Training on Generated Data Makes Models Forget” in Nature.
The paper demonstrated that repeatedly training models on AI-generated data causes a phenomenon known as model collapse.
The researchers concluded that:
“Without enough fresh real data in each generation of a model, future generative models are doomed to lose quality.”
Rather than failing immediately, models progressively lose information, rare events disappear first., statistical diversity reduces, outputs become increasingly repetitive. Eventually, models become less representative of the real world.
The University of Oxford summarised the research by warning that as AI-generated material increasingly dominates the internet, future AI systems face an increasing risk of learning from previous AI outputs instead of original human knowledge. They concluded that continued access to authentic human-generated data is essential if future AI capability is to continue improving.
Why This Matters
Many organisations assume AI capability will continue improving year after year.
That assumption may no longer be guaranteed.
If the quality of training data deteriorates, future generations of AI may improve more slowly than expected, or in some areas, become less capable.
This is particularly important because many organisations are beginning to embed AI into critical business activities including:
Executive decision support
Software development
Financial analysis
Customer service
Regulatory compliance
Medical decision support
Cyber security
Product design
These systems increasingly influence human decisions.
If the underlying models gradually become less capable, organisations may not notice the decline until significant business processes have become dependent upon them.
Unlike traditional software failures, degradation caused by poorer training data is likely to be gradual rather than sudden.
This makes governance considerably more difficult.
The Wider Strategic Issue
The challenge extends beyond technical performance. It reflects a broader deterioration in the information ecosystem upon which AI depends. Historically, organisations assumed that digital information represented independent human knowledge. Increasingly, that assumption no longer holds.
Reports suggest that large volumes of internet content are already AI generated. Organisations themselves are now using AI to produce policies, reports, documentation, software code, marketing material and customer communications. Future AI systems may therefore be trained on content originally generated by previous AI systems.
This creates what economists describe as a negative feedback loop. Each generation becomes incrementally less connected to original human expertise.
Implications for Boards
This issue has implications across multiple areas of board responsibility.
From a strategic perspective, organisations should recognise that AI capability is not guaranteed to improve indefinitely. Roadmaps built on assumptions of continual model improvement should include contingency planning.
From a risk perspective, boards should understand how dependent critical business processes have become on third-party foundation models and whether suppliers can demonstrate the provenance and quality of their training data.
From a technology perspective, organisations should avoid assuming that newer models are automatically better models. Independent evaluation against business-critical use cases should become routine.
From a governance perspective, organisations should place greater emphasis on human oversight, recognising that maintaining human judgement may become more valuable rather than less valuable over time.
Questions Boards Should Ask
Boards should consider asking executive teams:
How dependent are our most important business processes on third-party AI models?
How do our AI suppliers assure the provenance and quality of their training data?
Have we independently tested whether newer AI models actually improve our critical business outcomes?
Which decisions will always require human expertise regardless of future AI capability?
How are we protecting internal human knowledge from being replaced entirely by synthetic content?
Are we creating organisational knowledge that future AI systems can learn from, or simply recycling AI-generated outputs?
What governance exists to ensure AI-generated content does not progressively contaminate our own internal knowledge base?
Recommendations
The emerging evidence does not suggest organisations should slow AI adoption. Rather, it suggests they should become more disciplined about how AI is governed. Boards should consider five immediate actions.
Treat high-quality human knowledge as a strategic asset. Encourage experts to continue producing original thinking, decision rationales and lessons learned rather than relying solely on AI-generated documentation.
Require AI suppliers to demonstrate appropriate governance over training data, model evaluation and quality assurance as part of procurement and ongoing assurance processes.
Establish independent evaluation of AI performance against business outcomes rather than assuming capability improves with each new model release.
Preserve meaningful human decision-making within high-impact activities. Human expertise should be seen as complementary to AI rather than something to eliminate.
Recognise that information quality is becoming a board-level governance issue. As synthetic content becomes more prevalent, organisations will need stronger controls over the provenance, integrity and trustworthiness of the information that informs both human and machine decision-making.
Conclusion
The discussion surrounding artificial intelligence has largely focused on what AI can do today. Boards should begin paying equal attention to what determines AI’s capability tomorrow.
The emerging evidence from Nature, the University of Oxford and investigative reporting by New Scientist points to a common conclusion: the long-term performance of AI depends as much on the integrity of its training data as it does on advances in algorithms or computing power.
This should not be viewed as a reason to delay AI adoption. Instead, it reinforces a broader governance principle: sustainable advantage will come not from using the most AI, but from using the most trustworthy AI.
For boards, the strategic question should therefore shift from “How quickly can we adopt AI?” to “How do we ensure the AI we depend upon continues to deserve our trust?”