Researchers Warn AI Firms Could Run Out of Training Data

Joseph Henry, Tech Times 15 November 2023, 02:11 pm

The AI revolution, driven by data, faces a critical challenge - the impending scarcity of high-quality training data.

As AI models thrive on an abundance of diverse, natural data, the industry grapples with the realization that this invaluable resource is finite, potentially leading to its downfall.

Data Depletion and the AI Forecast

Researchers Warn AI Firms Could Run Out of Training Data

(Photo: Steve Johnson from Unsplash) The AI economy is freely operating because of the availability of data for training. However, it's not as infinite as what we imagine as data wells might run dry someday.

AI researchers, alarmed by the diminishing data supply, have issued warnings for nearly a year. This was indicated in an essay from The Conversation.

A study from the AI forecasting organization Epoch AI predicts that AI companies may exhaust their reservoirs of high-quality textual training data by 2026. The situation is even more precarious for low-quality text and image data, expected to deplete between 2030 and 2060.

Impact on AI Advancements

The role of data in AI models is pivotal; continuous improvement and functionality depend on the influx of quality, human-made data. The stagnation of this data supply poses a potential threat to the advancement of AI systems, hindering the industry's growth.

Synthetic Data as a Mitigation Strategy

While the use of synthetic data, generated by AI models, emerges as a potential solution, challenges persist.

Research suggests that training AI models on AI-generated content may result in an inbreeding effect, causing distorted and uncanny outputs. Despite these challenges, some companies are already experimenting with synthetic training sets.

The Crucial Role of Data Partnerships

Amid this looming problem, data partnerships stand out as a practical solution. Companies or institutions possessing vast and sought-after datasets can strike deals with AI firms to provide essential data in exchange for financial compensation. Somehow, firms are finding ways to avoid this problem which can happen anytime.

"Modern AI technology learns skills and aspects of our world - of people, our motivations, interactions, and the way we communicate - by making sense of the data on which it's trained," OpenaI wrote on its latest blog.

Competing for Valuable Datasets

As data becomes an increasingly precious commodity, the dynamics of AI companies competing for datasets will be intriguing.

To make it clearer, the datasets currently used for AI training often originate from internet-scraped data created by online users. The feasibility of securing these datasets through partnerships raises questions about the willingness of institutions and individuals to contribute their valuable data to AI endeavors.

The Uncertain Future of Data Wells

Even with data partnerships, the long-term sustainability of AI's data supply remains uncertain, Futurism writes. The illusion of an endless internet is dispelled by the realization that few resources are truly infinite.

Since not all data are suited for AI training, some countries like China will blacklist sources with illegal training data.