OpenAI has transcribed over a million hours of YouTube videos to train its latest model. Various strategies employed by major players in artificial intelligence to enhance their access to data have been outlined.
A photo taken on February 26, 2024, shows the logo of the ChatGPT application developed by US artificial intelligence research organization OpenAI on a smartphone screen (L) and the letters AI on a laptop screen in Frankfurt am Main, western Germany. (Photo: KIRILL KUDRYAVTSEV/AFP via Getty Images)
Navigating Legal and Ethical Boundaries
Recent challenges in acquiring high-quality training data for AI models have prompted major players in the field to explore innovative solutions. Earlier discussions underscored the limitations AI companies faced in obtaining such data.
Today, The New York Times delves into the strategies companies have employed to address this issue, often operating within the ambiguous boundaries of AI copyright law.
This sheds light on OpenAI's approach, which involved developing its Whisper audio transcription model to amass training data.
OpenAI reportedly transcribed over a million hours of YouTube videos, leveraging this vast dataset to train its advanced language model, GPT-4.
Despite acknowledging the legal uncertainties surrounding this endeavor, OpenAI believed it fell within the realm of fair use. OpenAI's president, Greg Brockman, directly sourced the videos utilized in this initiative.
Spokesperson Lindsay Held emphasized the company's commitment to tailoring unique datasets for each model, aiming to enhance their understanding of the world and bolster global research competitiveness.
Held further explained that OpenAI utilizes various sources, including publicly available data and partnerships for non-public data, while exploring the potential of generating synthetic data internally.
Exploring Alternative Data Sources
In 2021, the company faced a shortage of valuable data resources and, after exhausting other avenues, began exploring options such as transcribing YouTube videos, podcasts, and audiobooks.
Before this, their models had been trained on diverse datasets, including computer code sourced from Github, chess move databases, and educational material from platforms like Quizlet.
Matt Bryant, a spokesperson for Google, responded to inquiries regarding OpenAI's activities, stating that the company had heard unverified reports about their actions.
Bryant emphasized that both the robots.txt files and the Terms of Service of Google explicitly prohibit any unauthorized scraping or downloading of YouTube content, aligning with the company's usage policies.
In a strategic move, Google's legal department instructed its privacy team to revise its policy language to broaden the scope of permissible actions with consumer data, including data generated from office tools like Google Docs.
The updated policy was allegedly deliberately unveiled on July 1st, strategically timed to coincide with the Independence Day holiday weekend when public attention was expected to be diverted.
Google, OpenAI, and the broader AI training world are grappling with the dwindling availability of training data.
According to recent reports, companies may surpass new content by 2028. Potential solutions include training models on "synthetic" data or adopting "curriculum learning."
However, the effectiveness of these approaches remains uncertain. Alternatively, companies may resort to using available data despite legal and ethical concerns evidenced by recent lawsuits.
