Diverse perspectives among gamers worldwide were reported to be instrumental in refining AI algorithms that could help everyone pick the right games, according to recent research from Cornell, Xbox, and Microsoft Research.
The collaborative effort, which enlisted the input of over 5,000 gamers, demonstrates that predictive models fueled by extensive datasets annotated by gamers from varied cultural backgrounds outperform those labeled exclusively by gamers from a single nation.
TOPSHOT - Gamers from the Vietnam team compete in the qualifying rounds of the eSports event between Malaysia and Vietnam at the SEA Games (Southeast Asian Games) in Manila on December 5, 2019.
How Gamers Help Predictive AI Models
The team's discoveries and associated recommendations extend beyond the realm of gaming, offering insights for researchers and practitioners seeking data labeling methods with broader global applicability, ultimately leading to more accurate predictive artificial intelligence (AI) models.
"We show that, in fact, you can do just as well, if not better, by diversifying the underlying data that goes into predictive models," said Allison Koenecke, assistant professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science.
Koenecke, the lead author of "Auditing Cross-Cultural Consistency of Human-Annotated Labels for Recommendation Systems," presented at the Association for Computing Machinery Fairness, Accountability, and Transparency (ACM FAccT) conference, highlights that the accuracy of predictive models hinges on the quality of their foundational data, particularly the meticulous labeling of individual elements within extensive datasets.
While crowdsourced workers have become integral in this labeling process, their composition tends to be relatively uniform, potentially introducing cultural biases that distort the intended global outreach of predictive models, according to the study.
5,174 Xbox Gamers
The research involved surveying 5,174 Xbox gamers across the globe to assist in labeling various gaming titles. Participants were tasked with assigning labels such as "cozy," "fantasy," or "pacifist" based on their gaming experiences, taking into account factors like game complexity and controls.
Certain labels, such as "zen" denoting peaceful and calming games, exhibited consistent application across cultures. In contrast, descriptors like "replayable" showed varying degrees of application.
The research team attributed these disparities to cultural distinctions among gamers and idiosyncrasies in translation and language that influenced labeling variations across different nations.
Subsequently, the team developed two models capable of predicting how gamers from different countries would label specific games. One model incorporated survey data from a globally diverse pool of gamers, while the other relied exclusively on data from American gamers.
Compared to the model relying solely on American labels, the results demonstrated an 8% improvement in predictions for gamers worldwide when employing the model trained on labels from culturally diverse populations.
Furthermore, the researchers developed a framework to assist colleagues in academia and industry professionals in thoroughly examining the foundational data labels for worldwide inclusiveness.
"With our framework, any academic researcher or practitioner could audit their own underlying data to see if they might be running into issues of representation via their data labels or choices," Koenecke noted.
Related Article: Pokemon Go Players Outraged by September 2023 Shadow Zapdos Raids; Exciting City Safari Event on the Horizon
