The Massachusetts Institute of Technology (MIT) and Microsoft Research have collaboratively developed a novel system called "Ekho" to tackle synchronization challenges in cloud gaming and broader networking scenarios.
According to MIT News, the system utilizes inaudible white noise sequences to synchronize audio and video streams transmitted to different devices.
The Ekho System Developed by MIT and Microsoft Research
Cloud gaming has witnessed remarkable growth, particularly during the COVID-19 pandemic, with a global market estimated at $6 billion and over 23 million players worldwide.
However, one persistent issue in cloud gaming and networked environments is video, audio, and haptic feedback synchronization across multiple devices, which often operate on separate networks, causing noticeable delays between streams.
The Ekho system, developed by MIT and Microsoft Research, introduces inaudible white noise sequences to the game audio streamed from the cloud server.
Simultaneously, it listens for these sequences in the audio recorded by the player's controller, using the mismatch between these sequences to continually measure and compensate for interstream delays.
In actual cloud gaming sessions, Ekho demonstrated its reliability by keeping streams synchronized within less than 10 milliseconds of each other in most cases. In contrast, other synchronization methods consistently resulted in delays exceeding 50 milliseconds.
While initially designed for cloud gaming, this synchronization technique holds potential for broader applications, such as synchronizing media streams for multiple devices in training scenarios, including augmented or virtual reality headsets.
Pouya Hamadanian, an electrical engineering and computer science graduate student and lead author of the research, said: "Sometimes, all it takes for a good solution to come out is to think outside what has been defined for you."
"The entire community has been fixed on how to solve this problem by synchronizing through the network. Synchronizing two streams by listening to the audio in the room sounded crazy, but it turned out to be a very good solution," Hamadanian added.
Read Also: MIT Engineers Develop DNA-Assembled Quantum Rods to Enhance TVs, VR Devices
Bypassing Synchronization Methods
The report further explains that clock synchronization, a fundamental problem in networking, lies at the heart of interstream delay in cloud gaming.
Existing approaches, like ping-pong messaging, involve sending a ping message to the server, which responds with a pong message, and the device calculates network delay based on this exchange.
However, asymmetric network paths and other factors could make this method unreliable, leading to significant errors. The Ekho system bypasses traditional synchronization methods and leverages inaudible white noise sequences to achieve sub-millisecond accuracy in interstream synchronization.
It adds these sequences to game audio, which are later used for precise synchronization, overcoming challenges posed by background noise and compression in the recording.
The Ekho system consists of two modules: Ekho-Estimator and Ekho-Compensator. The Ekho-Estimator module adds pseudo-noise sequences to the game audio and listens for them in the recorded audio to calculate inter-stream delay. The Ekho-Compensator module adjusts the game audio sent by the server to synchronize the streams.
Future work on Ekho aims to evaluate its performance in more complex scenarios, such as synchronizing multiple controllers with a single-screen device. Additionally, researchers are exploring ways to enhance Ekho's capabilities for larger environments, such as concert halls.
Related Article: Microsoft Will Unbundle Teams to Allay EU Antitrust Concerns
