NYU has initiated a groundbreaking project aimed at promoting the advancement of immersive 3D video technology. This primarily focuses on its application in dance education and potentially other fields.

(Photo : NYU Tandon)
A groundbreaking research project spearheaded by NYU Tandon School of Engineering will bring cutting-edge immersive three-dimensional (3D) video to dance education.

Promoting Advancement of 3D Video Technology

A revolutionary research initiative led by NYU Tandon School of Engineering will introduce state-of-the-art immersive three-dimensional (3D) video technology to dance education, increasing accessibility for a wide spectrum of students interested in this art form.

Supported by a generous $1.2 million grant from the National Science Foundation spanning four years, Engadget reported that this seeks to render Point-Cloud Video (PCV) technology suitable for streaming.

Crafted over the course of a decade, PCV represents an advanced form of 3D video technology, providing viewers with the ability to virtually navigate an immersive video setting, interact with objects, and explore scenes from various angles and distances, replicating a physical presence.

Unlike conventional video formats that rely on two-dimensional pixels, PCV captures content in the form of point clouds, which are sets of data points in three-dimensional space. Presently, the streaming of PCVs poses challenges due to its high bandwidth and computational demands, making it less practical for real-world applications.

NYU Tandon's project is dedicated to overcoming these obstacles by optimizing bandwidth usage, reducing delivery latency, and enhancing power efficiency. The ultimate goal is to make PCV technology more accessible and manageable for seamless streaming. 

Also Read: Facebook Blocks COVID-19 Misinformation Researchers from NYU, Cuts off Access to Ad Library

Yong Liu, a professor of electrical and computer engineering (ECE), leads a research team, which includes ECE Professor Yao Wang and R. Luke DuBois, an associate professor in the Technology, Culture and Society Department.

As per the press release, their work is advancing the capability to teleport holographic representations of real-world entities through the global Internet, leveraging recent progress in enabling technologies.

This breakthrough has the potential to revolutionize immersive augmented, virtual, and mixed reality experiences and create new opportunities in education, business, healthcare, and entertainment.

Collaborative Effort

DuBois is overseeing a collaborative effort between the Mark Morris Dance Group's Dance Center and the Department of Dance at NYU Tisch School of the Arts as part of this initiative.

They aim to evaluate PCV streaming in the context of human motion, and the collaboration is especially focused on dance education.

Dancers from both institutions will showcase their performances on a volumetric capture stage at NYU Tandon's facility in the Brooklyn Navy Yard, with their movements being live-streamed and made available on-demand using the new PCV testbed developed by the research team.

This project signifies NYU's commitment to interdisciplinary research by uniting engineering with diverse fields, notably the art of dance.

The Wall Street Journal reported that NYU Tandon is leading an effort to fuse emerging technologies into the realms of dance and education, promising to enhance accessibility and expand the creative horizons for artists.

The collaboration between the Department of Dance at NYU Tisch School of the Arts, NYU Tandon, and the Mark Morris Dance Group marks a significant leap in research at the intersection of technology and art.

This innovative project will allow students watching 3D streams to observe dancers from various angles and distances, offering a comprehensive learning experience for dance techniques and choreography. 

Related Article: NYU Scientists Develop Algorithm That Teaches Machines To Learn Like Humans