The researchers from the Massachusetts Institute of Technology together with Google have developed an artificial intelligence system that can create realistic 3-dimensional object models.

In a paper, the team presented Visual Object Networks, a new system that is capable of creating shapes complete with 3D tweaks, including texture, lighting, and viewpoints.

The paper was accepted at the 2018 Conference on Neural Information Processing Systems (NeurIPS), a summit focusing on artificial intelligence. The conference will run from Dec. 2 to Dec. 9 in Montreal, Canada.

AI Generating Realistic Images

"Most computational models have only focused on generating a 2D image, ignoring the 3D nature of the world," explained the researchers. "This 2D-only perspective inevitably limits their practical usages in many fields, such as synthetic data generation, robotic learning, visual reality, and the gaming industry."

Researchers bypassed these limits by going through a process they call "disentangled object representation." Basically, what happens is that the AI decomposes the image into three factors: shape, viewpoint, and texture. It also learns how to create 3-dimensional shapes before adding "2.5D" sketches.

The process also allowed the researchers to train the system by using large troves of data from Google image search, Pix3D, and ShapeNet. However, to get VON to create images on its own, the team trained an adversarial network, also known as GAN, which is a two-part neutral network. The researchers revealed that after only two to three days, the AI can already produce realistic 128 x 128 x 128 objects.

To test, the researchers calculated the Fréchet Inception Distance of the images generated by the system and showed 200 pairs of generated images plus models to Amazon's Mechanical Turk. In both cases, the VON performed with flying colors.

However, the work is not finished yet. The researchers want to later on work on the "coarse-to-fine" modeling of generating images at a higher resolution and synthesize natural scenes.

2018 NeurIPS Top Honors

Aside from the VON project, the 2018 NeurIPS also announced three other best paper selections during the event's opening remarks, including submissions from Huawei Noah's Ark Lab and Microsoft Research.

The Test of Time Award went to the researchers from NEC Laboratories of America and Google Zurich who submitted a paper on "The Tradeoffs of Large-Scale Learning."

The NeurIPS 2018 conference is expected to be attended by almost 9,000 people throughout the week.