- in News
A new Artificial Intelligence and Augmented Reality application is being developed that makes it possible to take 2D images and not only convert them into 3D, but also to place them into a new (real) environment.
Called Volume, it is the brainchild of Israeli artists and technologists Or Fleisher and Shirin Anlen. It is very much in the early stage of development, but the creators have done a video on YouTube, showcasing the application’s potential by taking the dance contest scene from Quentin Tarantino’s Pulp Fiction and transporting John Travolta and Uma Thurman into a living room.
Fleischer explains it thus:
Our experiment with Pulp Fiction allows users to step inside one the film’s scenes in Augmented Reality, using Apple’s ARKit framework on an iPad. This experiment, is one of a few we are conducting at the moment, which illustrate the power of being able to reconstruct 3D scenes from 2D images. The possibilities of being able to reconstruct archival and static footage into 3D environments are one of the main motivations behind the development of the tool used to create these experiments called Volume.
The system is based on a deep learning tool called a “convolutional neural network” to teach the system. The neural network analyzes a two dimensional image and process, looking at contrast and changing colors of pixels and groups of pixels. It then tries to re-imagine the image in 3D and place it into the designated new environment. The extracted former 2D image thus becomes a 3D AR image in a new environment or context.
The agenda here is not to create another high-end tool for the video industry, but rather to create a web-based tool that will enable ordinary users to capture video images with their smartphone, tablet or camera and then upload them for automatic conversion.
The makers are the first to admit that the project is in its early stages. But they see great potential for the technology. It’s not just about having actors dancing in your living room. It could be Winston Churchill making a speech in war-torn England, Muhammad Ali slugging it out with George Foreman, or your favorite band making a comeback just for you and your friends.