Editing photographs is almost as old as, well, photography itself, but researchers at Carnegie Mellon University and the University of California, Berkeley have taken image manipulation to another dimension — literally.
They’ve developed software that enables users to move and animate objects in a photograph — exposing angles, sides and surfaces unseen in the original image.
For example, maybe you snap a picture of the back of a chair, but you really want to see the front. Using a massive database of stock 3D models, the software finds a copy of the chair and automatically adjusts the lighting and texture, generating a realistic image that can be moved 360 degrees in any direction.
CMU PhD student and lead author of the study, Natasha Kholgade said the software is different from editing programs such as Photoshop, which lack sufficient 3D manipulation tools.
“Movements you make are solely in pixel-space,” she said. “You cannot really lift objects up out of an image and move them around. Yet, we as humans, we actually imagine doing that because we are more used to dealing with the world in three dimensions.”
While the tool was designed for digital images, users can edit paintings, as well as historical photographs. The software can also bring still images to life in an animated scene.
Kohlgade said she was inspired by the need to create a more dynamic photograph.
“It’s this drive to be able to convey our imagination, which is so huge, in a physical medium,” she said. “This is what propelled us to develop the software.”
The program does come with limitations. It’s completely dependent on the number of 3D models that exist online, but Yaser Sheikh, an associate research professor of robotics at CMU, said that problem will solve itself thanks to the development of 3D scanning and printing technologies.
“The more pressing question will soon be, not whether a particular model exists online, but whether the user can find it,” he said.
Researchers are also having challenges manipulating transparent items, but Kohlgade said the most difficult objects to edit are human beings.
“One of the most important components of a photograph is the person you’re taking a photo of,” she said. “So, an exciting direction is ‘can you edit in a person in a photo to do a funky looking pose that they couldn’t have done in the real world?’”
Kohlgade hopes this technology could one day be added to popular photo editing tools such as Instagram or Photoshop.
“What we would like to see is people combining their current expertise at using 2D techniques with 3D techniques to create even more drastic changes to their photographs, to visualize their imagination,” she said.
Kohlgade is traveling to Vancouver later this month to present the software at the Special Interest Group on Graphics and Interactive Techniques Conference.
