Reposted from Motherboard
At around midnight just over 28 years ago a pair of thieves broke into the Isabella Stewart Gardner Museum in Boston, Massachusetts masked by darkness. The intruders, who were disguised as police officers, tied up museum security guards and used box cutters to rip the paintings. They made off with $500 million worth of art from the museum’s walls.
Among the stolen pieces were three paintings by Dutch Baroque artist Rembrandt and five by French impressionist Degas. Today, the case remain the largest unsolved art heists in history and no arrests have been made. The spaces where the 13 artworks once hung lay bare.
While we may never know the location of the physical paintings, a group of technologists are using augmented reality to virtually return these stolen works.
Using Apple’s ARKit, a team of nine technologist at Cuseum—a group that helps bring AR experiences to museums—have recreated Rembrandt's, “A Lady and Gentleman in Black” and “Christ in the Storm on the Sea of Galilee,”—two of the 13 stolen artworks. The group is calling the project, which is not sanctioned by the museum, Hacking the Heist.
"Something like this would not have been technologically possible a year ago but now with some of the recent advances in AR the phone has the ability to recognize vertical surfaces and images and react accordingly,” Brendan Ciecko, a technologist at Cuseum told Motherboard over the phone. “It dawned on me that a lot of people don't actually realize what the paintings that were looted look like.”
Ciecko’s Apple ARKit build runs on the latest iOS currently available only to developers. As soon as Apple releases the new version of iOS to the public, museum goers can download an app and view the virtually restored paintings on their own devices. Until then, the technologists hope to lend some iPads to the museum so that visitors can experience the work even if they don’t have the app.
Currently the team’s work is still a proof of concept, but according to Ciecko they intend to work with the museum to create a project open to the public in the near future.
Augmented Reality technologies have made strides in commercial use in the past year. The technology has also been utilized by many for artistic endeavors, like this group that recently used the technology to host their own AR exhibit in the Metropolitan Museum of Modern Art.
“Back in the day AR used to be pretty twitchy,” Ciecko said. “You would scan some sort of marker and it would pull up the video but it would be pretty twitchy because it couldn't track in high fidelity.”
Apple’s ARKit uses an iPhone or iPad camera to detect surfaces and does not require a marker or a QR code to place 3D objects in the environment. In the case of the Gardner museum paintings these technological advancements are essential because the actual space where the painting should be is empty.
Rather than rely on a marker, Ciecko and his team had to detect the canvases adjacent to the missing paintings. The team used the surrounding paintings as anchors rather than the more traditional approach of laying an image over another.
Since the work was not sanctioned by the museum, Ciecko and his fellow technologists experienced a close call with one of the museum's guards during their early stages. According to Ciecko after spending over 45 minutes in front of the paintings to test the technology, the guard became suspicious and a supervisor was called.
Tourists passing by saw the images of the paintings on the technologists’ ipads as they tested their AR and would often ask questions and take pictures according to Ciecko.
Looking to the future Ciecko said he hopes this work will inspire more people to use AR to create cultural and educational experiences.
“Looking at this specific institution there were 13 pieces that were looted," he said."It is a dark and evil act to loot an institution that is open for the public enjoyment so we really wanted to highlight our feelings about this especially as technologists, especially as artists and especially as Bostonians.”
See Original Post
