Letting Google’s AI dream in 3D

I’m a big fan of Google’s trippy explorations at the intersection of photography and artificial intelligence via the DeepDream neural network software, and I’m a longtime enthusiast of virtual reality. Recently, I wondered what might happen if I connected these two interests.

Fortunately, my friends Joseph Smarr of Google and Chris Lamb of NVIDIA had recently helped our mutual friend and professional photographer Dan Ambrosi overcome the technical hurdles to apply DeepDreaming to his multi-hundred megapixel panoramic photos, with some pretty stunning results. Might they be willing to run a few of my Photospheres through their highly scalable, cloud-based DeepDreaming instance? Indeed, they were.

The first Photosphere we put through the process was something I captured during a photoshoot for the homepage of my new startup, Parietal VR. We chose to let the neural networks go full “animalistic,” so the forest scene got fully populated with a nightmarish assortment of creatures. Here’s the Photosphere as a 2D projection, before and after DeepDreaming:

PANO_20160201_152511 (1)

PANO_ McCrea_dream

If you’d like to see it in 3D in your browser, I uploaded the processed Photosphere to VRCHIVE. But if you’re on Android and have the Cardboard demo app and a Google Cardboard, you should save the image to your phone and check it out in stereoscopic 3D.

[Note: For the Cardboard demo app to recognize it as a Photosphere, you may need to edit the filename to change “pano” to “PANO”. WordPress decapitalized it upon upload.]

For our second “dream sphere” we decided to veer away from animalistic in favor of more abstract, using a Photosphere I shot of an outdoor sculpture in Palo Alto, made from willow branches, “Double Take” by Patrick Daugherty. Here’s the 2D projection of the source Photosphere before and after processing:

PANO_20160205_161829

PANO_20160205_161829_dream_inc_3b_5x5_reduce

The transformation is much more subtle, at least at this scale. But check out a small section:

Screen Shot 2016-02-27 at 10.47.32 AM

And it’s way better in 3D, as you can imagine. Check it out on VRCHIVE. Or, as above, save to your phone and experience in Google Cardboard.

Note: For those curious, I captured these Photospheres with Google’s Camera app for Android on a Samsung Galaxy S6.

 

 

Tagged , , , , , ,

2 thoughts on “Letting Google’s AI dream in 3D

  1. claytantor says:

    What I would love to see is deep dream creating renderable 3D scenes from faceted models. Take any 3d model and it would process it algorithmically. You would then have a trippy world to walk in.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: