Thursday, July 2, 2015

Setting Up Deep Dream, Google Research's Hallucinatory Work of Genius

Google Research wrote recently about a technique they call "Inceptionism", which needs to be seen to be believed. Click through and check out their pictures. A full gallery of official images can be found here. The techniques used to generate these sorts of images were described in broad strokes in this blog post, but the level of detail stopped just short of what someone wanting to replicate their results might have wanted.

That is, until they published their source code.

In this repo (which was created just this morning, as of this post's writing) is an IPython notebook which contains everything necessary to duplicate the results you see in their blog post. That's so cool that it's honestly a little hard to believe. The dependencies are detailed in the repo. There are surprisingly few of them, and they're all surprisingly easy to get set up. Let me walk you through what I had to do on my Debian Stretch system to get everything up and running.

NumPy, SciPy, PIL, and IPython top the list of dependencies. These are all libraries that the savvy Pythonista likely already has installed. Anyone without them should  hasten to change this state of affairs. If you have Python's library manager, pip, installed -- and you should -- then the best way to install all these libraries is by invoking,
sudo pip install numpy scipy Pillow ipython
You're also going to need "protobuf". This is a Google library, and should be available through your package manager. On Debian,
sudo apt-get install protobuf-compiler
did the trick for me.

That takes care of the easy dependencies. Next, we have to install Caffe. This is where I spent most of my time. It turns out that the painless way to handle this whole process is as follows.

First off, get Caffe's own prerequisites out of the way. I used the list of apt-get packages from their Ubuntu install reference. To wit:
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler libatlas-base-dev
I've had smooth sailing just installing that list. As ever, YMMV. If anyone finds that they need to install other packages in addition to these, please leave a comment, and I'll amend the list.

Anyhow, once that's taken care of, cd to whatever directory you want to install Caffe in (I put it in my usual ~/workspace directory), then download the Git repo and configure whatever options you'd like. There are several ways to get the Git repo. First, make sure to set up Git, and follow the instructions on that page for either connecting over HTTPS or SSH. Then, depending on which of those you set up, run one of the two following commands. If you set up HTTPS, use
git clone https://github.com/BVLC/caffe.git
to clone over HTTPS, or if you set up SSH, then
git clone git@github.com:BVLC/caffe.git 
to clone the repo that way. (thanks misterque and Pui for their comments on this section). Once you've run one of these commands, which will create a new "caffe" folder in the current working directory, do the following:
cd caffe 
cp Makefile.config{.example,} 
open Makefile.config in your favorite editor
and change config parameters as needed. Make sure you already have CUDA set up, and if you're going to uncomment the cuDNN line, make sure you have that installed too, of course. Now, to actually build this monstrosity...
mkdir build 
cd build 
cmake .. 
make pycaffe
make runtest 
All of these will produce a fair bit of output, especially the last two. The last one runs tests and is not actually necessary, but it is highly recommended as a way to make sure everything's gone right. Hopefully all the output produced by these commands is encouraging. If it isn't, you have a problem -- likely have an unmet dependency -- to troubleshoot. As a first step in troubleshooting, you could try running the above pip and apt-get commands again (possibly with --upgrade in the case of pip). Past that, it's all up to you. Google your error message along with "Caffe" and you can likely find some documentation on what to do.

If you managed not to hit any errors, you're now out of the woods! The hard part is over. The last step is to put Caffe in your PYTHONPATH so you can import it. I just did this the quick-and-dirty way, by putting
export PYTHONPATH=/home/fish/workspace/caffe/python:$PYTHONPATH
at the end of my .bashrc file. Obviously, you're going to want to change the path to reflect wherever you cloned Caffe into.

Now, restart your shell or source ~/.bashrc and you're set! You now have all your dependencies taken care of and can follow along with the IPython notebook in the deepdream repo linked at the top of this post. Well--almost. Your Caffe install comes with the skeletons necessary to use a few different models out of the box, but the trained models themselves aren't included with the repo because they're so large. You'll have to download them individually. To get the model used in the notebook, navigate to your Caffe install dir, then
cd models/bvlc_googlenet/
wget http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel
This link is unlikely to change any time soon, but if it does, you can expect to find an updated link in the directory's readme.md file. Once wget finishes up, you'll have the model you need to work through the example. If you want to see the images as you work (and you probably do) then you can't run IPython this on the command line, though -- you have to start up its fancy Qt GUI console (which I didn't even know about until today). You can get this by running,
ipython qtconsole
and then you're ready to follow along!

Note, by the way, that if you took the bold route and compiled Caffe with GPU support, and if you have a compatible GPU, then you can run caffe.set_mode_gpu() right away before dreaming, and it'll likely speed the process up by several orders of magnitude. There is an open pull request to integrate information on this into the Google notebook; you can find the pull request here. It has some discussion on the topic.

I'm having a whole lot of fun with this. Expect a follow-up post in the next few days with a bunch of cool pictures, and possibly some variations on the theme.

17 comments:

  1. ubuntu@ubuntu:~$ git clone git@github.com:BVLC/caffe.git
    Cloning into 'caffe'...
    Permission denied (publickey).
    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    and the repository exists.

    ReplyDelete
    Replies
    1. misterque's solution, below, should solve this problem for you. I've amended the original post as well.

      Delete
  2. git clone git@github.com:BVLC/caffe.git is for an ssh connection. Git recommends using https

    make sure git is installed.

    https://help.github.com/articles/set-up-git/

    git config --global user.name "YOUR NAME"

    git config --global user.email "YOUR EMAIL ADDRESS"

    git config --global credential.helper cache

    Then use below to clone.

    https://github.com/BVLC/caffe.git

    ReplyDelete
    Replies
    1. That's correct. If using the ssh URL, one needs to have previously set up SSH keys on github. Apologies for not bringing that up; I'll edit the post.

      Delete
  3. Does anyone know a way around using Cuda? I have an AMD GPU, not Nvidia unfortunately. Thanks for nothing, Apple.

    ReplyDelete
    Replies
    1. comment out the CUDA line in the Makefile

      Delete
    2. My bad... the line to change in Makefile.config is

      # CPU-only switch (uncomment to build without GPU support).
      CPU_ONLY := 1

      Delete the comment (#) at the beginning of the CPU_ONLY line. The build then ignores the CUDA stuff.

      Delete
  4. I thought I had built everything correctly, but right off the bat I am getting :

    ImportError: No module named google.protobuf

    Any ideas?

    ReplyDelete
    Replies
    1. Did you ever figure this out? I know my caffe install is correct, and protobuf is installed, because caffe uses it, but every time I run the first cell in dream.ipynb I get the same error you get: ImportError: No module named google.protobuf. I even added

      import sys
      sys.path.append('/usr/local/lib/python2.7/dist-packages/google/')

      to the first cell, but still no luck. Really want to get this working...

      Delete
    2. Figured it out:

      import sys
      sys.path.append("/usr/lib/python2.7/dist-packages/")
      sys.path.append("/home/caffe/python")

      Good luck!

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  6. I followed it all along, installed all the packages I didn't have that you must have already had (Atlas, iPython etc.). Got the tests to run clean. Now what?

    I have no idea how to 'run' the Google code.

    ReplyDelete
  7. -- Boost version: 1.55.0
    -- Found the following Boost libraries:
    -- system
    -- thread
    -- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
    -- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
    -- Found PROTOBUF Compiler: /usr/bin/protoc
    -- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
    -- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
    -- Found Snappy (include: /usr/include, library: /usr/lib/libsnappy.so)
    -- CUDA is not detected by cmake. Building without it...
    -- OpenCV found (/usr/share/OpenCV)
    CMake Error at /usr/share/cmake-3.0/Modules/FindPackageHandleStandardArgs.cmake:136 (message):
    Could NOT find Atlas (missing: Atlas_CLAPACK_INCLUDE_DIR
    Atlas_CBLAS_LIBRARY Atlas_BLAS_LIBRARY Atlas_LAPACK_LIBRARY)
    Call Stack (most recent call first):
    /usr/share/cmake-3.0/Modules/FindPackageHandleStandardArgs.cmake:343 (_FPHSA_FAILURE_MESSAGE)
    cmake/Modules/FindAtlas.cmake:43 (find_package_handle_standard_args)
    cmake/Dependencies.cmake:74 (find_package)
    CMakeLists.txt:28 (include)


    -- Configuring incomplete, errors occurred!

    Now what? I can't seem to find a straightforward way to get ATLAS

    ReplyDelete
    Replies
    1. Try: sudo apt-get install libatlas-base-dev
      Worked for me, hope this helps.

      Delete
    2. Thanks. I've added libatlas-base-dev to the big list.

      Delete