Caffe On Joules - Part two: Cmake soup

I assume that now you have Cmake and OpenCV installed as detailed on the previous post on Caffe. So this post will be mostly about "cmake"ing stuff and sticking them together to compile caffe. First we need to go back into the docker container that the Intel® System Studio IoT uses. To do this fire up the ISSI workspace which also results in getting the docker container running, then probe the docker containers and get a bash shell running in the ISSI container:

docker ps
docker exec -i -t  /bin/bash

And now that we are are inside the ISSI docker container, with no further ado, let's dig in to download and cmake and install a fresh copy of Atlas[1]. Note that code was used from [2]:

cd /home/root/
wget http://freefr.dl.sourceforge.net/project/math-atlas/Stable/3.10.3/atlas3.10.3.tar.bz2
wget http://www.netlib.org/lapack/lapack-3.5.0.tgz
tar xjvf atlas3.10.3.tar.bz2
cd ATLAS
mkdir build
cd build
../configure --prefix=/usr/local --with-netlib-lapack-tarfile=/home/root/lapack-3.5.0.tgz --nof77 -v 2 --cripple-atlas-performance
make build
make install

Then we go onto installing the HDF5 library according to [3]:

cd /home/root/
wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.0-patch1/src/CMake-hdf5-1.10.0-patch1.tar.gz
tar xzvf CMake-hdf5-1.10.0-patch1.tar.gz
cd CMake-hdf5-1.10.0-patch1
./build-unix.sh
cd build
cmake .
make
make install

Then lets do protobuf:

cd /home/root/
wget https://github.com/google/protobuf/archive/master.zip -O protobuf.zip
unzip protobuf.zip
cd protobuf-master/
./autogen.sh
./configure
make
make check
make install

Then we have LMDB:

cd /home/root/
wget https://github.com/LMDB/lmdb/archive/mdb.master.zip
unzip mdb.master.zip
cd lmdb-mdb.master/libraries/liblmdb/
make && make install

Then gflags:

cd /home/root/
wget https://github.com/gflags/gflags/archive/master.zip -O gflags.zip
unzip gflags.zip
cd gflags-master/ && mkdir build && cd build
cmake .. && make && make install

Then glog, yes life is hard:

cd /home/root/
wget https://github.com/google/glog/archive/master.zip -O glog.zip
unzip glog.zip
cd glog-master/ && mkdir build && cd build
cmake .. && make && make install

Finally we get to caffe:

cd /home/root/
wget https://github.com/BVLC/caffe/archive/master.zip -O caffe.zip
unzip caffe.zip
cd caffe-master && mkdir build && cd build
ccmake ..

Notice that this time we are using ccmake instead of cmake. Indeed we need to do some surgery to get things working. In the ccmake dialog press "c" and you will be presented with an error. Something about HDF5 not being configured properly. To get past this quickly we are going to set the HDF5 variables manually. To do this first press "e" to exit the help dialog. And then press "t" to enter advanced mode, yes you are now officially CMAKE NINJA. We'll have to manually copy the following lines to their corresponding variables in ccmake:

HDF5_CXX_INCLUDE_DIR:/usr/local/myhdf5/include/
HDF5_C_INCLUDE_DIR:/usr/local/myhdf5/include/
HDF5_hdf5_LIBRARY_RELEASE:/usr/local/myhdf5/lib/libhdf5.a
HDF5_hdf5_cpp_LIBRARY_RELEASE:/usr/local/myhdf5/lib/libhdf5_cpp.a
HDF5_hdf5_hl_LIBRARY_RELEASE:/usr/local/myhdf5/lib/libhdf5_hl.a
HDF5_hdf5_hl_cpp_LIBRARY_RELEA:/usr/local/myhdf5/lib/libhdf5_hl_cpp.a
CMAKE_EXE_LINKER_FLAGS:-ldl -lszip -lz -L/home/root/CMake-hdf5-1.8.17/build/_CPack_Packages/Linux/TGZ/HDF5-1.8.17-Linux/HDF_Group/HDF5/1.8.17/lib/

While you are in ccmake you better turn off these flags: USE_LEVELDB, USE_CUDNN, BUILD_SHARED_LIBS, and turn on CPU_ONLY. Now press "c" and "g" then "make" caffe! :) With this Caffe is built, and you can run it on the docker container. Furthermore you can try to run it on Joule, but you will have missing libraries. For fixing this you can copy all the lib files from the docker container into a folder on the caffe /root/home/ directory. Of course you'd have to set the LD_LIBRARY_PATH to the said directory. With all this work, caffe still wont work on Joule. If you remember, we built atlas on the docker container, hence it uses instructions that work on your CPU and as soon as it start computing something, caffe will crash. To remedy this we should build the atlas libraries on the Joule device, but this is the story of another day. gl hf. [1] http://math-atlas.sourceforge.net/
[2] http://theworldofinteledison.blogspot.com.es/2015/04/getting-scipy-working-on-edison.html
[3] https://support.hdfgroup.org/HDF5/release/cmakebuild.html