Caffe on Intel Joule - Part one: return of OpenCV

TL;DR:
I have a Caffe. I have a Joule. UH: Caffe-Joule [1]:

Long story is that Caffe is a deep learning framework that is primarily in C++[2]. Essentially it is a package that allows training and deploying deep neural networks. It uses protobufs to define a language for constructing different networks and uses GPUs and/or BLAS libraries to speed up the training and testing process.

Joule is a new hardware platform by Intel that was just released by Intel[3]. It features a quad-core Atom CPU, 4 GB of ram, and 16 GB of eMMC all in a package half the size of a credit card. That is it packs the computational power of a laptop in a package that fits in a pocket.

Now I am sure there is a ton of new things that we can do with the new Joule platform, but my prediction is that performing deep neural network computations is the main breakthrough that it achieves. With deep learning the device can use microphones and camera images to form an understanding of the surroundings that has previously been impossible to achieve.

In this article I will show how to setup and run a sample OpenCV program on the Intel Joule. The next article will continue with the installation of Caffe.

To get started first go a head to [4] and complete it up to and including the section: "Setting up the Intel® System Studio IoT Edition (Linux*)". So now you have a project setup that blinks the LED on the board. Now we'll transform this project to one that uses opencv to access a USB camera and save images to the root's home directory.

First, we shall copy the following piece of code in the place of the main file:

/*
Hey! this is a Hack. don't use for production.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/

#include <unistd.h>

#include <csignal>  //Library from the C/++ standard libraries to allow for clean exits.
#include <cstdlib>  //
#include <unistd.h> //
#include <iostream> //The most commonly-included file. This allows you to use cin and cout to print to the console and collect user input.

#include <mraa.hpp> //Intel's MRAA Library. This library allows the programmer to address the GPIO pins on the development board.

#include <cv.h>
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include <stdio.h>


using namespace std; //Commonly considered bad practice, this command prevents some simple mistakes that can be difficult to track down for a beginner. Generally, though, you wouldn't do this.
using namespace cv;
//This is a global variable which will hold our GPIO object. (General Purpose In/Out)
mraa::Gpio *gpio;

//This function is called a "signal handler." It allows a signal like "SIGINT" to exit the program cleanly.
//In this case, the SIGINT will be generated by the OS when you press Ctrl+C on your keyboard.
void signal_handler(int sig) {
 delete gpio;
 cout << "Exiting." << endl;
 exit(0);
}

//The main entry point for your program. This is the function that the OS (OSTRO) calls when it "runs" your code.
int main(int argc, char **argv) {

 signal(SIGINT, signal_handler); //This sets the event handler to the SIGINT signal. This is the signal generated by Ctrl+c.

 cout << "Hello from Intel on Joule! again" << endl //Remember, c++ isn't whitespace-sensitive, so you can use "carriage returns" (split lines) in the middle of function calls.
 << "Press Ctrl+c to exit..." << endl;    //This is very useful when the function call will be long and otherwise unwieldy.

 gpio = new mraa::Gpio(100); //Instantiate the GPIO object so that we can access the pin. (Pin 100)


 Mat image;

 cout << "going to test new stuff" << endl;

 VideoCapture capture;

 int c = 0;
 if(!capture.open(c))
  cout << "Capture from camera #" <<  c << " didn't work" << endl;
 else
  cout << "Capture from camera #" <<  c << " actually worked holy shit!" << endl;

 Mat frame;

 if( capture.isOpened() )
 {
  cout << "Video capturing has been started ..." << endl;

  for(;;)
  {
   capture >> frame;
   if( frame.empty() ){
    cout << "empty frame recieved?" << endl;
   }

   Mat frame1 = frame.clone();
   imwrite("coolFile.jpg",frame1);
   cout << "new frame saved" << endl;
   usleep(100);
  }
 }


 delete gpio; //Dispose of the object, now that we no longer need it. This is only needed because the object was allocated on the heap instead of the stack. (I.e. created with a pointer.)
 return 1; //As always, don't forget to return something. In this case, if the code reached this point, there was a problem, so we'll return 1 here.
}

Now you should try to build this, and you will be presented with the error

fatal error: cv.h: No such file or directory

And this makes perfect sense because we have not yet installed OpenCV. To do so, first we find the docker container that has been created with the eclipse workspace:

docker ps

this will show you information about the running container:

CONTAINER ID        IMAGE                                   COMMAND    
257c608539a3        inteliotdevkit/intel-iot-ostro:latest   "/bin/bash" 

note the container ID and proceed to launch a shell inside the container:

docker exec -i -t  /bin/bash

Now you should be inside a bash shell. The next step is to download a copy of Cmake and install it. We can get a copy at [5]. untar. bootstrap and make install:

cd /home/root/
wget https://cmake.org/files/v3.7/cmake-3.7.0-rc3.tar.gz
tar xzvf cmake-3.7.0-rc3.tar.gz
cd cmake-3.7.0-rc3
./bootstrap
make
make install

That's it now you have a fresh copy of cmake installed.

Now for OpenCV, you can get a download link for version 3.1 at [6]. This has to be "cmake"ed, "make"ed and "make install"ed:

cd /home/root
wget https://github.com/Itseez/opencv/archive/3.1.0.zip
unzip 3.1.0.zip
cd opencv-3.1.0/
mkdir build
cd build
cmake ..
make
make install

Although OpenCV is now installed on the docker container, the project will not yet build. We need to show eclipseCDT where it can find the include files and the lib files. In the "project explorer" window right click the project and choose properties. A dialog pops up, in the left pane of the dialog, open the "C/C++ Build" branch by click on the black arrow beside it then choose "settings". In the right pane on the top there will be a dropdown box, select "[all configurations]" from the dropdown. Now on the bottom right part of the top pane select "Includes" from under the "IoT Ostro 64-bit G++ compiler". Finally add the following line to the "include paths (-l)" sub pane:

"//${DOCKER_IMAGE}${DOCKER_SYSROOT}/usr/local/include/opencv/"

The following image should make things clearer:

If we try to build the project now it will compile but it will it will give you linker errors such as:

/usr/local/include/opencv2/core/cvstd.hpp:625: undefined reference to `cv::String::allocate(unsigned long)'

To also fix this we had to link with the OpenCV libraries. This is done in the same dialog as the before. In the left pane of the dialog choose "libraries" from ""IoT Ostro 64-bit G++ Linker". Now add the following libraries one by one to right top pane:

opencv_core
opencv_imgcodecs
opencv_highgui
opencv_imgproc
opencv_videoi

And that's it folks, your OpenCV project should build and get deployed to the device in question. As soon as it starts running, it will probe the USB and open the camera, if any. Then it will start saving images to the file named "/home/root/coolFile.jpg" on the device.
In the next post I will elaborate how we can compile and run caffe on the device.

[1] https://youtu.be/Qu5G443dQ4A
[2] https://github.com/BVLC/caffe
[3] https://software.intel.com/en-us/iot/hardware/joule
[4] https://software.intel.com/en-us/getting-started-on-joule
[5] https://cmake.org/download/ [6] http://opencv.org/downloads.html