Today is a good day!

Introduction

Today is a good day! I finally managed to get OpenCV to work with groovy. Finally, because it took me almost a week to get everything installed on a dual boot ubuntu 12.04 + Windows 7 laptop (see this post).

My laptop is a HP EliteBook 8760w, which has a NVIDIA Quadro 3000M  in it. That card is roughly equivalent to a Tesla computing card. It also makes your laptop more like a "schlepptop" as it weighs several kilos and looks like a stack of bricks.

I convinced management to get me this box, so that I can code GPUs "on the road". At work, we have a couple of Dell Precision workstations with each a Tesla card in them. Nice machines, but not easy to carry along. We have been dabbling in CUDA for 3 years now, demonstrating how their performance matches our increasing volume of image data we use in our daily work. CUDA kernel programming and, especially, optimisation is a pain, however. Most of our folks are not IT specialists, but rather domain experts in image use.

Our business is mostly in satellite and airborne image processing. One of my personal missions is to demonstrate processing solutions with new approaches that one can tool together with Open Source (OS) software tools. With satellite imagery like Landsat-8 now being made available under "free and open access" licensing schemes, and the similarly licensed European Sentinel data streams to start flowing next year, the time is ripe to put those capacities in "the hands of the masses", including the scale step in processing speed that is needed to handle these potentially massive data flows.

I am also a Java fan and, even more, a Groovy fan. So, one of my user requirements for a viable OS library is that it should have, at least, some wrappers for Java. Normally, if you can do Java, you can do Groovy. I am not a Java or Groovy expert either, but I can normally resolve most issues via debugging and on-line discovery, My single claim to coding fame is that I once repaired a bug in the TIFF reader of Java Advanced Imaging.

The best of all worlds, given the above, appears to be the  combination of OpenCV with it GPU enabled routine set and its Java bindings. OpenCV is a mature set of computer vision routines, many of which cover the basic functionality that we require in satellite image processing. The GPU enabled routines are supposedly hiding the complexity of the CUDA programming, while unleashing the raw power of these, low cost, computing hardware solutions and the Java binding matches all that with the fairly flat Groovy learning curve. So, here's my version of the 4th paradigm: GPU + OpenCV + Groovy, and tomorrow's satellite Big Data problem is your uncle. 

But, before we get carried away, let's look at the nitty gritty details.First we have to get Groovy to work with OpenCV (this post). In the next post, I'll demonstrate how this works with CPU- and GPU-versions of some of the more interesting OpenCV routines.

Setting up OpenCV with Java binding

So, when the ubuntu 12.04 was finally installed and stable (with all the dependencies resolved), I followed the OpenCV installation as suggested in the
Introduction to Java Development document.

That document suggests to use either an ant, Eclipse or sdt build environment to develop the code. The choice of those environments are definitely recommendable for more complex coding frameworks, but one of my base assumptions is that image processing code is typically a series of simple sequential workflow steps, if needed, organised in modules. Thus, simple command-line based compile, debug and run should be possible. In fact, it is as easy as:

$ javac -cp ../build/bin/opencv-245.jar:. SimpleSample.java
$ java -cp ../build/bin/opencv-245.jar:. -Djava.library.path=../build/lib SimpleSample

which gives you the expected output as:

Welcome to OpenCV 2.4.5.0

OpenCV Mat: Mat [ 5*10*CV_8UC1, isCont=true, isSubmat=false, nativeObj=0x7f5bf02b2d80, dataAddr=0x7f5bf02b2e40 ]

OpenCV Mat data:

[0, 0, 0, 0, 0, 5, 0, 0, 0, 0;

  1, 1, 1, 1, 1, 5, 1, 1, 1, 1;

  0, 0, 0, 0, 0, 5, 0, 0, 0, 0;

  0, 0, 0, 0, 0, 5, 0, 0, 0, 0;

  0, 0, 0, 0, 0, 5, 0, 0, 0, 0]

Groovify that!

So far, so good. The next step is to groovify the java code as follows:

import org.opencv.core.Core
import org.opencv.core.Mat
import org.opencv.core.CvType
import org.opencv.core.Scalar

LibLoader.load()
println("Welcome to OpenCV " + Core.VERSION)
def m = new Mat(5, 10, CvType.CV_8UC1, new Scalar(0))
println("OpenCV Mat: " + m)
def mr1 = m.row(1)
mr1.setTo(new Scalar(1))
def mc5 = m.col(5)
mc5.setTo(new Scalar(5))
println("OpenCV Mat data:\n" + m.dump())

Beyond the abolition of semi-colons at the end of the line, and the use of the def dynamic typing construct, this may not seem too spectacular, but the merit of groovy is expected later, in some more complex coding artefacts.

There are 2 points to make here: first, the static block in the Java code has been replaced with a call to a static method of the java class LibLoader:

import org.opencv.core.Core;

public class LibLoader {
  public static void load() {
    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
  }
}

which is compiled as:

javac -cp ../build/bin/opencv-245.jar:. LibLoader.java

This may seem a bit obsolete, but I have not found a way to get around this. There is apparently something fishy with calling static library functions in Groovy. Why this is not a problem in Java beats me. In any case, since the LibLoader is needed in all future scripts, multiple re-use is guaranteed. As soon as the groovy issue is resolved, the Java class can be replaced by a functional groovy equivalent (which should be a one-liner as well).


Second, if you try to run this as a groovy script, you will get a java.lang.UnsatisfiedLinkError which is thrown by the Mat statement (the first OpenCV function). This relates to the same issue, i.e. while the LibLoader appears to work, the actual function call can't find the entry point in the library (my explanation). The work-around solution to this problem is to compile first and then run the compiled class as follows:

groovyc -cp ../build/bin/opencv-245.jar:. TestGroovy.groovy

java -cp ../build/bin/opencv-245.jar:/usr/local/groovy/embeddable/groovy-all-2.1.4.jar:.
 -Djava.library.path=../build/lib TestGroovy

which will give you the expected results.

The issue was already filed as a bug request on the OpenCV Q&A forum.

Conclusions

 

Even if not perfect, I can now start looking into some more serious OpenCV/GPU/Groovy coding, which is more or less where I wanted to be last week...

As a little bonus, here is the DetectFaceDemo example in groovy that comes with the OpenCV java example code. It contains some code simplifications available in groovy (e.g. printf, list iterator in a loop)

import org.opencv.core.Core;
import org.opencv.core.Mat
import org.opencv.core.MatOfRect
import org.opencv.core.Point
import org.opencv.core.Rect
import org.opencv.core.Scalar
import org.opencv.highgui.Highgui
import org.opencv.objdetect.CascadeClassifier
/*
 * Detects faces in an image, draws boxes around them, and writes the results
 * to "faceDetection.png".
 */

LibLoader.load()

println("\nRunning DetectFaceDemo")

// Create a face detector from the cascade file in the resources directory.
// Note: I took a little shortcut here: I got rid of the getClass.getResource.getPath
// construction (I don't like the idea that my data should reside in the code directory)
def faceDetector = new CascadeClassifier("resources/lbpcascade_frontalface.xml")
def image = Highgui.imread("resources/NotSoAverageMaleFace.jpg")
        
// Detect faces in the image.
// MatOfRect is a special container class for Rect.
def faceDetections = new MatOfRect()
faceDetector.detectMultiScale(image, faceDetections) 

// Note: use printf rather than println(String.format... 
printf("Detected %d faces%n", faceDetections.toArray().length)

// Draw a bounding box around each face.
// Note: I replaced the object array loop in Java with a Groovy object list iterator
faceDetections.toList().each() { rect ->
  Core.rectangle(image, new Point(rect.x, rect.y), new Point(rect.x
      + rect.width, rect.y + rect.height), new Scalar(0, 255, 0))
}
// Save the visualized detection.
def filename = "faceDetection.png"
printf("Writing %s%n", filename)
Highgui.imwrite(filename, image)

Which works as expected as well...

QED! Time for a celebratory beer!

(to be continued...)