Benchmarking OpenCV’s integration with Shogun

I have made some codes for converting Mat objects from OpenCV library to CDenseFeatures object in Shogun Library. There are basically three methods of conversion.

  • Using the for loops (i.e the manual way)
  • Using the Memcpy command.
  • Using constructors.

I have enlisted here the benchmarking output for the conversion of Random matrices from Mat to CDenseFeatures. We will try to draw major conclusions based on them in the end like to choose the fastest one as the default. All the tests are done on my Local machine which is a Ubuntu 12.04 Precise.

Let’s start with the conversion of Mat to CDenseFeatures.

For instance: a cv::Mat object initialized randomly in CV_8U type, and converted to the following types:

  1. To uint8_t
  2. To int8_t
  3. To uint16_t
  4. To int16_t
  5. To int32_t
  6. To float32_t
  7. To float64_t

These 7 conversion are shown through sky blued bars(Different Mat types are marked in the horizontal axis label at the bottom most region of each of the chart).

Similarly all the orange bars marks show the conversion from CV_8S to their respective conversion output types.

Similarly all the green bars marks show the conversion from CV_32F to their respective conversion output types and so on….




Few inference that we can draw directly are

1. Manual methods are consistent. i.e it doesn’t matter what the type of or on which other type are we converting our matrices into. It gives an approximately equal results in all of them.

2. Memcpy method is very very good with respect to the time taken. It is kind-off 2 times faster than the Manual method on smaller sized matrices especially for the types of uint8_t and int8_t.

For the back conversion, we will require the float CDenseFeatures object. And we will be converting It into different other types of Mat.



Few inferences that we can draw from these 3 figures are :

1. We see that in the constructor method, the conversion time for converting Float64_t CDenseFeature objects into CV_64F is almost negligible.

2. The order in which they can be arranged are Constructor> Memcpy < Manual.

2nd week report.

My work is primarily to develop a clean way to convert between OpenCV’s Mat object into our Shogun’s SGMatrix object. This would be really cool, once we have it as there will be a seamless interaction between both the libraries. Whenever we need machine learning capabilities, we can switch to Shogun and whenever we need the Image processing manipulations we can work on with OpenCV.


To develop a Factory such that if in its arguments I give a cv::Mat object’s name, It returns me with a DenseFeature object of shogun containing that matrix. The flexibility should be like that whatever type the input Mat might be (i.e uint8, uin16, float32,..etc), the user can choose what kind of DenseFeature he wanna work with. And yeah going vice versa will also be needed.

Things that are needed to be done:

We have 3 methods of converting data as suggested by Kevin in his gist here . Those are namely:

1. Using Constructors.

2. Using for loops (i.e doing it all manually).

3. Using Memcpy.

So, Using this I got a nice idea of how the main conversion would look like. Now comes the design part.

That means the functions declaration that we want would look something like this :

CDenseFeatures<T>* CV2FeaturesFactory::getDenseFeatures(cv::Mat, CV2SGOptions).

The 1st argument is to hold the name of the cv::Mat object that we want to transform.

The 2nd argument is to hold the name of the different options.

The template argument T would specify the type of the output CDenseFeatures that we want.

Initial Steps that came to my mind:

Now doing such a thing would require to define a template function in a non-templated class.

The non-templated class would look like :

class  CV2FeaturesFactory
template<class T> static CDenseFeatures<T>* CV2FeaturesFactory::getDenseFeatures(cv::Mat, CV2SGOptions);

We need to somehow extract few things from the cv::Mat before we start with the Kevin’s methods. Those are:

1. Number of Rows.

2. Number of Columns.

3. What type is cv::Mat object  to start with!!

4. I think we may even need to convert it into the required type before we feed it into the 3 methods mentioned at the start.

Setting It all up:

Most things are quite straight forward.

  • To calculate number of rows/cols for a obj named cvMat, we have :
int r=cvMat.rows;
int c=cvMat.cols;
  • No need to know the type of cvMat. We just need to convert whatever it is into the required type using the
cvMat.convertTo(cvMat, the output typename);
  • The only thing thats needed to be done here explicitly is to transfer the typename from the template argument (i.e <class T> in the function definition) to something understandable to OpenCV in its convertTo() function ‘s second argument.
  • I checked out the way OpenCV handles it’s Types, and found about the enums:
8-bit unsigned integer (uchar)
8-bit signed integer (schar)
16-bit unsigned integer (ushort)
16-bit signed integer (short)
32-bit signed integer (int)
32-bit floating-point number (float)
64-bit floating-point number (double)
enum { CV_8U=0, CV_8S=1, CV_16U=2, CV_16S=3, CV_32S=4, CV_32F=5, CV_64F=6 };

This was a great finding. What this means is that we only need to somehow convert the given T into the required integer. Say, if the user has specified ‘uint16_t’ as the ‘T’. Then we need to get integer value 2 into the arguments of convertTo() function.

Now, the thing is that I cannot possibly play with adding default enums for uint8_t, uint16t_t, float32_t …. . It is not allowed as I kept on getting the error as it being ‘redeclared’.

  • The only thing I could come up with was a templated function which returns the required integers depending on the template arguement T.  For eg:
int ans = get<uint8_t>() would set ans to 0;
int ans = get<float64_t>() would set ans to 6;

I searched and found USER SPECIALIZATION. It does exactly this thing.

By default, a template gives a single definition to be used for every template argument. This doesn’t always makes sense in the way that If I want my implementation to be different for different types say, for each of the 3 following types:

uint8, int16, float32 — I want to write three different implementations of the same named templated functions. This can be achieved using Specialization.

So, I came up with this :

the general template must be declared before any specialization.

“The critical information supplied by the general template is the set of template parameters that the user must supply to use it or any of its specializations.”

template<typename T> class CV2SGTypeName;
template<> struct CV2SGTypeName<uint8_t>{
 static const int get(){return 0;}
template<> struct CV2SGTypeName<int8_t>{
static const int get(){return 1;}
template<> struct CV2SGTypeName<uint16_t>{
static const int get(){return 2;}
} and so on

Here I have created few specializations which can be used as the common implementation for the user.

So now we have a function CV2SGTypeName<T>::get() which can be used very flexibly to return integers based on the template argument.


const int myType = CV2SGTypeName<T>::get();

I will add the source codes Once It gets ready to be merged.

First weekend after getting in GSoC!

This week turned out to be pretty amazing! Had been working and enjoying a lot with my mentor. He is a super-cool guy. Hope to meet him one day!

So till now, I basically scratched the surface of few things like:

  • CMake
  • C++
  • Design Patterns.

I have written 3 broad terms (that shows my amateurishness though !!) here. I would love to write off a detailed post on Design Pattern in C++ in future.

From whatever I have learned about C++, It appears out to be more than a language itself. You will really feel this if you have gone through the book of “The C++ Programming Language” by Bjarne Stroustrup, the creator itself. He cradles the language as a baby! and at times get emotional too. Moreover most people having expertise in C++ feel the same way.

This week gave me a chance to dig deep into Builder design pattern in C++. It had been a worthwhile experience to read more and more about C++. Sure!, It’s a bit tough to implement all by myself!. So I searched for examples and tried dummy coding it as much as I could do. This approach has so-far worked nicely.

Apart from coding, I devote 30 min of peace time with me, my shoes and asphalt ( Running it is!). It’s an awesome way to sweat out every bit of negativity. And yeah, spending some time with my neighbor (who is a GRE verbal teacher) to get my comprehension part strong!

So this is it! I will have few technical post following this.

The Life as it appears to be…

The last month proved quite terrific for me. I got selected in the Google Summer of Code 2014 for Shogun Machine Learning Toolbox. I hope to have a fun time this summers with lots of Computer Vision stuff and coding. It’s always good to flaunt with shiny things authored under your name :).

Sadly my End-Semester exams broke out in between. I knew, I didn’t had any preparation for the exams this whole semester ( I was pretty hooked up with pre-GSoC work!! ) and thats why I decided to shift my priorities. For the last 15-20 days, I have read and re-read my course again and gave all these exams.

These recent examinations gave me some further insights about me and the education system and that how unfit It is for me. My mind is right now surging with thoughts that I need to pour down. So here I begin:

We are a bunch of people here, who practically never study except with an examination nearby. We waste time all long, do queer things, discuss aspirations but we never study. As examination time comes, there’s some inner power that takes on most of us. The power which reflects the importance of grades that we will obtain over our Job, our masters schools and whatnot. We study like as if the hell broke out!. I have seen people in my lobby who have the gift of writing amazing answers and score a perfect 8.5 GPA (yeah! 8.5/10 is a pretty amazing GPA here!) with a meager study of a single day. To think of, just single day study for a single semester course! Thats terrific ability….or is it not?

Ah! I have fought many times with myself to understand, “why I am not like them?” . Why god didn’t give me this crucial ability to get over things really fast. To produce the best output with the minimum input. You don’t know how terrible it gets to have a person nearby working less heartily the whole semester, but scoring better! I blamed god. Sometimes I even tried working there way, knowingly it might not suit me. I wanted to be like them. To remain cool the whole semester and just strike in the end.

Under this utter chaos, God answered!. He didn’t answered me directly. He gave me slow and painful realizations. With each passing day, he made me realize the reason he created me. Firstly, he never wanted to create everybody the same. Life would be boring with everybody or most of them having the same skills and talents. He fed each one of us with what he could think best at the moment! But he seemed sad. When I reflected deep and asked him his sadness, he told me that we humans have thwarted his this effort. How come, “Is it our education system that you are talking about Sir?”. He replied “No, It is you that I found to be the culprit”. We blame our system and lots have been written over it already. My first implications over his reason of sadness for creating single headed automatons was the Education System created by us. But no, he said it’s me because of which he got sad. “Why Sir?”, I asked. “Because you are you. I created you different for some reason”, He replied.

I understood what he meant to say. He was taken aback the day I tried to follow other’s steps of scoring high. Knowingly that I am different and that I need time to get the hang of the subject, why did I try to complete it in one day when It is supposed to be worked out in the whole semester. Presumably, to appear cool right? Is this coolness. Well if it is, I am not cool with it. Is it wrong to be uncool with things that you have less control over?

This whole transaction resulted me in jotting down the conclusion:

“Studying 10 more days and scoring the same as those guys won’t make me uncool. It’s the result that matters. To maximize my result, I will work hard unconditional to how lesser work they need to do. It’s me! I am different! Lets keep it that way! If the system favors his talents, let’s bridge it with hard-work. Because thats what I have got full control over. ”



It’s 1:38 am here…I need a break. I am having my control system exam tomorrow and I haven’t started yet! Time to go to those neighbors of mine and take some help! 🙂