I know many people will look at me in frowning eyes after reading the title. Yes, OpenCV is an image processing and computer vision library. But believe me, the most rudimentary works you need to do in these fields are essentially “Linear Algebra”. Think about an image — it is a matrix; and most of the time you’ll work on matrices. Even other data structures like vectors and points — all are some kind of matrices. Necessity of doing linear algebra is one of the main reasons for burgeoning MATLAB (which is an acronym for MATrix LABoratory) libraries for image processing. In fact, the power of linear algebra libraries enabled MATLAB to incorporate so many different toolboxes ranging from image processing, statistics, bioinformatics, neural network to so many more. So, let us do some kind of matrix processing in OpenCV. We’ll use the C++ interface.
First thing that you need to do to work with opencv is to install it. If you are using Windows and Microsoft Visual Studio, I have a set of instructions to install it. However, if you are using Linux, Mac, Android or any other OS, you need to look for the instructions in OpenCV Wiki. Once you’ve successfully installed it, let’s jump into the coding part.
From OpenCV 2, it is possible to use the C++ coding style in OpenCV. It has different modules focused on different kinds of works. For example, “core” is mainly devote to the core processing like forming the matrix, doing some linear algebra etc. The “highgui” module deals with GUI related works such as creating windows etc. “ml” works with machine learning procedures. If you work with computer vision, you will eventually see that it is heavily dependent on the machine learning field. Anyway, the header files for different components of OpenCV are located in <OpenCV>includeopencv2<ModuleName> folder. If you don’t want to bother to know which function belongs to which module, you can include only one header “opencv.hpp” which includes all the other header files. So, all the parts of OpenCV is added in your project.
Lets first see some basic input-output of a matrix. Write the following code in your MSVC project which you should configure according to the instructions specified here.
[code lang=”cpp”]
#include <opencv2/opencv.hpp>;
#include <iostream>;
int main(int argx, char** argv){
float trainingData[3][3] = { {1, 2, 3}, {4, 5, 6}, {7, 8, 9}};
cv::Mat MyFirstMat(3, 3, CV_32FC1, trainingData);
std::cout<<MyFirstMat;
std::cout<<"nPress Enter to exitn";
std::cin.get();
}
[/code]
When you’ll run this small program, you’ll see a matrix in the output screen. The entries in each row are separated by a comma. After every row, there is a semicolon. Actually, this is a way of writing matrices in many other languages including MATLAB. Look at lines 4 and 5 very carefully. In 4, you are defining a 3×3 array where you’re loading the entries of the matrix. Line number 5 is the main OpenCV command where you are creating an OpenCV matrix named “MyFirstMat”. Look at the constructor arguments carefully. The first two arguments indicate that the matrix contains 3 rows and 3 columns. The third argument refers to the type of the matrix. There is an OpenCV standard for specification of the types of OpenCV matrices. After the first common “CV” part, the number 32 means it is a 32 bit number (i.e. each entry in the matrix will be constituted with 32 bits). The “F” means it is a “float” type (i.e. contains decimal point). Some other common type specifiers are “64F”, “32S”, and “8U” representing “double”, “int”, and “uchar” respectively. In the last part, C1 means it has only one channel. In other words, you can write only one number in each entry. If it would be C3 then you could write 3 numbers. For example, in an RGB image, for each pixel you need to write 3 numbers for red, green and blue channel. In the last argument, we just fed the 2D array into the matrix. Notice that we specifically mentioned the scope using a scope resolution operator “:”. This is because, many functions in OpenCV matches with the functions of standard library “std”. So, it is a good practice to specify the scopes in every case.
Now let us do an experiment. Replace line number 4 by this line:
[code lang=”cpp”]float trainingData[9] = { 1, 2, 3, 4, 5, 6, 7, 8, 9}[/code]
It is giving the same result, right? This is because, when the data is store into a matrix, it is stored as just by writing the bits one after another. It is the responsibility of the first three arguments (number of rows, number of columns and the datatype) to interpret the bit sequence as a human understandable form. Unlike MATLAB, OpenCV vectorizes a matrix row-wise first.
Now let us see some other examples of matrix input. The following code will also do the same as previous example:
[code lang=”cpp”]
Mat cameraMatrix = (Mat<double>(3,3) << 1000, 0, 320, 0,
1000, 240 , 0, 0, 1);
[/code]
Sometimes, it is necessary to dynamically input the data after initializing the headers. This can be done as below.
[code lang=”cpp”]
int main(int argx, char** argv){
cv::Mat MyMat = cv::Mat::zeros(3,3,CV_64FC1);
for(int i=0;i<3;i++)
for (int j = 0;j<3;j++)
MyMat.at<double>(i,j)=j+i*3+1;
std::cout<<MyMat;
std::cout<<"nPress Enter to exit";
std::cin.get();
}
[/code]
It gives the same output as earlier example. But the difference is, this time you don’t need to specify all the values in coding time. The matrix entries gets assigned in run-time (i.e. dynamically) by evaluating a mathematical function. Notice, in this code, the matrix is initialized with zeros using a static function named “zeros” (all elements will be zero). You can also initialize it with the “ones” (all elements will be one), or the “eye” (identity matrix). No matter how the initialization is done, the two for loops are actually replacing each element with the value of j+i*3+1. The “at” in line 5 is a template for accessing the elements of the matrix. Since it is a template, you need to specify the data type. This method can be used to both input and output of a matrix element. You need to be careful in deciding whether you are putting (i,j) or (j,i) as the arguments of the “at” function. Since OpenCV assigns the values of a row first, the “for” loop defining the column variable (j in this case) should immediately precede the line of at function. But don’t worry much, if you get the matrix in wrong orientation, you can always transpose it using the t() function. You can test it by inserting the following line in between lines # and # in the previous code .
[code lang=”cpp”] MyMat.t();[/code]
You can use the following code to load a matrix from a csv file.
[code lang=”cpp”]
CvMLData Data;
Data.read_csv("test.csv");
csvContent = Data.get_values();
[/code]
Any OpenCV structure can be saved in xml/yml file. If you are interested to do so, you can see here. I’ll not discuss it here.
Now, you know the basic methods to load a matrix and show it in screen. You can also do element by element operation since you can access matrix elements. However, there are several convenience function. For example, a matrix can be displayed in an output stream (or file-stream) using the << operator. In next post, we’ll see some basic operations in opencv. We’ll also see some linear algebra operations.