How to use OpenCV with Raspberry Pi CSI camera
by viilaamo.com
Making of
Raspberry Pi CSI onboard camera RAW image retrieval with OpenCV
You can actually do the coding on the Raspberry quite nicely, just activate ssh and install Code::Blocks. Then you can run the codeblocks over ssh from basically any linux machine. It runs quite well and makes coding easier. The package name is codeblocks and you can apt-get install it to raspberry.
pi@raspberry ~ $ sudo apt-get update pi@raspberry ~ $ sudo apt-get upgrade pi@raspberry ~ $ sudo apt-get install codeblocks
First you need to make sure that your CSI onboard camera is working, images can be captured with raspistill, for more info please see Raspberry Pi.org site.
The next step is to install OpenCV, we recommend taking the latest version 2.4.5. You need to manually compile and install that, the compilation will take time - the other steps are quite fast.
Get the latest OpenCV tar ball (opencv-2.4.5.tar.gz) from the Sourceforge.
You can also refer for more info about OpenCV installation on Raspberry Pi from mitchtech's site.
Next you should get the raspberry Pi userland code from GitHub.
pi@raspberry ~ $ git clone git://github.com/raspberrypi/userland.git
You can also refer for more info about getting the userland code from GitHub from darkoperator's site.
Assuming everything is working so far, the rest of the steps are quite easy...
Here is a new needed cvproc.cpp file, which will get the RAW buffer from the raspistill.c code. You also need to add following libraries to your project (from /opt/vc/lib and /usr/local/lib/) in order to compile.
//Search Directories /opt/vc/userland-master/host_applications/linux/libs/bcm_host/include /opt/vc/userland-master/interface/vcos /opt/vc/userland-master /opt/vc/userland-master/interface/vcos/pthreads /opt/vc/userland-master/interface/vmcs_host/linux /usr/local/include/opencv /usr/local/include //Needed Libraries /opt/vc/lib/libmmal_core.so /opt/vc/lib/libmmal_util.so /opt/vc/lib/libmmal_vc_client.so /opt/vc/lib/libvcos.so /opt/vc/lib/libbcm_host.so /usr/local/lib/libopencv_core.so /usr/local/lib/libopencv_highgui.so /usr/local/lib/libopencv_imgproc.so
The cvproc.cpp is a separate file since raspistill.c contain a lot of C specific stuff that does not mingle with C++. It is almost self explanatory code.
//**************************// // Copyright: // // VIILAAMO.COM // // Raspberry Pi RAW capture // //**************************// #include <vector> #include <stdio.h> #include <opencv2/opencv.hpp> #define RAWBLOCKSIZE 6404096 #define HEADERSIZE 32768 #define SAVEJPEG 1 using namespace cv; using namespace std; unsigned char * rawbuf; long bptr; extern "C" void cvProcess(void) { int i, j; long l = 0; bptr=bptr-RAWBLOCKSIZE+HEADERSIZE; #ifdef SAVEJPEG //save the jpeg fromVC4 for reference FILE *jout = fopen("raspiVC4.jpg", "wb"); if (jout != NULL) { fwrite(rawbuf, 1, bptr, jout); fclose(jout); } #endif // Create a single plane matrix for Bayer data Mat R(1944, 2592, CV_16UC1); for (i = 0; i<1944;i++) { for (j = 0; j<648; j++) { // Copy the Bayer date into the Matrix R.at<ushort>(i,(j*4))=(rawbuf[bptr+l]<<8)+(rawbuf[bptr+8+4]&0xc0); R.at<ushort>(i,(j*4)+1)=(rawbuf[bptr+l+1]<<8)+(rawbuf[bptr+8+4]&0x30); R.at<ushort>(i,(j*4)+2)=(rawbuf[bptr+l+2]<<8)+(rawbuf[bptr+8+4]&0x0c); R.at<ushort>(i,(j*4)+3)=(rawbuf[bptr+l+3]<<8)+(rawbuf[bptr+8+4]&0x03); l+=5; } l+=24; } free(rawbuf); Mat C; // Do the demosaicing, i.e. generate missing colors cvtColor(R, C, CV_BayerRG2BGR); // Add your own processing here! // Convert to 8 bit for saving the jpeg C.convertTo(C, CV_8U, 1.0/256); std::vector<int> params; params.push_back(CV_IMWRITE_JPEG_QUALITY); params.push_back(92); imwrite("raspiraw.jpg", C, params); //imwrite("raspiraw.png", C, params); return; }
Only processing done is to generate missing colors (Bayer conversion to RGB) - you can add the stuff that you need to make a perfect image.
Oh Yes, here is the code.