How to use OpenCV with Raspberry Pi CSI camera


Making of

Raspberry Pi CSI onboard camera RAW image retrieval with OpenCV

You can actually do the coding on the Raspberry quite nicely, just activate ssh and install Code::Blocks. Then you can run the codeblocks over ssh from basically any linux machine. It runs quite well and makes coding easier. The package name is codeblocks and you can apt-get install it to raspberry.

pi@raspberry ~ $ sudo apt-get update 
pi@raspberry ~ $ sudo apt-get upgrade 
pi@raspberry ~ $ sudo apt-get install codeblocks 

CodeBlocks on Raspberry Pi image

First you need to make sure that your CSI onboard camera is working, images can be captured with raspistill, for more info please see Raspberry site.

The next step is to install OpenCV, we recommend taking the latest version 2.4.5. You need to manually compile and install that, the compilation will take time - the other steps are quite fast.

Get the latest OpenCV tar ball (opencv-2.4.5.tar.gz) from the Sourceforge.

You can also refer for more info about OpenCV installation on Raspberry Pi from mitchtech's site.

Next you should get the raspberry Pi userland code from GitHub.

pi@raspberry ~ $ git clone git:// 

You can also refer for more info about getting the userland code from GitHub from darkoperator's site.

Assuming everything is working so far, the rest of the steps are quite easy...

CodeBlocks on Raspberry Pi image

Here is a new needed cvproc.cpp file, which will get the RAW buffer from the raspistill.c code. You also need to add following libraries to your project (from /opt/vc/lib and /usr/local/lib/) in order to compile.

//Search Directories

//Needed Libraries

The cvproc.cpp is a separate file since raspistill.c contain a lot of C specific stuff that does not mingle with C++. It is almost self explanatory code.

// Copyright:               //
// VIILAAMO.COM             //
// Raspberry Pi RAW capture //

#include <vector>
#include <stdio.h>
#include <opencv2/opencv.hpp>

#define RAWBLOCKSIZE 6404096
#define HEADERSIZE 32768
#define SAVEJPEG 1

using namespace cv;
using namespace std;

unsigned char * rawbuf;
long bptr;

extern "C" void cvProcess(void)
  int i, j;
  long l = 0;


  #ifdef SAVEJPEG
  //save the jpeg fromVC4 for reference
  FILE *jout = fopen("raspiVC4.jpg", "wb");
  if (jout != NULL)
      fwrite(rawbuf, 1, bptr, jout);

  // Create a single plane matrix for Bayer data
  Mat R(1944, 2592, CV_16UC1);
  for (i = 0; i<1944;i++)
    for (j = 0; j<648; j++)
        // Copy the Bayer date into the Matrix<ushort>(i,(j*4))=(rawbuf[bptr+l]<<8)+(rawbuf[bptr+8+4]&0xc0);<ushort>(i,(j*4)+1)=(rawbuf[bptr+l+1]<<8)+(rawbuf[bptr+8+4]&0x30);<ushort>(i,(j*4)+2)=(rawbuf[bptr+l+2]<<8)+(rawbuf[bptr+8+4]&0x0c);<ushort>(i,(j*4)+3)=(rawbuf[bptr+l+3]<<8)+(rawbuf[bptr+8+4]&0x03);

  Mat C;
  // Do the demosaicing, i.e. generate missing colors
  cvtColor(R, C, CV_BayerRG2BGR);

  // Add your own processing here!

  // Convert to 8 bit for saving the jpeg
  C.convertTo(C, CV_8U, 1.0/256);

  std::vector<int> params;

  imwrite("raspiraw.jpg", C, params);
  //imwrite("raspiraw.png", C, params);

Only processing done is to generate missing colors (Bayer conversion to RGB) - you can add the stuff that you need to make a perfect image.

Oh Yes, here is the code.