Archive for May, 2012

Microbo(t) – Advanced line follower

Although it is one of the classics of robotics for beginners, the line follower robot, made ​​at high level, is quite complex. On Youtube you can see line follower running at 3 m / s and more, with optimization and maximum performance research that achieves discrete levels.

Usually a line follower competition is based on pure speed and the path is quite linear and simple, with large curves and wide straights. In this case the hardware part is the most important, because the sw part is quite simple to build. it is important to find the right compromise between the mechanical parts and the engine, to make it running as fast as possible. Unfortunately requires a lot of money, because the engine performance search or the wheels with special tires can be very expensive.The alternative is based on paths that are rather difficult or very twisted, with right angles, intersecting lines, lines breaks, obstacles and so on. In this case the HW is much less important. More important is the algorithm and then the sw. The ability of the robot builder is to build a powerful algorithm that allows the robot to drive fast in every situation.

In this tutorial I would describe the robot that I have realized, that is able to face a tortuous path made ​​of curves at 180 ° rather than tight coils, trying to  have a good speed.
The robot ,according to tradition, was home built. It is also already equipped with an infrared sensor front in case you wish to use it in a path with obstacles.

Questo è Microbo(t) – line follower robot:

It ‘s built with the following elements:
– Chassis 5 mm PVC foam (few cents)
– Arduino Nano (about 40 euros)
– Drive Pololu 50:1 HP (about 30 euros both)
– Pololu motor bracket (both about 6 euros)
– Caster wheel  Pololu 3.8 “plastic (about 3 euros)
– Pololu Wheels 42×19 (both about 10 euros)
– Motor driver SparkFun TB6612FNG (about 9 euros)
– Sharp IR Distance sensor 4-30 cm (about 15 euros)
– Sensor line Pololu QT-8RC (about 12 euros)
– 7.4V 900mAh Battery Li-Po (about 9 euros)
– some screws and wires
As you can see the components expensive are Arduino Nano and the motors. Nothing forbids to use a traditional Arduino with an obvious saving of money. In my configuration it weighs about 160 grams. You can do much better, in fact on the internet there are line follower with Arduino that weigh 100 grams.



Some comments on hw components used. The driving system, formed by Pololu wheels and  motors is almost good. I think that even better is using HP 100:1 Pololu motors which provide even more traction and fluidity in the changes of direction, even if it has a lower number of revolutions/minute. I tested also the 30:1 motors, but they were too nervous. Absolutely bad is the plastic wheel caster form Pololu. Unfortunately I had no other small than this, so I kept it. Not recommend its use. Terrible.

The motor driver SparkFun TB6612FNG is a little undersized. In fact it provides 1.2A at full speed, when the Pololu HP motors  may require up to 1.6A. But I had no problem with these drivers and they still work, so I can only advise them. Unfortunately they have a lot of wires, 7 for driving, 2 for the Vcc voltage (+5 V), one (or two with the ground) for the motor voltage and 4 wires for motors. In total 14 wires that are several. They are simple to program and manage. The full brake function is not good. In fact, if you need to brake, however, takes a little bit to stop, victim of inertia. I think that the engines 100:1 can help on this.

The line sensor Pololu QT-8RC is good. It provides out-of-the-box a digital output and can be connected directly to the Arduino digital ports. After a stage of self-learning during the first few seconds (see the video), it is able to calibrate with the best values ​​taking into account the reflectivity of the surface and the ambient light conditions. It can follow the black lines on white background or white on black background. For this sensor there is an Arduino  library written by Pololu that provides a value between 0 and 8000 depending on the sensor (from left to right) which is above the black line. I modified the library to get a response that goes from -800 to +800 with zero when the line is well below the IR sensor in the middle. This from my point of view makes easier the control. The only real problem I had was in the presence of right angles. In this case the robot has not always stopped in time and managed to turn. Unfortunately I have no videos on this particular problem.

Once built the robot, the biggest difficulty is the control parameters determination, that allow the robot to come back to the line when goes away. In this case I have put in place the usual PID control, although in my version was reduced only in PD, proportional and derivative. The integral control, with the frequent changes of direction, is extremely difficult to detect.
I also wrote  some simple routines for the detection of right angles. In this case the robot tries to stop and turns 90°. This technique has worked quite, but you can sure do better.

For the  IR front sensor I didn’t have any important test for obstacle avoidance. The main problem is due to the need to rotate the robot for the obstacle avoidance operating in open-chain, ie, without feedback.  The robot should pass the obstacle, but that succeeded a few times because the motors battery voltage varies over time, with a variation of path. I did not do further tests, but there are two possibilities:

  • a dc-dc converter boost 4-25v ​​for example like in the Pololu 3pi robot that maintains a constant input voltage to the motors and that have a good precision even in the absence of encoder
  • the encoders in the wheels with angles measurement

The dc-dc converter costs less than 10 euros, while the cost for the Pololu encoder  is about 25 euros. Maybe in the next version of the Microbo(t) I will decide which solution to adopt. Clearly the solution encoder is more sophisticated but requires more programming effort (and verify that there are still pins available on the Arduino!).

This is the code for Arduino, commented but not very clean. Link

This is the Microbo(t) in action:


Tags: , ,

Processing and Opencv 2.3.1 – Contours detection and polynomial approximation

In the last episode of this short introduction to Opencv and Processing I want to show an algorithm that allows  to appreciate how with the libraries JavaCvPro you can use the OpenCV to implement an algorithm more sophisticated than the basic ones that I have proposed till now.

In this example I make the image edge detection. Then the contours detection is obtained and hence the contours are approximated by polynomial curve. In particular, the polynomial is a set of points that can be represented for example with the segments.
The code is organized as a function java but it is simple to adapt for different uses.



// -- opencv linearization variables
int iterazione=0;
int cc=-1;
int [][] E;
// ********************************************************
// OpenCV - Processing with JavaCvPro
// Edge detection - contour detection - polynomial approximation
// ********************************************************
Pimage goFilterCV(PImage draft) {

  E = new int[ow*oh][3]; // matrix for vertex list - weigth * height

  opencv.allocate(ow, oh); //allocate space for image in opencv

  //-- copy draft image to opencv IplImage with javacvpro library
  opencv_core.IplImage opencvImgSrc=opencv.fromPImage(draft);  // copy draft --> IplImage
  opencv_core.CvSize mySize=opencvImgSrc.cvSize();             // take the size of IplImage

  opencv_core.IplImage opencvImgDest= opencv_core.cvCreateImage(mySize, opencv_core.IPL_DEPTH_8U, 3); // build an image IplImage , 3 channels

  //--- bilateral filter effect
  for (int i=0; i<10; i++) {
    opencv_imgproc.bilateralFilter(opencvImgSrc, opencvImgDest, 3, 20.0, 50.0, 0 ); // applique un effet Flou gaussien 
    opencv_core.cvCopy(opencvImgDest, opencvImgSrc);
  filter1 = createImage(ow,oh,RGB);

  //-- define IplImage
  opencv_core.IplImage iplImgGray;
  opencv_core.IplImage iplImgGray1;
  iplImgGray = opencv_core.cvCreateImage(mySize, opencv_core.IPL_DEPTH_8U, 1);   // build an image IplImage , 1 channels - only gray
  iplImgGray1= opencv_core.cvCreateImage(mySize, opencv_core.IPL_DEPTH_8U, 1);   // build an image IplImage , 1 channels - only gray

  //-- transform colors in gray levels
  opencv_imgproc.cvCvtColor(opencvImgSrc, iplImgGray, opencv_imgproc.CV_RGB2GRAY);

  //-- edge detection - canny algo
  opencv_imgproc.cvCanny(iplImgGray, iplImgGray1, 100.0, 130.0, 3 );
  //opencv_imgproc.cvDilate(iplImgGray1, iplImgGray1, null , 1); // in alternative you can use the dilate function to better find the edges

  //-- contour detection
  opencv_core.CvMemStorage mem = opencv_core.CvMemStorage.create();
  opencv_core.CvSeq contour = new opencv_core.CvSeq(null);
  int total=opencv_imgproc.cvFindContours(iplImgGray1, mem, contour, Loader.sizeof(opencv_core.CvContour.class), opencv_imgproc.CV_RETR_LIST, opencv_imgproc.CV_CHAIN_APPROX_NONE);
  println("total point cvFindContours:"+total);

  //-- polynomial approximation
  while (contour != null && !contour.isNull ()) {
    iterazione++; //number of interactions
    opencv_core.CvSeq poly = null;
    if (contour.elem_size() > 0) {
      poly = opencv_imgproc.cvApproxPoly(contour, Loader.sizeof(opencv_core.CvContour.class), mem, opencv_imgproc.CV_POLY_APPROX_DP, 3, -1);

      //transfer to matrix
      int n =;
      opencv_core.CvPoint points = new opencv_core.CvPoint(n);
      opencv_core.cvCvtSeqToArray(poly, points.position(0), opencv_core.CV_WHOLE_SEQ);
      for (int i = 0; i < n; i++) {
        opencv_core.CvPoint pt = points.position(i);
        int x = pt.x();
        int y = pt.y();
        // println("pts:"+i+" x:"+x+" y:"+y);
    contour = contour.h_next();
  } //end polynomial approx

 // -- print some values for debugging purposes
 // for (int i = 0; i < 100; i++) //contatore to have all the points
 // println(E[i][0]+" "+E[i][1]+" "+E[i][2]);

  draft=toPImage(iplImgGray1); // return draft - PImage
  return (draft);


// ********************************************************
// take an IplImage and give a PImage
// ********************************************************
PImage toPImage (opencv_core.IplImage iplImgIn) { // take an IplImage and give a PImage
  //--- put the IplImage in a bufferedImage
  BufferedImage bufImg=iplImgIn.getBufferedImage(); 

  //---- build a PImage with the same size of IplImage--- 
  PImage imgOut = createImage(iplImgIn.width(), iplImgIn.height(), RGB);

  // put the pixel of IplImage to imgOut.pixels of PImage
  bufImg.getRGB(0, 0, iplImgIn.width(), iplImgIn.height(), imgOut.pixels, 0, iplImgIn.width()); 

  imgOut.updatePixels(); // PImage update

  return(imgOut); // return the PImage

Processing and Opencv 2.3.1 – Using JavaCvPro

To use the OpenCV 2.3.1 in Processing  the shortest and easiest way is to use the JavaCvPro by X.Hinault.
The library JavaCVPro allows to use the OpenCV primitives in a very simple and immediate.
To use the webcam, it requires the library GSVideo  that can be installed in Processing / modes / java / libraries (for Processing 1.5).

Here’s an example of JavaCVPro usage:

// Programme d'exemple de la librairie javacvPro
// par X. HINAULT - octobre 2011
// Tous droits réservés - Licence GPLv3

// Exemple fonction contrast()

import monclubelec.javacvPro.*; // importe la librairie javacvPro

PImage img;

String url=""; // String contenant l'adresse internet de l'image à utiliser

OpenCV opencv; // déclare un objet OpenCV principal

void setup(){ // fonction d'initialisation exécutée 1 fois au démarrage

        //-- charge image utilisée --- 
        img=loadImage(url,"jpg"); // crée un PImage contenant le fichier à partir adresse web

        //--- initialise OpenCV ---
        opencv = new OpenCV(this); // initialise objet OpenCV à partir du parent This
        opencv.allocate(img.width, img.height); // initialise les buffers OpenCv à la taille de l'image

        opencv.copy(img); // charge le PImage dans le buffer OpenCV

        //--- initialise fenêtre Processing 
        size (opencv.width()*2, opencv.height()); // crée une fenêtre Processing de la 2xtaille du buffer principal OpenCV
        //size (img.width, img.height); // aalternative en se basant sur l'image d'origine

        //--- affiche image de départ --- 
        image(opencv.getBuffer(),0,0); // affiche le buffer principal OpenCV dans la fenêtre Processing

        //--- opérations sur image ---
        opencv.contrast(+50); // applique réglage contraste sur le buffer principal OpenCV

        //--- affiche image finale --- 
        image(opencv.getBuffer(),opencv.width(),0); // affiche le buffer principal OpenCV dans la fenêtre Processing

       noLoop(); // stop programme 

void  draw() { // fonction exécutée en boucle


The code is very simple. All the opencv instructions are called simply using ‘opencv.’ before the real instruction. You can use the opencv instructions translated in Processing by X. Hinault in the JavaCvPro libraries.

But it is very interesting to mix OpenCv instruction supported by JavaCvPro libraries and native OpenCv instructions. The JavaCvPro allow to use also the native OpenCV instruction. This is a powerful tool!!!

In the example below, only native
OpenCV instructions are used.

// Example of using native OpenCVPro instruction in Processing
// example by X.Hinault modified
// Licence GPLv3

// Bilinear filter example

import com.googlecode.javacv.*;
import com.googlecode.javacv.cpp.*;
import monclubelec.javacvPro.*;
import java.awt.image.BufferedImage;
PImage imgDest, OrigImg;
void setup() {
  size(640, 240);
  String cheminFichier="C:/Users/Nane/Dropbox/Sorgenti Arduino/Jarkman/Foto/gioconda1.jpg";; 
  //---- appel direct des fonctions de la librairie javacv ---- 
  //--- chargement d'un fichier image 
  opencv_core.IplImage opencvImgSrc= opencv_highgui.cvLoadImage(cheminFichier);

  opencv_core.CvSize mySize=opencvImgSrc.cvSize(); // récupère la taille de l'image 

  opencv_core.IplImage opencvImgDest= opencv_core.cvCreateImage(mySize, opencv_core.IPL_DEPTH_8U, 3); // crée une image IplImage , 3 canaux

  //--- application d'effet opencv ---
 for (int i=0; i<20; i++)
    opencv_imgproc.bilateralFilter(opencvImgSrc, opencvImgDest, 3,120.0, 190.00,1 ); // applique un effet Flou gaussien 

//============ récupérer une image openCV dans Processing ===== 
  //--- récupérer l'objet IplImage dans un BufferedImage 
  BufferedImage bufImg=opencvImgDest.getBufferedImage(); // récupère l'image

  //---- créer un PImage --- 
  PImage img = createImage(320,240, PConstants.RGB); 

  // charge les pixels de l'image buffer dans img.pixels
  bufImg.getRGB(0, 0, 320, 240, img.pixels, 0, 320); 
  image(img,320,0); // affiche l'image
void draw() {


In this case the OpenCv instructions for Processing are used only in the last part of the code, in order to show the images. The coding of the native OpenCV instructions is more difficult, but very very powerful. It is possible to use almost the whole set of instructions offered by the OpenCv 2.3.1.