Thermal Camera

Discussion Projects
1

I got the MLX90640 thermal camera working on a Pi today. On a pi zero also!

Example software is a bit rough and ready - I had to edit away a couple of errors - and the camera needs powering off when changing modes. I guess the software needs to reset state better. A good starting point I think.

I had a problem with using alongside a pijuice and display. They must conflict somehow. A clean install of Raspbian without the other hats sorted it.

Anyhow:

ThermalThumbsUp

1 Like
2

Itā€™s a very, very finickity camera! Iā€™m hoping to get software support to the point where itā€™s more widely available to beginners but itā€™s been an uphill struggle.

The big issue currently is that it uses the bcm2836s libraries i2c access which, irc, accesses the i2c registers directly and causes some serious problems with the kernel level i2c drivers. The two appear to be mutually exclusive, which is a pain :(

3

Iā€™m a beginner and got it working, so you must be doing something right! Any pointers in how to measure temp or which file I use to change the output size would be awesomeā€¦

4

In, for example, fbuf.cpp (I really need to pick better filenames for these examples!) these are the lines that display the temperature data on the screen:

At this point the value set to val is the temperature in degrees C of that particular pixel.

If you want to get the temperature at the centre of the sensor you would wait for y to be 11 or 12 and x to be 15 or 16 and measure that pixel, or perhaps take the average of these four.

If you want to measure at a specific point you could reverse engineer the loop into a function to get a pixel at a particular x/y:

float  getPixelAt(int x, int y, float *pixelData){
    return pixelData[32 * (23-y) + x];
}

(This may not work because, and this may come as a surprise to the untrained eye, I suck at C/C++)

As for displaying this value, thatā€™s another matter. With the framebuffer value Iā€™m doing a quick and dirty push into /dev/fb0 which is a truly horrible way to draw graphics but gets the job done. Thereā€™s no easy way to draw a text string without pixelling it in from a pixel font map. That said you only need the characters 0-9, ā€œ.ā€ and ā€œĀ°Cā€.

fbuf.c and interp.c (horrible, horrible filenames) both have an IMAGE_SCALE value which handles basic upscaling by redrawing the pixels as IMAGE_SCALE pixel wide blocks.

interp.c additionally has OUTPUT_W and OUTPUT_H which determine how much cubic interpolation should be applied to the image before it is draw.

These values can slow down output immensely, though, so donā€™t drive them too far! You can also use fbset on the Terminal to reduce your framebuffer resolution so that smaller images are stretched across your screen at no additional processing cost.

1 Like
5

This would be really useful if I could get something to continually return the ā€˜temperatureā€™ of the pixels! Will have a play!

6

did someone tried to use this camera with OpenCV? I want to buy such a camera, the 110Ā° FOV as soon it becomes available. My goal is to track persons movement direction and counting.

7

I did wonder if using something like v4l2loopback to create a virtual v4l2 device would make it accessible to opencv, plus other standard camera applications and tools.

8

not a bad ideea at all, but first we need to get the device itself, i observed that is not on stock anymore, i need the 110Ā° FOV version and no one knows when it will be available again

9

This code will allow the MLX90640 to be used with OpenCV. OpenCV is installed on a Raspberry Pi 3. I used gadgetoidā€™s fbuf code as a foundation with the assistance of StackOverflow.

#include <stdint.h>
#include <iostream>
#include <cstring>
#include <fstream>
#include <chrono>
#include <thread>
#include <math.h>
#include "headers/MLX90640_API.h"

#include "opencv2/opencv.hpp"

using namespace cv;
using namespace std;

#define MLX_I2C_ADDR 0x33

// Valid frame rates are 1, 2, 4, 8, 16, 32 and 64
// The i2c baudrate is set to 1mhz to support these
#define FPS 8
#define FRAME_TIME_MICROS (1000000/FPS)

// Despite the framerate being ostensibly FPS hz
// The frame is often not ready in time
// This offset is added to the FRAME_TIME_MICROS
// to account for this.
#define OFFSET_MICROS 850

int main(){
    static uint16_t eeMLX90640[832];
    float emissivity = 1;
    uint16_t frame[834];
    static float image[768];
    static float mlx90640To[768];
    float eTa;
    static uint16_t data[768*sizeof(float)];

    auto frame_time = std::chrono::microseconds(FRAME_TIME_MICROS + OFFSET_MICROS);

    MLX90640_SetDeviceMode(MLX_I2C_ADDR, 0);
    MLX90640_SetSubPageRepeat(MLX_I2C_ADDR, 0);
    switch(FPS){
        case 1:
            MLX90640_SetRefreshRate(MLX_I2C_ADDR, 0b001);
            break;
        case 2:
            MLX90640_SetRefreshRate(MLX_I2C_ADDR, 0b010);
            break;
        case 4:
            MLX90640_SetRefreshRate(MLX_I2C_ADDR, 0b011);
            break;
        case 8:
            MLX90640_SetRefreshRate(MLX_I2C_ADDR, 0b100);
            break;
        case 16:
            MLX90640_SetRefreshRate(MLX_I2C_ADDR, 0b101);
            break;
        case 32:
            MLX90640_SetRefreshRate(MLX_I2C_ADDR, 0b110);
            break;
        case 64:
            MLX90640_SetRefreshRate(MLX_I2C_ADDR, 0b111);
            break;
        default:
            printf("Unsupported framerate: %d", FPS);
            return 1;
    }
    MLX90640_SetChessMode(MLX_I2C_ADDR);

    paramsMLX90640 mlx90640;
    MLX90640_DumpEE(MLX_I2C_ADDR, eeMLX90640);
    MLX90640_ExtractParameters(eeMLX90640, &mlx90640);

    while (1){
        auto start = std::chrono::system_clock::now();
        MLX90640_GetFrameData(MLX_I2C_ADDR, frame);
        MLX90640_InterpolateOutliers(frame, eeMLX90640);

        eTa = MLX90640_GetTa(frame, &mlx90640);
        MLX90640_CalculateTo(frame, &mlx90640, emissivity, eTa, mlx90640To);

        Mat IR_mat (32,24, CV_32FC1, data); 

        for(int y = 0; y < 24; y++){
            for(int x = 0; x < 32; x++){
                float val = mlx90640To[32 * (23-y) + x];
                IR_mat.at<float>(x,y) = val;
            }
        }

        // Normalize the mat
        Mat normal_mat;
        normalize(IR_mat, normal_mat, 0,1.0, NORM_MINMAX, CV_32FC1);

        // Convert Mat to CV_U8 to use applyColorMap
        double minVal, maxVal;
        minMaxLoc(normal_mat, &minVal, &maxVal);
        Mat u8_mat;
        normal_mat.convertTo(u8_mat, CV_8U, 255.0/(maxVal - minVal), -minVal);

        // Resize mat
        Mat size_mat;
        resize(u8_mat, size_mat, Size(240,320), INTER_CUBIC);

        // Apply false color
        Mat falsecolor_mat;
        applyColorMap(size_mat, falsecolor_mat, COLORMAP_JET);

        // Display stream in window
        namedWindow( "IR Camera Window");
        imshow ("IR Camera Window", falsecolor_mat);
        waitKey(1);

        auto end = std::chrono::system_clock::now();
        auto elapsed = std::chrono::duration_cast<std::chrono::microseconds>(end - start);
        std::this_thread::sleep_for(std::chrono::microseconds(frame_time - elapsed));
    }


    return 0;
}