@skypi: makes sense to me that you only get to the top range when outside. There are a lot of VOC sources inside any house.
More on the humidity: the sensor seems to be woefully inaccurate … I tried to calibrate both an AMIR and BME680 sensor using the table salt test. The AMIR sensor very quickly settles on a humidity of 75%, which it should be. The BME680 sensor settles on 68%. This means the BME680 is 7% off. In fact, all of the BME680s seem to be 7% off. I did a fairly quick check of the code but all seems fine with the _calc_humidity function. Maybe it’s somewhere else in the code, or maybe the sensors are just off.
It would be really great if Pimoroni could release a binary based on the Bosch C code or, even better, based on the BSEC library. It feels like we are groping in the dark here as we have no other point of reference for these sensors.
PS: I put one of the sensors outside to compare it with my local weather report. I hid the sensor in an empty toilet roll, so it wasn’t exactly fool-proof when it came to wind. The sensor had a temp calibration of +0.x C and a humidity calibration of +7.y %. The outside temperature/humidity reported by DarkSky is 3.1C/85%. The calibrated BME680 oscillated in the range of 3.0-3.2C, and 83.5%-86.5% humidity. So, yeah, I’m fairly convinced that somewhere in the BME680 package there is an error of 7% humidity (but I can’t find it). Or all these BME680 sensors are off by 7%RH.
I tested the C code to an extent against the pimoroni python library and the results seemed very similar, so it seems to me the sensors are high temperature and low humidity which makes sense as there is a heater on the chip and as you said the mounting direct on gpios not good…
The air quality gas readings seem good as long as you keep polling the sensor every few seconds, and a fixed compensation on temperature and humidity will work well enough for me. But seems a chip to buy more for the air quality sensor than if you just want temp/humidity which a cheap chinese bme280 will do.
There is a C example using the Bosch C library on github that the link to is in the other thread on support forum.
Of course, the gas reading is the selling point of this sensor. Nevertheless, I’m sad to see such a strong error … It still feels to me there must be a programming error (which I then assume is in the Bosch library), but as you mentioned it is easy to correct.
I have been trying a lot of different approaches for the gas readings, but the readings do seem to be related to how often you sample and how hot the plate gets. For example, using the 320C, 40ms, and sampling every second I get a value around 250,000. If I sample every 30 seconds the readings stabilise around 70,000.
You mentioned you tested the C code. Do you mean the code that corresponds to the Pimoroni Python library, or the BSEC one? I think we need to get that BSEC one up and running on a Raspberry to make any sense of the gas readings, as you need temperature compensation and/or read time compensation.
I would not touch the bsec library, the readings make sense to me, bosch want to hide it in their proprietary library… gas readings make sense, (I noticed the code seemed to be doing a stepping thing) other sensors off by a fixed amount depenent on proximity to a heat or cooling source…
probably to stop the awesome Chinese cl0ners clowning them and selling them for a fiver!
30 seconds is too long, like I said, in default state of example need to keep it rolling at less than 3s (Ns undefined) per poll, there are other states of the sensor, like ultra low power, but no documentation of how any of it works… what does work is polling in 3s or les and just average it if you want slower readings… it retrigs calibration…
the mounting is crucial! do not mount it vertical on the raspberry, mounted remotely (about 20cm away from raspberry) alongside a bmp180 and htu21d readings are within tolerance/cross-correlation.
cross correlating gas sensor against an MQ135 and an MQ7 on an arduino hooked up to raspberry which both cross correlate (same response curves different names) the stepping drops that I mention fits with the plot of both analog gas sensors… so the stepping ranges I mention seem correct!
but bme680 definitely different gas responses (says not co2 sensitive) some dips/rises where no correlation, wondering if the bosch know that at different heater profiles it responds to different gasses in a known proportion…
The BME680 is running on a separate raspberry in a slightly different area of the same room. BME rises for better air quality, analog sensors rise for poorer air quality. BME Stepping makes it not that conducive to easy graphing… But sure you can all see they mainly match.
My observed readings after few days are like the one you mentioned in this topic. It’s showing usually a value between 400k and 500k when the ‘air is pure’
It took about a week of 24h readings to reach these values. In fact, during the first days I was not able to reach 250k and I was setting 250k as max value for iaq calculation. Now am always using the highest value reached as top iaq (I save the max value in a file)
Am using the standard settings of gas readings and the 75% 25% algorithm to calculate iaq based in adafruit example (bit accurate than pimoroni)
And I confirm there is high offset in temperature and humidity when you mount the bme680 near to the raspberry. Am using a zero w and the pimoroni garden hat
I have 7% offset of humidity probably because of the gas reading. I can see humidity going down during the startup. I think I will disable gas reading for calibration one day.
The temperature also I ve set -4 degree of offset, again it’s due to the gas reading but also my mounting
Am fine with these offset as far as they r stable ^^
I’m trying to build my own office weather station, meant to monitor bad working conditions.
I have been observing the bare readings for a few weeks now and I came to some conclusions regarding the gas resistance which may be interesting for the community:
- The gas resistance seems to have an exponential dependency on the humidity. When you plot the logarithm of the gas readings over the humidity you may notice parallel streaks in the scattered values, indicating a constant slope with varying offset.
- The offset observed varies over long timespans (several days) and seems to originate from the sensor drift. I think it could be compensating by just fitting a linear function to the measurements of the past days.
- I tried around with different heating profiles. There I observed, that a new profile is not immidiately loaded but after the next reading. So you have to set a new profile, thean read sensor values and after that you can read values using the new profile. I am not sure if this is due to the bme itself or the python library. In general I observed that the risistance decreases for longer heating (ant hence interaction) time and higher heating temperatures. In the response to air pollution I was not albe to find any difference aside from different scalings, but maybe I was just not probing the right substances.
Currently I am trying out different algorithms for compensating the sensor drift but I think I am pretty close to obtain stable results.
was wondering if humidity was directly influencing the result so interesting to see you reckon so, it makes sense at a cooler external temperature and humidity (probably two externals affecting the heater by interdependent amounts) the heating profile might need to be change to give same absolute sensor variables. guess it depends if the time in the heater profile is enough at all external conditions to heat to target temperature perhaps, if temperature is the primary target in the firmware.
Finally, after several month I now got time to come back to my bme680 project and can share some thoughts with you.
What I have forgotten to mention in my previous post: I did some research on the sensor type used in the bme680, hoping to find some useful information in research papers. In the datasheet on page 23 it is stated that the bme680 uses a metal oxide-based sensor. In these sensors, the resistance of a thin metal oxide surface layer is measured, which is altered by adsorbing volatile gas compounds from the surrounding atmosphere. Such sensors come in heated and passive forms, depending on materials, structure and desired sensitivity, where the bme is obviously a heated-type sensor. These sensors are inherently sensitive to water vapour (by cracking the molecules down) and hence the resistance reading depends on the humidity.
Now to processing the resistance readings: For compensating the impact of humidity, I took the logarithm of the resistance readings (R_gas) and substracted the humidity (hum) multiplied by a slope-factor, which I determined from fitting the streaks in my scatter plots belonging to weekends, where no additional voc-sources were active inside my office. As a foumula, this reads:
comp_gas = log(R_gas[ohm]) + 0.04 log(Ohm)/%rh * hum[%rh] .
The slope strongly depends on your heating profile and the polling frequency!
Looking at this compensated logarithmic gas value over serveral month of data aquisition, it follows quite well my personal experience of air quality inside the office. More importantly, it yields consistent results over several weeks!
In order to determine a quality index, I simply track the maximum readings into a list, that frequently drops the oldest readings to compensate for long-time drift and set the lists mean as my 100% air quality. This method requires that from time to time you open a window and of course that you trust your outide air to be clean. The official Bosch library also uses some compensation, where you can choose the timeframe taken into consideration, but they probably use some more sophisticated algorithms.
very interesting but sorry I haven’t understood your formula…could you please explain ?
And also R_gas[ohm] and ohm are the same?
hum[%rh] and %rh are the same?
Slope factor how ca be calculated?
the formula is actually quite simple, the square brackets indicate the units in which the quantities have to be fed into the equation: Gas resistance R_gas in Ohm and humidity hum in %rh. Hence, the slope factor is given in units of log(Ohm)/%rh to be mathematically correct.
In a program, you would just calculate
log(R_gas) + 0.04 * hum
If you want to determine your own slope factor you first have to run the bme for some time in a clean, still environment and record your sensor readings. From these you have to plot log(R_gas) over hum and your points should align on a few parallel lines. From these you have to determine the slope either by graphical construction, difference quotients of by fitting a linear function. Just choose the method that suits you best.
@Billy_the_nid Thanks for your research!
According to your last comment, your formula is:
comp_gas = log(gas_reading) + 0.04 * humidity_reading
Or is it this, according to your comment in January 2019?
comp_gas = log(gas_reading) + 0.04 log(gas_reading)/humidity_reading * humidity_reading
I’m kind of confused by it.
Right now I’m using the following formula:
comp_gas = log(gas_reading) + 0.04 * humidity_reading
And I’m reading the data once a minute and let it graph:
BME680 OK - Air Quality is good (13.027415110627395), Gas: 148634ohm, Temp: 12.5458203125 C, Humidity: 27.954323084484688%, Pressure: 956.7120210317003hPa |iaq=13.027415110627395;;0;500 gas=148634;;; temp=12.5458203125;;; humidity=27.954323084484688;;; pressure=956.7120210317003;;;
Could you please verify if the Air Quality (13.027…) is correctly calculated according to your formula using the sensor data?
Do I interpret the result from your formula correctly that
comp_gas is a value between 0 and 500 (where 0 stands for best and 500 for worst air quality)?
For your calculations, the formula in my last prosting is the one to use and your value is correct.
In January, I gave the same formula but with units! Since gas_reading is in the unit of Ohms and humidity_reading in % relative humidity the slope factors needs to be in units of Log(Ohm)/%rh to be mathematically correct. In practice, you can just drop the units.
The formula is meant to compensate the impact of water vapor on the gas resistance. It is also processed logarithmically due to the gas resistance spreading over several orders of magnitude, depending on the concentration of volatile compounds. It does not give you an absolute air quality value, as the Bosch library does. From the size of their code files I conclude that they do some way more advanced calculations in order to compute their air quality index.
What I do is, I record the maximum readings of the compensated gas value und then compute the percentage of the current value. In my office, a drop from 100% to 95% usually means that it is about time to open a window. When breathing against it or opening a sharpie right in front of it it drops even further.
Thanks for the confirmation! What I’m still not sure yet is whether the resulted IAQ value (
13.027 in the output above) is a value ranging between 0 to 500 or is it 13.027%?
How do you do this?
Sorry for all these questions. I’m currently working on a monitoring plugin to embed the BME680 sensor into open source monitoring software (like Icinga, Nagios, Sensu, etc).
I’m also trying to get Bosch to open up about their algorithm, but that will be dfficult or even impossible I guess.
So far, using your formula, the results are pretty steady:
My approach does not implement an IAQ value in the range of 0 to 500 as the Bosch library does. It is more an improvement on the IAQ example included in the pimoroni python library, which also returns a percentage value. However, this method needs a ceiling value to compare with.
What I observed is that the sensor has a much longer burn-in-time as the 5 minutes given in the Bosch documentary and even after some hours the resistance continuous to increase slowly. If it then runs for several weeks, the value is subjected to long-time drifts. Hence, you need to update your ceiling value in some way.
What I do is, after burn-in I take the current (compensated) gas resistance as ceiling. Then, about every 5 minutes, I compare the current resistance to my ceiling. If it is larger, it is appended to a list of at most 50 values. Their mean value then becomes the new ceiling. Once the list is full, the oldest value is dropped. This method helps to mitigate the impact of short time spikes in the resistance while also slowly raising the ceiling during upwards drifts. The oldest value is also dropped and replaced with the current reading every full hour, allowing the ceiling to slowly follow a downwards drift of the sensor.
Since my first post in this forum, I also got the original Bosch library running on an Arduino mega. There is a lot of math going on in there. I doubt they would ever disclose their code.
Hey Billy_the_nid have you made progress with the arduino library ? I am currently also struggling to get meaningful data out of the BME680.
I finally got around to clean up my code and uploaded it to github:
There, I summarize most of the information I was able to collect about the inner workings of the BME680.
Feel free to use it in your projects. It is still far inferior to the BSEC library (which in turn also seems to ignore some issues) but it may help to get at least some sense from the gas resistance.
thanks for publishing your library. Meanwhile I went for the BSEC Library in combination with a raspberry pi. Very nice finding of the +1.5°C self heating offset. That might explain the high IAQ readings after some days.
Awesome work! Just wondering what your thoughts were on other ways of calibrating the sensor presented on this forum (eg here and here)? Just wanted to pick your brain, given the work you have done.
I’m trying to write a Magic Mirror module for the BME680, so will be including your calculations there rather than going down the BSEC library route.