I have a bit of a weird issue (more accurately: idiot user) in that I have a fully-working bme688 that I can interact with when using the sample python code and working interactively, but which I’m unable to access via a cron-triggered bash script on my Raspberry Pi. Even more weird, I do have this exact setup working on an older Bookworm-based Pi, but that doesn’t seem to use the “virtual environment” solution and that is where I seem to be going wrong. I’d be very grateful for advice.
I have a working solution on a Pi running Bookworm, where the code was set up in February 2023 and has worked flawlessly since; I’m trying to recreate the environment on a Trixie-based machine built last week and I am seeing the following error. I have tested the sensor in the failing solution and it is working perfectly.
Problem Summary
Attempts to interact with a working bme688 via a bash-script-invoked python script fail with the message
File “/opt/scripts/atmosphereSensorCapture.py”, line 20, in
import bme680
^^^^^^^^^^^^^^
ModuleNotFoundError: No module named ‘bme680’
Successful Manual Launch:-
If I log in to my Pi as a regular user, then execute the statement:-
source ~/.virtualenvs/pimoroni/bin/activate
and then invoke my script, it works as expected.
Steps to re-Create Problem
However, if I set up a simple bash script, with the following content:-
and then try to invoke the Python script indirectly via the bash script, I get the error documented in “1. Problem Statement”, above.
Please note that I copied the entire contents of “.virtualenvs” to “/opt/pimoroni/…” because in earlier tests I seemed to get an issue prompted by the fact that virtualenvs is a hidden folder in my user’s home directory… Again, I have tested interactively with the relocated code and a “source” statement and that works perfectly.
Analysis
From what I can make out - and I am NOT an expert in any of this, the invocation of the python environment from my bash script is effectively creating a new thread space, or execute space, with a separate set of environment variables, which is why it is not retaining or remembering the fact that my bash script is setting that in the preceding step.
I am confident that there are no hardware issues and also that both the example scripts and my own script all work perfectly when invoked interactively. The issue all seems to be related to the use of this “virtual environment” in the later version of the codebase that I picked up when I installed the support libraries on my new, Trixie-based Pi.
I’ve looked for explanations/help on the generic question of passing the results of the “activate” script to the invoked session… but so far I’ve had no luck getting it to work.
I get the strong impression that this is an “idiot user” mistake and that there’s something really trivial that I’m missing… but for the life of me I can’t see it [yet].
Any/all suggestions would be very much appreciated.
OK, after a lot of experimentation and digging around, I’ve found one workable answer to my own question.
Unfortunately, Pimoroni’s adoption of virtual environments - while helpful for a lot of use cases - doesn’t help here. However, you can “work around” that by adding a line like this to your python script, before you get to your “import bme680” statement:-
Note that in the above statement, I have already copied the entire contents of ~/.virtualenvs/pimoroni/ to /opt/pimoroni, which I did because I didn’t want system level code - supporting hardware, no less, to be executing from a general user’s home directory!
You don’t have to use “sys,path.insert” if you don’t want or need your Pimoroni libraries to appear at the front of the path list - you can use “sys.path.append”, which takes just the pathname for the argument, if you wish. But either of these will allow you to adjust the list of places that Python will search for libraries. Note also that you should stop at “site-packages” and not be more specific; Python will work out the rest for itself.
I have to say that the change in approach - the addition of the virtual environments - might have benefits, but the implementation [and the lack of documentation] is disappointing. Pimoroni could have done a much better job of documenting this.
When using a script run from cron or system use the path to the venv python and all will be well.
So if your venv is /usr/fred/myproject there will be a bin directory and in that is pip3 and python3, and if you call /usr/fred/myproject/bin/python3 you will be using the venv.
If you want to use the BSEC2 library with your BME680 then you could look at my GitHub python wrapper for the BME680 and BME690, here
I appreciate your response and in particular your explanation of the need to use the version of python3 located in the virtual environment folder structure. That wasn’t documented anywhere I got to reading yesterday - I guess that is something one learns after experimenting with a python virtual environment, but this was a first experience of that setup for me and I found Pimoroni’s explanation of their implementation to be unhelpful. In particular it seems negligent to provide documentation for only interactively obtaining readings from a sensor module, when for anything beyond post-assembly testing, automation is going to be the way 99.99999% of users deploy the unit.
I’m also a little puzzled as to why the “install.sh” script would default [without choice] to deploying device driver code in an end-user’s home folder - that’s just simply a bad practice. And in digging further yesterday, I see that the …/bin/activate script, which is a shell script, is tailored during installation and has the “install user’s” home directory hard-coded in to the script itself.
Got there in the end, but I found this far less intuitive or documented as I would have hoped.
The venv changes were forced on us all by Debian which manages python and what it considers to be “core” packages via its package manager apt. Everything else has to be installed locally in one or more ‘venv’ using the ‘venv‘ copy of pip3. . However the BME688 predates these changes and some users will be on older versions of Linux, so we end up trying to support both worlds.
Once you get used to them ‘venv’ are useful.
For the newer BME690 python wrapper I just assume the user will have an up to date version of Linux and will be using ‘venv’ (I do give examples of creating and using them).
It’s been a while since I looked at the Pimoroni/Adafruit code for the BME688, but I believe that it is pure Python code and not a driver.
Your cron/bash script isn’t seeing the virtualenv, so Python can’t find bme680. The simplest fix: call the virtualenv’s Python directly instead of source activate: