Virtual Environments, Libgpiod, gpiodevice and the current mess I'm making of our Python libraries

Support
1

First off: Sorry. Everything Raspberry Pi Python is a bit of a mess right now.

Additionally, sorry for the radio silence: I’ve been having a (very, very much needed) first-time-in-a-long-time actual god’s honest break from touching my computer.

The shift to the standards-based libgpiod (in lieu of Pi-specific libraries like RPi.GPIO) took us (me, mostly because it’s my remit) much more by surprise than it should have (a cute little microcontroller might have distracted me for the last few years). On top of this, other changes with Python on the Pi 5 and Bookworm OS have been vying for my attention. Most notably the shift to requiring virtual environments versus the previous Python packaging free-for-all we’d all grown accustomed to.

I’m not going to contend these changes- they are reasonable and logical and should put us in a better situation for supporting both future Pi boards and alternate boards- but thinking how best to embrace them, and make them approachable to beginners has consumed my time over the last few months.

Additionally- While the Pi 5 was announced early, allowing me to work in the open and coordinate with other developers to get things ready, a month wasn’t even close to enough time to get the basic groundwork laid. I burned most of this time working with the developer of libgpiod in order to make a suitable Python package for our purposes- the result of this work is that pip install gpiod should Just Work inside or outside a virtual environment and irrespective if your system libgpiod version. If these words mean nothing to you don’t worry- it’s like laying the foundation for a house. There also remain - particularly in respect to Pi 5 - some unanswered questions about how things like PWM (on arbitrary pins) or pulse counting could or should work

Similarly small but important changes to things like Thonny have taken research and rationalisation - cpython_frontend: Support for ~/.virtualenvs directory on unix. by Gadgetoid · Pull Request #2978 · thonny/thonny · GitHub

With Pi 5s going out to customers, plus folks upgrading to Bookworm OS I’ve been trying to balance doing the right thing with doing things quickly, and that hasn’t led to the best planning or execution. Stuff that shouldn’t be broken is broken and this will - somewhat unfairly - disproportionately affect users of older Pi’s who should be the furthest from having to know or care about this nonsense.

This is all new code, new concepts and new territory even for me and - particularly with gpiod - there’s virtually no Python precedent to lean on because nobody has been keen to adopt it. I may not make the right choices or have all the answers and I really appreciate those of you figuring out your way around these problems, guiding others and bringing issues to my attention. Thank you.

In all cases, GitHub is the best way to raise any issues, input, comments or join me in screaming into the void, currently there are a few relevant places for this tumultuous transition-

Without sounding like a videogame publisher Solemn JPEG, I really appreciate any patience you can extend me, and understand any frustration you’re having getting things to Just Work as they should. I’m feeling that same frustration for every library, simultaneously, plus the basic building blocks.

I’m sorry that I couldn’t better handle this transition and spare you the pain of having to know or care that it was even happening at all. Please bear with me!

9 Likes
3

It’s a rough go Phil, the Pi 5 has really “shaken not stirred” things up, hardware and software wise.
My Pi 5 should arrive any day now. Once that happens I can dig in and see what I can get done. Not that I’m any kind of coding expert. More than happy to test anything you’d like tested though. I also have a fair bit of Breakout Garden breakouts on hand. And some Pimoroni hats.

Thankyou very much for the progress report. I’m sure its appreciated, it is by me.

2 Likes
4

This sounds stressful, being at the cutting edge of this tech. Thank you for doing vital work that makes us makers and learners able to access this amazing world at all.

5

This sounds like a big job to get everything working correctly. Appreciate the work that you are doing.

I have one question, will the hardware library install scripts work for the Pi5 in the future for things like Rainbow HAT, Unicorn HAT, Blinkt etc? thanks

6

Thank you. Could you please update the instructions to install the PiGlow library? The GitHub instructions don’t work directly since with the latest RPi OS and Python3, it complains saying we should use venv.

1 Like
8

ok, answering my own question here. @gadgetoid please review

Steps to install the PiGlow library with the latest
RaspberryPi OS & Python3 and venv

With the latest setup, you may not be able to install
PiGlow from the automated script available at this
url: GitHub - pimoroni/piglow: Python library & examples for the Pimoroni PiGlow

You may see this error:

Installing Piglow library for Python 3...

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this,
at the risk of breaking your Python installation or OS,
by passing --break-system-packages.

hint: See PEP 668 for the detailed specification.
Python 3 library install failed!
If problems persist, visit forums.pimoroni.com for support

Here are the steps to install the PiGlow library in a
modern setup:

(Note: This assumes that the following are already
enabled on your RPi: i2c)

  1. curl -sS https://get.pimoroni.com/piglow | bash

(This will fail with the warning listed above)

  1. cd Pimoroni/piglow/

  2. Create a Python virtual environment:

    python3 -m venv ./venv_piglow
    source ./venv_piglow/bin/activate

This should load the virtual environment (venv). Your
command-line bash prompt should look this this:
(venv_piglow) pi@raspberrypi:~/Pimoroni/piglow $

  1. Install the required packages and the piglow library within
    this venv:
   python3 -m pip install --upgrade pip
   pip install smbus
   pip install piglow
  1. If the above commands are successful, the simple
    test shown below should run. You should be able to
    light up the red LEDs on the PiGlow.
(venv_piglow) pi@raspberrypi:~/Pimoroni/piglow $ python
Python 3.11.2 (main, Sep 14 2024, 03:00:30) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import piglow
>>> piglow.red(64)
>>> piglow.show()
>>>

(to switch off the LEDs, type:

>>> piglow.off()
>>> <press CTRL-d to exit>
  1. Your virtual environment is all set up for piglow. To run additional
    piglow examples, execute them in python within this virtual environment.