I think it’s doable with some custom wiring. I’m in the process of building an equivalent Pico based setup with Breakout Garden breakouts etc.
Pi Pico based Weather Station Project - Discussion / Projects - Pimoroni Buccaneers
My plan is to put a Pico in a box on the pole with the sensors. Then send the readings via WIFI to another Pico inside to display the readings. That’s the plan anyway.
My code shows how to read the sensors. And it could be edited to use the one 240x240 display, once you remove the BME688 outside readings etc.
How good are you with Python coding? It will be a little work but should be doable.