You are aware that the Pi uses 3.3V logic, right? Not 5V logic. The scroll pHat will need 5V for power, but I would think all the other inputs should be fine with 3.3V signals. Have a look here, https://pinout.xyz/pinout/scroll_phat#
That’s one of the gotchas with a Pi. It’s powered by 5V but uses 3.3V logic. You have to be careful what you connect to it. Your only issue is the scroll pHat needs 5V for power.