Thanks for your reply!
Yes, I do have a new system with the X HAT. Upon further inspection I found that the ground wire wasn’t connected at the speaker end. I’ve soldered it back on (yet to try it out but I’m assuming that was the problem).
Also found this thread that shows what to edit to get the sound out via the HDMI.