Picade X-Hat - joystick not detected although buttons do

I have a Picade X-Hat connected to Raspberry Pi 4B, in a self built arcade machine.
I followed the instructions in Assembling your 8" or 10" Picade with PICO-8 and and ran the Picade Hat installer.In the Retropie welcome screen, all the connected buttons are detected when pressed and I can map them. However, none of joystick movements are detected. My joystick is connected to the five joystick pins of course. Tough I’m quite sure I can tell where the joystick ground is, I also tried the opposite end wire as ground. Furthermore, I tried to connect the joystick (one directional wire and the ground wire) as a button, but it was also not detected (was it supposed to be detected while connected as a button??).
I would highly appreciate any help.