Player X not detected

Hi everyone

I’m working on an arcade controller made of arcade buttons and joystick, and i’d like to put a Raspberry Pi4 in it to get a full gaming console with only usb and hdmi outplugs.
I use a Pimoroni Player X card to connect to buttons and the Pi, and it works pretty well.

The only thing is that the player X card isn’t detected by the Pi when i boot it. I have to unplug/replug to card to have it detected.

I feel like the USB card isn’t recognized by the Pi during boot or ES doesn’t recognize the card as a controller.

Do you guys have an idea on how could i fix this? I’m running out of solution…
Thank you for your help.