In order for volume control to work on pHAT DAC - since the hardware doesn’t support it - you need to set up the softvol plugin in asound.conf. See: Speaker Phat + Retropie Voume
I think the audio device setting panel for the Pi is somewhat quirky and tends to drop an ~/.asoundrc file to set the audio device, which overrides any custom settings in asound.conf. It’s been a while since I’ve looked into it, but this tends to break things so we favour disabling it, disabling built-in audio and just letting the DAC be the only option for output.