The pHAT DAC uses I2S, which is distinct from I2C, despite the incredibly confusingly similar names (computer industry acronyms are a continuous draw for ridicule).
You can see which pins pHAT DAC actually uses here: http://pinout.xyz/pinout/phat_dac
You can install pHAT DAC and then solder onto BCM 2/3 ( I2C SDA/SCL ) and the appropriate power pins to drive the LED driver.
You can wire the Adafruit amp to the holes intended for the RCA header on pHAT DAC. The larger holes are grounded which you should connect to the left and right (L- and R-) inputs on the Amp. The smaller holes are the signal, which you should connect to L+ and R+.
As the description mentions:
"Inputs of the amplifier go through 1.0uF capacitors, so they are fully 'differential' - if you don't have differential outputs, simply tie the R- and L- to ground."
How's the LED driver currently connected, by the way?