See: Speaker phat vs. phat dac, clicks only on speaker phat
Pulse Audio is probably the best solution, but has some overheads which the Pi Zero doesn’t deal well with.
I’ll lock this thread, since it’s best to have all the discussion in the one place (linked above) or I’ll lose track of this thread again!