Thanks @mcckevin that’s some really useful sleuthing, and it seems to correlate now with what I hear when enabling/disabling PUlse.
Lots of bashing my head against Pulse and while I can definitely re-create these problems to varying degrees I can’t seem any way to properly solve them.
The closest to a solution is @G3WZG’s suggestion to reduce VLCs gain setting, but Pulse Audio is still applying a tremendous amount of gain that it shouldn’t be applying. This means the VU Meter activity is pathetic, lighting the first couple of LEDs until volume is unbearably loud.
I have a VU Scale option coming in Pi VU Meter to compensate for this, but crushing the gain in VLC and then expanding it in Pulse can’t be doing the sound quality any favours.
I wonder if this is related to: https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug/1311525