As suggested above, filtering appears to be the main reason.
You really need to check the ESR of those capacitors before you go down a rabbit hole.
As also mentioned, the fact that the audio/noise increases with volume, suggests that the audio IC’s, preamp and amp are working.