Hi Tubby,
The datasheet shows how inefficient the LM386 is with a 4 ohm load.
It can supply only a whopping (hee, hee ;D) 300mW to a 4 ohm speaker with fairly high distortion.
With a 6V supply. it would get hot at only 300mW output.
With a 9V supply, it would be even hotter but the max output power of only 300mW doesn't increase.
Find a more powerful amp IC. Four ohm speakers are used in cars. Philips and ST Micro make about 140 amp ICs for cars. I use a National Semi amp IC that is obsolete now but it drives my 4 ohm speaker to 4.8W at very low distortion with a 7.2V supply. ;D
View attachment 38017