To avoid shorting outputs together and avoid a switch, you need an audio mixer circuit.
You can use electronic switches like Cmos transmission gates, or an audio mixer circuit.
In order to keep the LM386 and a volume control from overloading, the volume control is always ahead of the amplifier.