A volume control pot or any resistor will form a highpass filter and cut low frequencies if the value of the pot is too low or if the input coupling capacitor's value is too low.
If the value of the volume control pot is too high then stray capacitance will join it and form a lowpass filter which cuts high audio frequencies.
If the volume control is not capacitor-coupled to the amplifier circuit then it will change the DC bias of the circuit and cause distortion.
Attach the schematic with parts values and we will help you fix it.