Hi Autir,
1) Self-oscillation occurs with some circuits like a regulator that have a very high gain and a lot of negative feedback because of phase-shifts in the circuit and load.
2) The output cap improves transient response because it absorbs sharp transients that the fairly slow regulator can't deal with. If a regulator's load is suddenly disconnected, its output voltage will have a increasing-voltage transient until the slow regulator's circuit "catches-up" with it. Likewise, if a regulator's load is suddenly applied, its output voltage will have a decreasing-voltage transient.
An output transient also occurs if the input voltage suddenly changes.
I think that the LM317 regulator's datasheet shows these transients and the smoothing effect of an output cap.
3) An output cap minimises the effects of very high frequency load variation because it has a low impedance (good filtering) at high frequencies. The very high gain and a lot of negative feedback in the regulator minimises low frequency load variations and also slow input voltage fluctuations (ripple).