A PLL has it's beginnings in the ideal situation of zero time difference. Although there is a feedback loop, there should be practically zero time difference. The voltage at the VCO is what's important. The signal takes the frequency up and the correction voltage from the phase comparator pulls that frequency somewhere close to the reference oscillator. Somewhere close is the key. The margin of error directly affects the ability to modulate. If it were perfect, then the output of the VCO would never change and your left with one frequency. But since there is a sizeable error, you have a change in frequency.