HiFi is short for "high fidelity". The output of a high fidelity amplifier is the same as its input: very low added harmonic and intermodulation distortion, a wide flat frequency response and very little added noise.
An audio expert said the perfect high fidelity amiplifier is "a piece of wire with gain".