As you say, appears to be leakage across the output element(transistor, mosfet, whatever).
Putting an extra lamp in circuit will do nothing.
Without knowing the circuitry involved, difficult to say with regards to the relay but off hand I would say, because the existing are LED then the output...