You found the holy grail for your amp.
E1 is Signal Ground for Left(1) input, E2 is Signal Ground for Right(2) input, E3 is Output Left(1) Ground and E4 is Output Right(2) Ground. I don't see why you can't tie all of them together with the Power ground.
The Fuses are protection for your...