Not sure what is happening, but it sounds like there is some sort of voltage/ground problem when the USB is connected.
So I suggest you look for an isolated USB connection (something like this), which should prevent any problem from that (or at least if it fails, it's a lot cheaper than...