Exactly and many of those charge modules do not have floating output but rather common negative.You have the batteries in series but the chargers are in parallel.
Then the chargers do not charge and the switch does nothing.
You need a charger designed to charge at 8.4V which is two Li-PO cells in series.
"A" charger module? What about the other one?I configured a charger module to charge 8.2V