Yes, seal it BUT, as I recall, the individual caps may have pressure relief holes in them so please ensure you have the same arrangement. (can;t be too sure - you would have to compare your battery to an original to see if you spot a small hole).
Plastic cement (glue) should hold a suitably...