The high-voltage capacitor in a disposable camera's flash circuit doesn't make the high voltage, it just stores it for a few seconds. Making it bigger or using more than one will just take a longer time to charge and you will end up with the same voltage.
To make a higher voltage you need a transformer with a higher step-up ratio, like from a TV, and a more powerful transistor circuit to drive it . Since it will use more power a larger battery might also be needed.
In electronic circuits, if you change one thing that affects its power, you usually must change many other things to match.
Try a higher battery voltage with the flash circuit, then it will try to make a higher output voltage. You might get lucky so the parts won't break-down too soon. If you are unlucky, the parts will fail the first time you try it.
My very 1st portable radio was small and not very loud. It used a single 9V battery. I tried two 9V batteries to feed it 18V and it played much louder for luckily a long time without breaking down. But two batteries costed twice as much and I had to replace them much more often. It also looked stupid with the 2nd battery attached to its outside with a rubber band. ;D