Using a regulator will WASTE just as much energy, not matter how you configure it.
Just get 3 x AAA cells and connect it directly to the device, making sure you have the polarity correct. You can add a diode in one lead if you are not sure about the polarity and then remove it. The device may...