There isn't enough information here. I have a Chinese-made lantern with 2 4 w flourescents that uses a 6 volt sealed lead acid and it uses a 12 volt 500mA wall wart to charge it. Inside the light is a bridge rectifier (so the polarity of the wall wart doesn't matter -- it could even be AC!) followed by a few diodes to drop voltage and finally some resistors to limit charging current, all of which together drops about half the 12 volts and leaves the rest for the battery.
On the other hand, a kiddie car runs on a 6 volt 10AH SEL, too, but it uses a transformer that is 6 volts 1.2A and plugs directly onto the battery instead of the car's wire harness.
You could open up the light and trace some wires, if the charging jack is connected directly to the battery + & -, then then a charger about 6 volts is probably right. If the charging jack goes to a circuit board, then it's harder to say without tracing out what is in the circuit.
Another point: Wall warts are rarely the voltage stamped on them. They are a voltage source with a lot of internal resistance. So a 12 volt charger open circuit may measure 17 volts. Hooked to a discharged battery, it will be loaded down to arround 13, and as the battery charges the voltage will rise to something in between.
How safe it is just to try a wall wart depends on the power. Lower current ones, like 500mA or less usually have no fuse and so much internal resistance that even a short won't hurt them. Higher current ones often have a fuse inside the sealed case. Of course the other half of the story in inside the device you plug the transformer into, and it may not like the voltage of a transformer other than the one it came with.
If you can remove the battery, it could be charged externally. Most SEL batteries say what they like right on the case, even thought they all want almost exactly the same thing. For example, a 6 volt 4AH I have here says:"cycle use 7.2v-7.5v, Standby use 6.75v-6.9V Initial current less than 1.35A". If you have a volt meter, you could verify the battery is getting about the right volts using this information, and check the current to the battery tapers to some low "trickle" value once the battery is charged. If, after a reasonable charge time the charge current is still high or the voltage across the battery exceeds the maximum "standby" voltage, there is some danger of overcharging the battery. This does not necessarily mean the charger is wrong or broke, many cheaper systems are a bit of a compromise and depend on you to stop the charge after some reasonable time.
I'd expect it would be OK to substitute the 13 volt 1.2A transformer for the 12 volt 600mA transformer, but you need to know what is inside the device being charged, 1 Amp diodes are used everywhere and 1.2 amp would be a slight overload. On the other hand I doubt the full 1.2A is actually delivered for most of the charge cycle. So you either need to take it apart and find out, or wait to hear from someone who knows those specific lights.
Sorry I can't give a more certain answer.