Leakage from batteries, if they are not drastically defective or damaged, results from the normal electrochemical reactions inside them consuming the metal casing and opening a hole. Therefore, the first line of defense — how to prevent nearly all leakage — is to remove and discard discharged batteries. Since all batteries have some amount of “self-discharge”, this means you must change them periodically, but as long as they still have sufficient voltage to operate the device, you're probably safe.
If the device has a “soft” power switch, one which does not actually break the connection through the batteries but signals an electronic circuit to shut down, then for long term storage you should remove the batteries to avoid the extra drain of the controller waiting for the button to be pushed to turn on. Ways to tell:
If the power switch has two mechanically different positions (pushed in and popped out, sliding/rocking, or just a distinct feel and sound when turned on or off), then it's a “hard” power switch.
If the device has control features like an auto-off timer, that don't require you to turn the switch off and on to reset, then it's likely a “soft” switch.
If the device properly uses the IEC standard power symbols on its switch, then the symbols with an unbroken circle ( ) indicate a “hard” switch where the broken circle () is “soft”. (However, the latter symbol is so iconic these days that it has been misused on switches that are actually hard.)
If the device has a soft power switch, you can break the circuit inside the battery compartment. The simplest way to do this is to remove one cell (or all of them, but one will do). A fancier trick is to insert something thin and non-conductive, such as a thin piece of plastic sheet, between the end of one cell and the battery compartment's contact, then pull it out before use. (I wouldn't recommend paper, because I've done that and later found rust on the contacts — water absorption? acid paper? Don't know.) Suitably thin plastic sheets can be cut out of plastic retail packages or those little collar-stiffeners in dress shirts.
The battery chemistry affects the probability of leakage. I haven't found any reliable comprehensive sources on this matter, but here is the information I have managed to accumulate — if someone could offer some references I'd love to see them.
Common alkaline batteries have the highest risk of leakage.
Zinc chloride (often called “super heavy duty”) batteries are somewhat less prone to leakage and are cheap but have inferior performance (less lifetime when in use).
Lithium-iron batteries have superior performance and long shelf life (low self-discharge), but are generally much more expensive. Lithium-iron batteries are generally highly recommended for infrequent use like you're looking for. (Lithium-iron cells in AA and AAA shapes sold in stores will likely be just called “lithium”, but note that there are many other types of lithium cells which are not 1.5 volts and different in other ways.)
Rechargeable cells, such as NiMH (nickel metal hydride), are also less likely to leak because their chemical reactions are designed to be reversible (and chewing up the canister wouldn't be very reversible). For this application you would use “low self-discharge” (aka “precharged”) NiMH cells, and still want to recharge them occasionally. (By the way, there is no need to periodically discharge rechargeables; “the memory effect” is approximately a myth.)
(If I were to use my first-hand experience only, I'd say “Alkaline batteries leak. Nothing else does.”)
All of the above said, the safest thing you can do is keep the batteries outside of the device. This prevents discharging of all sorts other than self-discharge, and allows you to keep the device and discard the batteries if they do leak. On the other hand, you have to insert the batteries with the correct polarity before use.
If you need to insert AA/AAA/C/D-style batteries blind, here are some cues for the proper direction you can find by touch:
The positive end of the battery holder will have some kind of “shoulders” sticking out past the contact that are designed to prevent the negative end of a backwards cell (which has no bump in the center) from making contact.
Usually, there is one contact which is sprung, and that contact is the negative end, whereas the positive end is rigid. This is not 100% reliable.
I wouldn't worry about short circuits — it's hard for loose objects to make a complete circuit (as long as the insulating plastic shell around the cell is intact) in AA/AAA/C/D type batteries, unless you're doing something like keeping them in a box of metal parts. You can be even safer by keeping them in a bag (also handy for less mess in the event of a leak) or wrapped up in a bundle with a rubber band or cable tie (face them all the same direction).