Any short section of tube (a drinking straw, section of pipe, ball point pen with the ink cartridge removed etc) can be used to inflate a balloon without coming into contact with the rubber. Insert the tube into the neck of the balloon. If the tube is smaller than the neck, pinch the slack and then roll it so the balloon wraps the tube well. By leaving part of the tube sticking out of the balloon you can then inflate the balloon without touching the rubber
Before now I've had good success using one of those nozzles that comes with a cartridge of silicone sealant -
It can be trimmed so it has a big hole in one end but still has a tapered section that makes it easier to feed into the balloon neck
If you fill balloons and other things often it would be worth investing in a double action inflator pump:

The rate these things inflate balloons is unbelievable; maybe 3 - 5 cycles. They're also great for paddling pools, air beds, water inflatables etc - all things that stay mold free longer if they aren't inflated with moist air from your lungs