In the olden days before I got a car with a remote unlocking I had similar problems. I used to carry a cigarette lighter with me and I'd use it to heat the metal part of the key then insert it into the lock. It would sometimes take a while to get it into the lock if the tumblers had frozen and wouldn't let the key slide in, but once the key was in the lock things proceeded much faster
You might need to repeatedly heat the key to transfer sufficient warmth to the lock in order to get it to turn
If you have more patience, or no cigarette lighter, look for other heat sources, including your own hands/other body parts. You can additionally try placing your hand on the lock itself and waiting longer, for your body heat to warm the lock internals and melt any ice trapped in there
If you're more well equipped you might be able to find some dedicated lock de-icer. In colder countries de-icer is commonplace in shops, supermarkets and auto accessory stores. It's an aerosol or spray bottle of ethylene glycol solution that will melt ice by lowering its freezing point to below the ambient temperature, causing the ice to thaw. Even if you can't find an aerosol that has a straw nozzle that you insert into the lock you should be able to get similar success using the key to hold open any flap on the lock that tries to keep out water etc; insert the key just enough then spray the de-icer down the side of the key
Of course, if it's so cold the lock is frozen there is also a chance that the door seal rubber will have frozen to the door making it difficult to open; take care not to wrench the handle off. I normally find the best technique for a frozen shut door is to alternately pull and push the door. If you have a source of warm water, pouring it down the gap will help, resist the urge to use boiling water for a fast result though!