Rule #0) Dress in layers. The rest of the rules won't work without this pre-requisite.
Rule #1) Stay dry. Your outermost layer should block water but also be breathable so your sweat can escape. Your innermost layer should wick away water. Wool or polyester work well for that.
Rule #2) Keep out the wind. Your outermost layer should also block wind. Try to keep it tight where different layers meet. For instance, an elastic waist in a jacket helps keep the wind from billowing up underneath and circumventing the outer layer. Before you go out, stretch your arms around to make sure your wrists are waist aren't exposed.
Rule #3) Keep in the heat. Have a collar that blocks heat escape like an elastic band or scarf. Your middle layer should be insulating like wool, fleece, or even down.
Rule #4) Be mindful of changing conditions and ajust accordingly. If you're building up sweat, take off a layer or loosen it so you get better ventilation. If you start to feel cold somewhere, change something. Stand up if your butt gets cold. Pull your hands inside your sleeves or even all the way to your torso if you're losing feeling in your fingers. Go change your socks if your toes get cold.