A completely electricity-/stove-free method would be storing the baby milk not in rigid bottles, but in zip-lock-type baggies as they are sold for storing expressed breast milk, for example.
Slip the cool bag under your clothes and use your body heat to get it to temperature.
It shoud warm up quite fast as the bag ensures that a comparatively thin layer ...
Edit: the additional requirement of 'child safety' invalidates this answer. Fabric fastened at one end does not make the stairs safer.
If your requirement is cosmetic only: Carpet or fabric can be used on its own, but you'd have to fasten it on one side only: attach it to the bottom of each step, and leave the other end free.
When you walk up the stairs, ...
If you can place something around the desk (top, behind, underneath and front to connect on the top again) you can then 'lock' the chair underneath the desk to that item.
I would go for a rope and a good knot the kid can not yet untie, but most people will go for the ratchet straps. It you run two side by site and knot them together in two places you make an ...
Hand-warmers might do the trick, and they can be more space-efficient and portable than the boiling water.
What kind to get obviously depends on how much stuff you need to heat and how often you need to do it; you can go for packets, which are relatively cheap, or an electrical warmer, which is a bit more sustainable for repeated use.
Assuming that plywood covers the back of two long sides, but none covers the two short ends:
Position the desk so that one of the short ends touches a wall.
When you finish working at the desk, pull it away from the wall.
Position the chair against the wall as though you are going to sit at one end.
Push the desk back to the wall to trap the chair.
If your child has trouble moving the chair and it is in an inconvenient location to use for climbing, this idea may work.
Place the chair wheels/casters into cups (jar lids) so that the chair is difficult to roll around. You'll have to pull it into position; but, it shouldn't be too difficult for an adult.
When the chair is unoccupied, remember to slide it ...
Lift the grate and look underneath for a place in the duct where you could attach a piece of cord or wire, e.g. a hole, a projecting screw or sheet-metal overlapping a corner.
Put a piece of cord or wire at that point.
Pass the cord or wire through the grate.
Twist it around a thicker part of the grate and knot or bend it in place.