You might try stuffing the holes with steel wool or copper wool. Both rats and mice don't like the feel of it and they generally can't chew through it. With the copper wool, if they try, the website below says it gets stuck in their teeth, so they can't really do anything after taking a bite. The site also says that the rodents will try to remove the plug, so anchoring it is recommended.
I ran a computer store that had this same problem, unfortunately. We used poison bait blocks. These work best if you put them next to a wall, since rodents tend to hug a wall when walking and will eat whatever they first find. With that young of a baby, you don't have to worry about them crawling yet, so you can get rid of the bait when the problem is resolved. Just make sure you know exactly where the blocks are so you can remove them all and clean up around the area to prevent any residual poison or forgotten blocks.
Using poison does cause the rodent to die "somewhere", and this will likely be in the nest, but the one thing at least rats hate most is the smell of a dead rat. Any remaining rats will likely vacate and not come back.