White polyurethane (PU) sealant is a good candidate and should yield a nearly invisible repair. It's not foam but of a similar softness and flexibility as foam so there is little danger of mechanical separation during use.
Instructions:
Clean the hole thoroughly inside to de-grease it. I usually use ethanol. In your case a cotton bud would make a good cleaning tool.
Push in the tip of a sealant gun into the hole and fill it with PU sealant. It's fine if some comes out the hole, at least you'll know it's full …
Use a plastic scraper to remove excess material and create a smooth exterior. An old credit card works well.
Let it cure for a few days. PU sealant hardens by reaction with air humidity at a rate of 1-3 mm/day.
Disadvantage: If you don't have it around, most likely you'll have to buy a full 300 ml cartridge of PU sealant. It's universally useful and lasts ca. 6 months once the cartridge is open, so not a big deal.
Variation: Note that there are also PU based construction glues. These are a higher strength material but also become harder than sealants, so I think that a sealant mimics the mechanical properties of the foam better in this case. Anyway, both would do.
Source: Educated guess from experience. I glued maybe 8 shoe soles back in place over the years, some with a foam backing. And PU sealant / glue worked best.