It seems to me that your best bet that will combine portability and modernity is an mp3 player. Despite the rise of the smartphone and its effects on dedicated mp3 players — the iPod peaked around 2005/6 and steadily declined till 2014 when sales numbers were last available — similar devices continue to exist under a plethora of brand names.
Many are marketed with your use case in mind. For example, this one (picked after a very cursory read of a couple reviews) is lighter than most phones and claims to be safe for sports through shock absorption. It also resolves the cable issue through Bluetooth connectivity for earbuds.
A further step in that direction is the clip form factor for mp3 players, e.g. this one. This has no base unit — the earbuds with their wraparound band are themselves an mp3 player. What a time to be alive.
I can't think of a better way to get portable music without bringing a serious device. That said, I also like Willeke's suggestion. As someone who plays and writes for piano, I do find that my attention can sometimes be fully occupied by mentally playing or coming up with music. Can't hurt to try before buying a device.