This is very common thing to happen. If the material is rubber/ plastic based, immersing it into hot water could help. You can try scrubbing it of while the area is submerged in hot water. Please note that the bottle is resistant to hot water. Also, do not over heat the water as it may damage the bottle, as well as burn your hand. The paper residue might not vanish instantly. You might have to do this a couple of times. The residue should vanish after a few days of usage.
Also, I would advice you not to scratch it off using anything points, as you may end up damaging the bottle, or leaving on it scratch marks.
Another way into to moisten the area and Gently use sandpaper. But, then again be careful you don't damage the bottle.