Wash the area down with water to get rid of the grit, then use a car polish which is slightly abrasive to remove the rubber mark. I've used a brand called Mer in the past both on bikes and on cars - apply it with a damp cloth, polish out the mark and then buff off with a dry, clean, soft cloth. It should leave a beautiful lustrous finish behind because it also has some Carnuba wax in it which shines paint finishes beautifully.
You'd be surprised how many marks can be removed in this way with a bit of elbow grease. It'll hopefully also deal with the front derailleur band marks, too.
The only problem is that your seat tube will then probably look a lot better than the rest of the bike so you'll have to do the whole thing