too late to see this but would have suggested grinding away the race on opposite side and then poking/prying with screwdriver to make the race collapse .... but you did it anyway.
It looks like the remains of a sealed bearing - is that correct for the frame? - it did look like it was too far down in the head tube.
It looks like the remains of a sealed bearing - is that correct for the frame? - it did look like it was too far down in the head tube.