Gawd, is this thread still going. You lot are certainly more inquisitive than my class!
The bullet fired straight upward would (in principle) keep going 'sideways' at the same speed it was before it was fired. So it would stay above the same place on the surface of the Earth. For the most part, the atmosphere travels round with the Earth as it spins and so would the bullet while it was going up and then back down. But any slight atmospheric disturbances would carry it off course.
I've often wondered how many people get killed by falling bullets when people 'celebrate' in the streets by firing guns in the air.
There was some guy a long time ago who proposed a 'free' global tranportation system. He said you could just release a balloon into the air and wait for the Earth to spin underneath it, then just 'land' it when the right part of the Earth got under it. Sounds good, but it wouldn't work.