Not necessarily, there are several regions around the world named Georgia. Georgia as in "the state of Georgia" is not the same as a city inside that state.
Georgia could also refer to the Georgian...
If you mean Georgia state it was always Georgia. As for country it was called Georgia from ancient times, but before that it was called kolkhis and then iberia.