The author of Matthew's Gospel did not use the term 'wise men', he said that
magi followed a star from the east, and even then, he still does not say there were three of them.
The
magi were priests of the Zoroastrian religion which the Jews had encountered during the Babylonian Exile. So, the 'wise men', or
magi would have come from Persia or Babylon (Iran or Iraq in modern terms) where Zoroastrianism was practised.
John Shelby Spong (
A Bishop Rethinks the Birth of Jesus) says that among people he knows in New Testament circles, the universal assumption is that Matthew's
magi, or wise men, were not actual people. Matthew, by having the priests come to worship Jesus, wanted to show that even the priests of this great religion looked up to Jesus.
In more recent times, the Zoroastrian connection became less important, and the priests began to be called wise men. Historically, the wise men were a literary fiction.