Answer:
Unknown
First there is nothing in the Bible saying there were a total of three wisemen. There might of been 2 or 100. All that is mentioned is 3 different gifts.
"Now when Jesus was born in Bethlehem of Judaea in the days of Herod the king, behold, there came wise men from the east to Jerusalem" Matthew 2:1
"When they saw the star, they rejoiced with exceeding great joy. And when they were come into the house, they saw the young child with Mary his mother, and fell down, and worshipped him: and when they had opened their treasures, they presented unto him gifts; gold, and frankincense and myrrh." Matthew 2:10,11
Also unlike the TV shows he wasn't a baby in a manger by the time they got to Him but a young child. Maybe 2 or 3 years old.
Jesus was already born and an unknown age by the time the Magi visited Herod. They could of started traveling after He was born or spent a year traveling prior to His birth. We don't know.