A country with a prime minister, such as Great Britain or German has a parliamentary system of government. This type of government emphasizes the role of a party; people do not vote for individuals, they vote for parties and the party leaders decide who will be representative of the party and its supporters.
A country with a president such as the US has a representative government; in this type of system, voters vote for individuals backed by parties instead of parties backing individuals.
A representative system of government tends to lead to fewer parties, each encompassing a wider variety of politicians with wider ranging views while a parliamentary system has more parties that each occupy a smaller part of the political spectrum.
There isn't necessarily a difference. Certain countries, such as France, Germany, Ireland, Russia, etc. have both a president and a prime minister. Countries that have a prime minister but not a president usually have a monarch and/or viceroy that fills a similar constitutional role to that of a president, such as in the UK, Canada, or the Netherlands. The real distinction comes from whether a country follows a
presidential,
semi-presidential, or
parliamentary system.
In a presidential system, like the US, the executive is responsible to the president alone. Thus, the president is the dominant political figure who represents the country internationally, and the country will likely not have a prime minister (though there are exceptions).
In semi-presidential systems, such as France, the executive is responsible to both the president and parliament, with the distribution of power between the president and prime minister varying based on the political climate, though the president will often still represent the country internationally.
In parliamentary systems, such as Germany, the executive is responsible exclusively (or primarily) to parliament, and the role of the president is largely ceremonial. In this case, the prime minister (Chancellor, in the case of Germany) is most likely to represent the country in international politics.