The world population in 2004 was 6.4 billion people. As the Down syndrome rate is about 1 per 1000 births, this gives us an estimate of 6.4 million people who had Down syndrome in 2004.
As Down syndrome people have a lower life expectancy than other people, this may be a slightly high estimate.
(Please see the related links for sources.)