No, it doesn't. Well, unless you use your fingers to do dirty things and then lick them...then I can assure you that it will not affect the sense of taste. (:
Much of what most people consider "taste" actually is smell. Most fruit really tastes pretty similar... if you plug up your nose so you can't smell it, the difference between two fruits of similar...