I don't think color-blindness would have much of an effect on an individual. Most colorblind people can still see in color, but have a deficiently in one of their cones. They might confuse similar looking colors. People with this mild type usually don't even know they're colorblind until they run across a dot test in a science textbook.
Complete color blindness (ie like black and white television) is very rare but does happen. A completely colorblind person might have to have someone label the color of their clothes, but I don't think that will have a huge social impact on them.
On the positive, it's thought that people who are colorblind are better able to pick out subtle details when viewing something.
Colorblindness is also a non-medical term used in psychology. It means ignoring any information on race when dealing with people. I'm not a psychologist though.