say yes and give him a chance! its not going to hurt you at all! It is not the easiest thing to ask someone out! so be courteous and say yes! You dont have to say yes to him again, just give him a chance once.
just be yourself and if that doesn't work she\he is not the one for you, or try to get the persons attention by staring at them or giving them a little smile once in a while ;)