Answer:
Absolutely it is. I'm sure if you have a good relationship with whoever you are saying it to... or if two people of the same race are saying it to each other, it could be funny or you could just laugh it off, but it has the potential to be offensive because you are criticizing someone just for being who they are. Similarly, "Why are you such a girl" is also dismissive of the self-image of whoever you say it to. Generally, asking someone why about something that is an obvious part of who they are is offensive. If you want to get to know someone, or know about that part of them, instead of asking them why, try asking questions about how they grew up and get to know them a little more. They'll tell you who they are, and that will include any cultural aspect to the racial or gender-based facts.