Not really--he might be bisexual, of course. There are also things like "down-low" and other round-about ways of avoiding a gay identity while having gay sex from time to time. But in general someone who is sexually or romantically attracted to someone of the same sex is at least bisexual if not also gay.
Since one can be gay and never have sex with another man, one can certainly have sex with another man and not be gay. Getting off and feeling emotional attraction to another man are two entirely different things.