If it has rabies or its a wild mouse or something like that, go to the doctors a soon as possible. get a rabies test, and shot. the test wont come until three days so rather be safe then sorry right?...
check if it has rabies first, if it does, go to the hospital immediatly. if it does, put neosporin on it and put and band-aid over it. don't expose until healed completely.
Mice are pretty panicky and difficult to tame.
Unfortunately, if the mouse is still biting then you haven't handled it enough or properly.
Let it get used to you before handling it.