Yes, you may remove a lateral water hazard stake, under no penalty. The only stake you are not allowed to remove is an OB stake, this is because it marks the boundary of the course.
You have to take a penalty drop, which is one shot. So if you start a hole, and hit your tee shot into the water, you take your penalty drop and then play your approach, which will be your third shot.
If you hit it into a water hazard, you can drop the ball as far back as you want, keeping the point where the ball entered the hazard between you and the flag. Or you can go back to where you hit the...