Answer:
No, the Harlem River does not flow through Harlem. It flows around it.
The island of Manhattan is surrounded by three rivers: the Hudson River, the Harlem River, and the East River. The East River is not actually a river, but an estuary, or a tidal strait. It connects to the ocean on both sides. It flows south from the Long Island Sound, down the east side of Manhattan and the west side of Queens and Brooklyn, and into the Atlantic Ocean.
The Hudson River flows down the west side of Manhattan, and the east side of New Jersey. (Interesting bit of trivia: the Hudson River is actually a fjord. It is the only fjord in North America).
The Harlem River branches off of the Hudson River, and flows down over the top of Manhattan, into the East River.
See the Related Link below for a complete NYC subway map. The Hudson, Harlem, and East Rivers are all labeled.