Answer:
Lee's father was really a sad case. He was not good with money. He went broke trying to promote a land development, but he was just ahead of his time. The land in question was right across the Potomac River from where Washington DC was created, just after he went broke.
At the time of the War of 1812, Lee helped a friend who was a newspaper editor in Baltimore. The mob in Baltimore had somehow gotten the idea that this editor was pro-British. Lee was in his office with him when the mob came, and suffered a fearful beating. He never recovered from this, eventually dying of the effects. He had lived long enough to travel to the Caribbean, hoping the warm island climate would help him, but realized he was dying. He boarded a ship for home, hoping to die with his family, but along the way realized he would not make it. He had the captain put in at Cumberland Island, off the coast of Georgia, where his old Revolutionary Army friend Nathaniel Greene had lived. Lee was carried into Greene's old home (Greene was already dead) and into an upstairs room, where he died a week or two later. He is buried on the island. Lee only got to visit his father's grave once.