Dimmesdale has the scarlett letter seared into the flesh on his chest. This is exposed in the novel when Dimmesdale confesses his adulterous relationship with Hester Prynne.
Dimmesdale, since he did commit adultery with Hester, saw himself as a failure. He thought that he had commited a great sin and the only way to make things any better in the light of God was for him...