Either phrase can be correct, as long as the rest of the sentence agrees with it; e.g. '(have/had) the pleasure of..' or 'take pleasure in..'.
pleasure (the opposite of pain, a positive sensation)
You take pleasure in not sinning. By that logic, everything that isn't torture is a sin.
You must have duty before pleasure because pleasure is not to be rushed