OK.
You have the title, but I bet on that title the lender you own money to is listed as the lien holder. He can repossess the car at any time if you miss payments. Having the title means nothing.
Yes, late fees indicate that payments were not made on time, which renders the original agreement invalid and allows the lender to repossess the vehicle if they so choose.