In his book "MY STRUGGLE" Hitler said: "I could have burnt all Jews in the world, but I left some of them so you will know why I was killing them".
Ich bin Hitler
Because Jews are not a race, no. He is famous for genocide, crimes against humanity, etc, but not racism.
He is infamous because he was famous for something that was terrible. From Dictionary.com Infamous - having an extremely bad reputation
Perhaps his most famous was:"Make the lie big, make it simple, keep saying it, and eventually they will believe it."