Then it would be very easy not to get shameful because you would not have anything to give you shame in the first place. So, if you do not feel shame from anything but shame, you would not feel shame...
It means to be ashamed, specifically: you know how when you're ashamed you're kind of embarrassed to look anyone in the eye, so you look at the ground? That's what the phrase is talking about.