Answer:
Yes.
For arguments sake, let's define a child as a prepubescent, and an adult as a person over 18.
I would argue that pedophilia is wrong not because sex is harmful to children, but because the child is really unable to give consent because the lack the true freedom to say yes or no. This is because legally in our culture, children are under the authority of adults. Psychologically, children have a hard time saying no to an adult because adults control their needs, shelter, food, love. The wrongfulness does not depend on the proof of harm, but rather the asymmetry of power in a child-adult sexual relationship.