It probably means you've done or said something that embarrassed him or made him angry. You may not even know what it was but something occurred that upset him.
It means that the person is as important to you as is the entire world. You care for them as much or more than all of the wealth in the world. That they are your world and your interests revolve...