The USB 1.x and 2.0 specifications provide a 5 V supply on a single wire from which connected USB devices may draw power. The specification provides for no more than 5.25 V and no less than 4.75 V (5...
A standard USB port (non 3.0) can provide up to 500mAh of power. This is on full power mode. A 3.0 port can, if plugged into a 3.0 device, provide up to 900mAh if configured to do so.
For manufacturers, the cost to implement them on a motherboard is only a couple of cents. If you want to add a USB hub to your computer, they can usually be had for under $20.