it is probably the bearings going out in the ac compressor pulley or belt slippage. the reason it is different when it is on is because the belt is not under "load" or has resistance.
The Air conditioning compressor is likely seized. There is a clutch in the pulley that engages the compressor when the A/C is turned on. You can easily check the compressor (with engine off make sure...