CPLDs have several programmable PLDs which are interconnected via a programmable connections at the global interconnection matrix.
FPGAs on the other hand have individual logic blocks - each block can be programmed to implement a logic function such as AND, OR, NOT etc. These blocks are then connected via programmable switches to implement complete logic functions.
Compare the FPGA way of programming individual AND, OR, NOT logic functions and then putting them together to implement a complete logic function, to the CPLD way of programming several complete logic functions at once on inividual PLDs and then bringing these PLDs together via programmable connections.
Useful web resources (as visited on 15-09-2008)
1) http://www.xess.com/fpgatut.htm: Good explanation + diagram of chip architectures
2)http://www.edaboard.com/ftopic61996.html: 4th paragraph - good technical explanation of the difference.
2)