Answer:
-- Multiply the first averages by the number of observation for each set of these.
-- Add up the sets of averages.
-- Divide the sum by the total number of observations (Add cardinaility of each set).
-- The result is the average of the averages.
If you say have 4 "average" value and just add these, and divide by 4, the result is "unfair" because average may be of 3 observations, while another of 1000. So, to "compensate" and make every observation just as valuable, you re-generate the "sum of sums" and then divide by the total number of observations.
If all sets are the same you can divide by number of observations.