Answer:
Almost always, potholes in a sealed road are caused by a weakness in the sub-structure, that allows excessive flex in the seal coat, and increasingly poor bonding with the sub-structure.
In gravel roads, the cause is different. A small hollow collects water, and this is splashed out, along with some gravel, by a passing vehicle. Thus the hollow rapidly deepens and becomes larger.