One of the major causes of driveway cracks is the intrusion of large tree roots under the surface. Tree roots, if they are large enough, can burrow under the concrete or asphalt and actually lift the material up over time so that it splits under the pressure. This is not usually a problem with younger trees, but trees that have been around for decades will usually have the root strength to break through the driveway.
Driveways can actually sink down into the ground if they are not built upon the proper soil. Soil with a high clay content, or soil that is prone to erosion over time, will often expand or shrink in inclement weather, which will cause the driveway to either raise or lower depending on the motion of the soil. This will not cause a problem if the motion is uniform throughout the entire driveway, but if only a part of the driveway sinks, then cracks and other deformities can appear quite easily. Also, driveways are usually installed over a base constructed of compacted stone. If the base wasn't properly compacted or installed, it could cause structural problems with the driveway later on.
All driveway surfaces are not created equal. Residential driveways are usually not thick enough to withstand the weight of construction vehicles, for example, or vehicles that weigh in excess of 3 tons. A driveway that experiences regular heavy vehicle traffic will eventually give in to the pressure and experience cracks or deformities.
A single day of inclement weather will not do much to hurt your driveway, but a decade of exposure to torrential rains or extreme cold will eventually chip away at the driveway's integrity until it starts showing signs of wear.