There is a long-recognized association in plants between small stature and selfing, and large stature and outcrossing. Inbreeding depression is central to several hypotheses for this association, but differences in the evolutionary dynamics of inbreeding depression associated with differences in stature are rarely considered. Here, we propose and test the Φ model of plant mating system evolution, which assumes that the per-generation mutation rate of a plant is a function of the number of mitoses (Φ) that occur from zygote to gamete, and predicts fundamental differences between low-Φ (small-statured) and high-Φ (large-statured) plants in the outcomes of the joint evolution of outcrossing rate and inbreeding depression. Using a large dataset of published population genetic studies of angiosperms and conifers, we compute fitted values of inbreeding depression and deleterious mutation rates for small- and large-statured plants. Consistent with our Φ model, we find that populations of small-statured plants exhibit a range of mating systems, significantly lower mutation rates, and intermediate inbreeding depression, while large-statured plants exhibit very high mutation rates and the maximum inbreeding depression of unity. These results indicate that (i) inbred progeny typically observed in large-statured plant populations are completely lost prior to maturity in nearly all populations; (ii) evolutionary shifts from outcrossing to selfing are generally not possible in large-statured species, rather, large-statured species are more likely to evolve mating systems that avoid selfing such as self-incompatibility and dioecy; (iii) destabilization of the mating system—high selfing rate with high-inbreeding depression—might be a common occurrence in large-statured species; and (iv) large-statured species in fragmented populations might be at higher risk of extinction than previously thought. Our results help to unify and simplify a large and diverse field of research, and serve to emphasize the importance that developmental and genetic constraints play in the evolution of plant mating systems.