Comparative studies of social insects and birds show that the evolution of cooperative and eusocial breeding systems has been confined to species where females mate completely or almost exclusively with a single male, indicating that high levels of average kinship between group members are necessary for the evolution of reproductive altruism. In this paper, we show that in mammals, the evolution of cooperative breeding has been restricted to socially monogamous species which currently represent 5 per cent of all mammalian species. Since extra-pair paternity is relatively uncommon in socially monogamous and cooperatively breeding mammals, our analyses support the suggestion that high levels of average kinship between group members have played an important role in the evolution of cooperative breeding in non-human mammals, as well as in birds and insects.
Most female mammals provision and rear their own young but, in a small proportion of species, breeding females either pool their young and share care and provisioning (communal breeders) [1,2], or are assisted in protecting and feeding their offspring by non-breeding helpers (cooperative and eusocial breeders) [3,4]. Cooperative and eusocial breeding systems are of particular interest to evolutionary biologists as it is necessary to explain why adults should forego breeding and raise young other than their own. In many of these systems, few individuals that disperse from their natal group survive or establish new breeding units with the result that female recruits commonly remain in their natal group, where they are frequently precluded from breeding by dominants [4,5]. Explanations of the evolution of cooperative behaviour based on kin selection suggest that, under these conditions, non-breeders may maximize their inclusive fitness by assisting dominant relatives to rear their young [5–7]. However, an alternative view is that cooperative breeding can arise through group selection operating between groups of individuals who do not need to be close kin [8,9].
In line with explanations based on kin selection, the evolution of eusocial breeding in insects appear to have been restricted to species with strictly monogynous mating , where coefficients of relatedness between helpers and young receiving help approximate to 0.5, so that relatedness between them is equivalent to that between parents and offspring . In addition, comparative studies of the evolution of cooperative breeding in birds show that cooperative rearing is associated with mating systems where the frequency of extra-pair paternity is low and relatedness between the offspring of established breeding females is relatively high .
Among mammals, most species have polygynous mating systems and the tenure of breeding males is often short  so that the proportion of group members that are full siblings and coefficients of relatedness between group members are relatively low . However, around 5 per cent of mammals are socially monogamous  and, in these species, the frequency of extra-pair paternity is relatively low and the breeding tenures of males are relatively long [14,15] so that coefficients of relatedness within and between litters are relatively high. Cooperative breeding systems are uncommon in mammals, but are widely distributed and have been documented in the Canidae (dogs), the Herpestidae (mongooses) and the Callitrichidae (New World monkeys), as well as in several families of rodents, including the Bathyergidae (mole-rats), Castoridae (beavers), Hystricidae (porcupines), Muridae (mice and rats) and terrestrial Sciuridae (squirrels). In this paper, we investigate whether the evolution of cooperative breeding has been restricted to socially monogamous mammals, or whether it also occurred among the more common polygynous ones.
2. Material and methods
(a) Classification of social system
For the purpose of our analyses, we have adopted a conservative definition of cooperative breeding. We include species as cooperative breeders only if a proportion of females do not breed regularly and show alloparental care (such as contributing to provisioning or carrying young born to other females) [3,4,11]; as communal breeders, if most adult females breed regularly and share care such as allonursing or feeding offspring ; and as social breeders, if females live in groups and are neither cooperative or communal breeders. Our definition of cooperative breeding consequently excludes some species that are sometimes regarded as cooperative breeders, including species, like alpine marmots , where there is no evidence that females modify their behaviour in order to benefit juveniles born to other females and others, like banded mongooses , where breeding females are assisted by males whose contributions to cooperative rearing could represent a form of parental investment or courtship [18,19]. Information on the occurrence of cooperative and communal breeding was collected from relevant reviews, as well as during our systematic search of the available literature on social systems (see below). We were able to identify 34 mammal species as cooperative breeders and 23 species as communal breeders.
For the comparison of contemporary species, values for the proportion of all offspring born in the group that were produced by the most dominant female were collected for cooperative, communal and social species from the available literature on wild populations. Information on genetic estimates of paternity was extracted from Soulsbury 2010 . Relative male tenure length was calculated by taking the maximum values of alpha male tenure that have been reported for the respective species from a wild population and dividing it by the interbirth interval (see the electronic supplementary material, table S1 for values and references). We investigated whether levels of female and male reproductive skew differ between social and cooperative breeders using Mann–Whitney tests. Comparisons controlling for phylogenetic relatedness were performed using the function ‘pglmEstLambda’ of CAIC . All analyses were performed in the statistical software R v. 2.11.1 .
To establish our database, we searched for information on the breeding system for every species listed in the ‘Mammal Species of the World’ . We obtained information for 1874 species from entries in ‘Walker's Mammals of the World’  and by typing species names into ISI Web of Science and Google Scholar to browse references (see the electronic supplementary material, table S2 for full list). We classified species as solitary breeders if females live alone in an individual territory or range and male ranges include several female ranges (1270 species); as socially monogamous, when groups include a single breeding adult of each sex (184 species); and as polygynous or polygynandrous social breeders if groups include several breeding females and one or more breeding males (420 species). For convenience, we refer to polygynous/polygygnandrous species as polygynous. Genetic determination of paternities is available for relatively few species but those available confirm that, in contrast to birds, social monogamy is associated with high paternity certainty and genetic monogamy while, in species where groups include several breeding adults of both sexes, females commonly mate with more than one male ([14,15]; also see §3). For phylogenetic reconstructions, we used the recently published supertree for mammals , including dated branch lengths. The tree was pruned to match the species in the dataset using functions of the package ‘APE’  in R. Reconstruction of transitions to cooperative breeding based on parsimony, using a majority rule of the three (or more) branches connecting to a node and reconstruction based on the maximum-likelihood approach for discrete characters  gave the same result. We used the module DISCRETE  of BayesTraits  to simultaneously reconstruct the evolution of cooperative breeding and monogamy. This approach reconstructs the most likely scenario if two traits evolved independently in the phylogenetic tree and compares this with a scenario in which the two traits are linked and transitions occur between the four possible combinations. To account for different possible modes of evolution, separate analyses were performed using transformations of the dated tree, creating equal and ultrametric branch lengths . We also allowed the rate of character change to vary for different parts of the tree. Likelihoods for all different models were estimated using 25 optimization attempts per run. Significance of results was measured by performing a likelihood-ratio test, comparing twice the difference between the independent and dependent models with a chi-square test having 4 d.f. to reflect the higher number of parameters in the dependent model. Models assuming a dependent model were considered significant at α = 0.05 if the respective likelihood was 4.75 lower than the comparable independent model .
(a) Reproductive skew and cooperative breeding
In cooperative breeders, a single female monopolizes reproduction and is responsible for over 90 per cent of breeding attempts (n = 26 species, median = 100%, range 88–100%); species where reproduction is shared between plural breeding females were notably absent. In species for which genetic data were available, a single male fathered a substantial majority of the young born to the dominant female (n = 13 species, median = 88%, range = 76–100%), though subordinate females bred with other males. The tendency for a single female to monopolize reproduction in cooperative breeders contrasts with similar data for communally and socially breeding mammals, where the most dominant female is rarely responsible for more than 30 per cent of recruits (n = 20 species, median = 20%, range 8–69%; figure 1a; t20 = 16.7, p< 0.001). In addition, in communal and social species, the most dominant resident male rarely fathers more than 75 per cent of offspring born in the group (n = 30 species, median = 54%, range 5–100%; figure 1b; t41 = 6.3, p < 0.001). In most cooperative breeders, the tenure of dominant males spans several breeding seasons, so that the turnover rate of breeding males is low and recruits born in successive breeding seasons are often full-siblings (n = 7 species, median = 4 seasons, range 3.4–6.5), whereas in social breeders, the turnover of dominant males is higher and the proportion of full-siblings is lower (n = 29 species, median = 2.3 seasons, range 1.0–5.4; figure 1c; t9 = 3.57, p = 0.006). Differences between cooperative and social breeders remain statistically significant after controlling for phylogeny (female reproductive dominance: n = 41 species; fit to phylogeny: Lambda statistic 0.97, F = 250.55, p < 0.001; male reproductive dominance: n = 38 species; fit to phylogeny: Lambda statistic < 0.001, F = 21.17, p < 0.001; male tenure length: n = 36 species; fit to phylogeny: Lambda statistic 0.46, F = 11.51, p = 0.002; sample sizes are smaller as some species are not included in the supertree).
(b) Evolutionary transitions to cooperative and communal breeding
Phylogenetic reconstructions suggest that there have been 14 transitions to cooperative breeding from other breeding systems; nine of these occurred in the rodents (genera Cryptomys, Heterocephalus, Microtus, Meriones, Rhabdomys, Castor, Atherurus and two in Peromyscus), four in the carnivores (Alopex, Canis, Lycaon and in the mongooses) and one in the primates (Callitrichidae). In no case is it likely that cooperative breeding evolved from a plural-breeding ancestor with a polygynous mating system. Eight of the 14 transitions to cooperative breeding occurred in ancestors that are likely to have been socially monogamous while, in the remaining six cases, the change to cooperative breeding occurred on the same branch as the change to monogamy and probably involved a transition from ancestors where females were solitary and territorial. Results from the Discrete modelling approach indicate that a model assuming that the evolution of social monogamy and cooperative breeding are linked is always significantly more likely than models assuming that they evolved independently (log-likelihood of respective best models assuming equal branch lengths and rate variation: dependent model, −428.15 versus independent model, −457.19; see the electronic supplementary material, table S3 for full list). All dependent models (independent of branch length transformation and transition rate variation) indicate that cooperative breeding has never evolved in mammals which were not socially monogamous, and that cooperative breeding without monogamy is unstable (figure 2).
Our data indicate that there may have been at least 18 transitions to communal breeding. In 12 of these, communal breeding is likely to have evolved from a plural-breeding ancestor with a polygynous mating system. In at least two cases (banded mongooses and Goeldi's monkey), a communal breeding system appears to have evolved from a cooperatively breeding ancestor.
Our analyses of the distribution of cooperative breeding in mammals confirms previous evidence of a close association between cooperative breeding and monogamous or monogynous mating systems in other animals [10,11]. Among contemporary mammals, cooperative breeding (as defined above) is almost totally restricted to species where a single female virtually monopolizes reproduction and breeds with a single dominant male. In addition, phylogenetic reconstruction shows that transitions to cooperative breeding have been restricted to lineages characterized by monogamy, whereas communal breeding systems have commonly evolved in lineages characterized by polygynous mating systems. Our results support the widespread view that relatively high levels of kinship between group members are a necessary condition for the evolution of cooperative breeding [31,32] and provide no evidence that transitions to cooperative breeding ever occur in species with polygynous mating systems, or where multiple females breed per group, as has been suggested [8,9].
While the association between cooperative breeding and relatively high relatedness between group members suggest that kin selection plays an important role in the evolution and maintenance of cooperative breeding, this should not be taken to suggest that direct fitness benefits do not also play an important part in maintaining cooperative and eusocial breeding systems, for there is both theoretical  and empirical evidence  that they do so. It is important to distinguish between the different components of cooperative breeding systems for they are often confused  and the relative importance of kin selection probably differs between them. Cooperative and eusocial systems have four main characteristics: the formation of stable groups of females; the retention of offspring in their natal group for part or all of their lives; the suppression of reproduction in subordinate females; and support in rearing young produced by the dominant females by non-breeding subordinates. The formation of stable female groups is common in social mammals and often generates substantial direct fitness benefits for the survival or breeding success of group members [4,5]. As would be expected if female sociality can be explained by direct fitness benefits alone, female groups consist of unrelated individuals in some social animals . There is also no need to invoke indirect benefits to account for the reluctance of subordinate females to disperse from their natal group, or for the evolution of reproductive suppression . In many social animals, the costs of dispersal are high and subordinate females may maximize their fitness by remaining in their natal group and queuing for the breeding position rather than attempting to disperse to nearby groups even if they are temporarily prevented from breeding by dominant females [4,37]. Suppression of reproduction in subordinate females appears to occur where the breeding success of dominants would be jeopardized by successful breeding by subordinates . In social species, the extent to which dominant females tolerate subordinate reproduction varies within groups in relation to factors likely to be related to the cost of subordinate breeding to the fitness of dominant breeders. For example, dominant females are more likely to evict subordinates or to cause them to abort in large groups than in small ones [39,40] and are more likely to kill pups born to subordinates if they are pregnant themselves than if they are not . However, suppression of sexual development in subordinate females is common in cooperative breeders, and is rare in other species, whether groups consist of related or unrelated females, suggesting that indirect fitness benefits may play a role in the evolution of suppression.
Of the four characteristics of cooperative breeders, only the evolution of regular assistance in rearing young by non-breeding females is difficult to explain without invoking indirect benefits and kin selection and, even here, it is possible to envision scenarios where the evolution of cooperative breeding does not require high levels of kinship between group members [9,42,43]. However, evidence that all forms of cooperative breeding between females are rare in species where female group members are usually unrelated [44,45] and that the evolution of cooperative breeding systems has been restricted to species where female group members are close relatives ([10,11], this study), suggest that kin selection plays an important role in the evolution and maintenance of reproductive cooperation.
Our results also have implications for the evolution of reproductive cooperation in humans. The breeding systems of many human societies appear to combine elements of both communal breeding (shared care of offspring by breeding females) and cooperative breeding (assistance provided by post-reproductive grandmothers) . Both cooperative and communal breeding are probably a derived trait in humans as none of the three African apes (chimpanzees, bonobos and gorillas) would be classified either as a cooperative or as a communal breeder. In addition, the breeding systems of the African apes differ from those of mammals that breed cooperatively: in all three species, females commonly disperse from their natal community, so that few mature females belonging to the same community are close relatives; several females breed in each group and polygynandrous mating is normal .
Some authors have recently argued that cooperative breeding may have facilitated the evolution of human prosociality and cognitive development [48,49]. However, while it is possible that cooperative or communal breeding developed at a relatively early stage of human evolution and facilitated the subsequent evolution of prosociality and cognitive development , the existence of reproductive cooperation does not necessarily generate an extension of cooperation to other activities, such as foraging of hunting, and there is little evidence of any close relationship between cooperative or communal breeding and cognitive development in other animals . As a result, an alternative scenario is that the evolution of a syndrome of human characteristics (including cognitive development, social learning and advanced form of communication) led to the evolution of prosociality in humans, which encouraged the shared care of offspring by breeding females and assistance by non-breeding grandmothers.
We thank D. Blumstein and N. Solomon for critical feedback on the classification of species, and M. Bell, J. Boomsma, M. Cant, A. Manica, S. Sharp, A. Thornton and C. Spottiswoode, and the reviewers for feedback and discussions. The Leverhulme Trust and the Isaac Newton Trust provided funding for this study. The authors declare no competing financial interests.
- Received November 24, 2011.
- Accepted January 5, 2012.
- This journal is © 2012 The Royal Society