## Abstract

General models of the evolution of cooperation, altruism and other social behaviours have focused almost entirely on single traits, whereas it is clear that social traits commonly interact. We develop a general kin-selection framework for the evolution of social behaviours in multiple dimensions. We show that whenever there are interactions among social traits new behaviours can emerge that are not predicted by one-dimensional analyses. For example, a prohibitively costly cooperative trait can ultimately be favoured owing to initial evolution in other (cheaper) social traits that in turn change the cost–benefit ratio of the original trait. To understand these behaviours, we use a two-dimensional stability criterion that can be viewed as an extension of Hamilton's rule. Our principal example is the social dilemma posed by, first, the construction and, second, the exploitation of a shared public good. We find that, contrary to the separate one-dimensional analyses, evolutionary feedback between the two traits can cause an increase in the equilibrium level of selfish exploitation with increasing relatedness, while both social (production plus exploitation) and asocial (neither) strategies can be locally stable. Our results demonstrate the importance of emergent stability properties of multidimensional social dilemmas, as one-dimensional stability in all component dimensions can conceal multidimensional instability.

## 1. Introduction

From humans to microbes, social dilemmas (Sachs *et al*. 2004; Lehmann & Keller 2006; West *et al*. 2007) interact and intertwine, with individuals simultaneously constructing, defending and exploiting a diverse array of shared public goods. The most well-known areas of interacting social traits concern investments in cooperation and in the enforcement of cooperation (via various mechanisms, termed punishment, policing or sanctions; Frank 2003; Gardner & West 2004; Fehr & Gächter 2002). For example, in many species of eusocial hymenoptera, worker insects must navigate interacting investment decisions over whether to forgo direct reproduction (a potentially altruistic act, yielding reproductive costs to the actor and benefits to the hive) and whether to eat the ‘illegal’ eggs of other workers (a policing act, reducing the cost/benefits of worker egg-laying). A number of theoretical studies predict that policing is favoured under conditions of low relatedness, acting to maintain cooperation via enforcement; apparently altruistic traits become favoured in the presence of policing, as the alternative selfish actions are associated with punitive sanctions. As a result, cooperation can even increase with decreasing relatedness (Frank 1995, 2003; Wenseleers *et al*. 2004). Comparative studies of worker laying and policing in ants, bees and wasps support these conclusions (Wenseleers & Ratnieks 2006).

Another established field of study of multiple social trait evolution concerns altruism and dispersal. A large body of theory highlights the dependence of altruism on levels of dispersal, mediated both by relatedness and local competition (Hamilton 1970; Taylor 1992; West *et al*. 2002; Gardner & West 2006; Platt & Bever 2009). In return, dispersal can be shaped by levels of altruism, mediated in particular by habitat saturation (Hamilton & May 1977; Le Galliard *et al*. 2003). Theoretical models allowing for the joint evolution of both altruism and dispersal have predicted a range of outcomes, spanning both positive and negative correlations among the two traits (Perrin & Lehmann 2001; Le Galliard *et al*. 2005) and even the emergence of multicellularity (Pfeiffer & Bonhoeffer 2003; Hochberg *et al*. 2008). Finally, the evolution of discriminatory altruism has also been approached in a multi-trait context, with conditional helping behaviours co-evolving with potentially discriminatory markers or ‘tags’ (Axelrod *et al*. 2004; Jansen & van Baalen 2006; Rousset & Roze 2007).

Whereas cooperation/enforcement, altruism/dispersal and altruism/discrimination have received significant attention and generated intriguing results (and further specific examples of social trait pairings have been studied, e.g. cooperation and signalling, Brown & Johnstone 2001; belligerence and bravery, Lehmann & Feldman 2008), there has been to our knowledge no general study of social evolution in multiple dimensions. Here we treat multidimensional dilemmas as a general social phenomenon, and ask in the most general terms, what happens if social evolution occurs in a multi-dimensional trait space? We present a flexible model framework for the study of any multidimensional social dilemma, among relatives or non-relatives.

Our approach is to construct an adaptive dynamics system (Dieckmann & Law 1996) based on the inclusive fitness effects of the different traits. With the simplest possible assumptions (for example, traits determined at a single locus, linkage equilibrium, weak selection, additive gene action and small mutation rate) the inclusive fitness effect provides a measure of allele frequency change (Rousset & Billiard 2000; Taylor *et al*. 2007), and the system can reasonably provide the evolutionary trajectories of the set of traits under the action of selection. When these assumptions fail (as is almost always the case), the dynamical equations become complex and intractable (see Roze & Rousset 2008; for a multilocus model of social behaviour that incorporates recombination) so that our system is put forward as a simple approximation to actual evolutionary trajectories.

We illustrate our framework with a number of specific examples (the production and exploitation of a public good; the production of multiple public goods; investment in multiple selfish traits), revealing a number of surprising results owing to multi-trait feedbacks (e.g. more selfishness at higher relatedness; positive runaway public goods production at low relatedness; ‘either-or’ bistable interactions among selfish traits at low relatedness). A general result is that in two-dimensional models, Hamilton's classic rule (Hamilton 1964) for selection on each trait separately (appendix A, equation A 4) requires a further ‘interaction’ condition (appendix A, equation A 6) to adequately describe the two-dimensional system dynamics.

## 2. Results

### (a) Production and exploitation

We first illustrate our approach using a generic model of the joint evolution of the production and exploitation of a public good. For example, humans can share pastures, simultaneously enhancing them with the addition of fertilizer and exploiting them with the addition of livestock (the exploitation trait being the focus of the classic ‘tragedy of the commons’ of Hardin 1968). Analogously, microbes can enhance supplies of accessible extracellular iron through the secretion of scavenging siderophore molecules to bind insoluble ferric Fe(III), and then exploit this newly accessible iron with the expression of appropriate surface receptors to uptake iron bound to siderophores (Crosa *et al*. 2004; Griffin *et al*. 2004).

Consider two social traits: a cooperative, constructive trait *x* that builds a shared public good *G* at a direct cost to the actor (e.g. investment in fertilizer or siderophores), and a selfish, exploitative trait *y* that diminishes the public good and provides a direct benefit to the recipient (e.g. investment in livestock or siderophore-binding receptors). By looking at the joint evolution of *x* and *y*, we consider the simultaneous navigation of a tragedy of the commons (driven by exploitation, *y*) and a public goods or collective action dilemma (driven by contributions, *x*). A fitness function of the form *W*(*x*_{0}, *y*_{0}, *X, Y*) = *y*_{0} *G*(*X, Y*) − *C*(*x*_{0}, *y*_{0}) captures these relationships, where *x*_{0} and *y*_{0} are the focal individual's level of production and exploitation of the public good, and *X* and *Y* are the same traits averaged across the focal individual's group or neighbourhood. The direct benefits of exploitation are proportional to the density *G* of the public good, which in turn depends on the prevailing local levels of production and exploitation, while *C* captures the direct costs of investment in *x* and *y* (the assumption that exploitation is an active, costly process is necessary to generate intermediate levels of investment in exploitation).

Hamilton's rule (appendix A, equation A 4) offers a simple heuristic for understanding when selection would favour an increase in either trait, based on the inclusive fitness effects *H* and *K* of increases in *x* and *y*, respectively. Specifically, given within-group relatedness *R* and no genetic correlation between *x* and *y*, the inclusive fitness effects of an increase in *x* and in *y* can be calculated as:
2.1a
and
2.1b
with all derivatives evaluated at *x*_{0} = *X* = *x* and *y*_{0} = *Y* = *y* (inclusive of fitness effects calculated via the neighbour-modulated fitness technique; see Taylor & Frank 1996; Wenseleers *et al*. in press; appendix A, equations A 1–A 2). Hamilton's rule predicts that, considered as independent traits (holding other traits invariant), high relatedness should favour both cooperative production (high *x*) and restraint (low *y*); conversely, low relatedness should favour both free-riding (low *x*) and rapaciousness (high *y*), reflecting two distinct ways to be a social cheat (Brown *et al*. 2002). However, this independence assumption fails under many realistic scenarios.

Figure 1 illustrates the joint dynamics of public goods investment *x* and exploitation *y* for an explicit form of the above model, chosen to demonstrate the qualitative features that can emerge owing to simple ecologically mediated fitness interactions among social traits. Figure 1*a* illustrates two joint equilibria (where the nullclines intersect), one stable and the other unstable owing to the strong trait interdependence at this point (changes in *x* have too large an impact on the equilibrium value of *y*, and *vice versa*). Condition A 6 in appendix A formalizes and generalizes this distinction into a threshold condition for the stability of a social equilibrium in two dimensions. This condition has the form *H _{x} K_{y}* >

*H*(subscripts denote partial derivatives:

_{y}K_{x}*H*= ∂

_{x}*H*/∂

*x*, etc.). Here,

*H*and

_{x}*K*measure the strength of selection on each trait at equilibrium. In a sense they measure the restoring force for departures from equilibrium in

_{y}*x*and

*y*. On the other hand the cross-terms

*H*and

_{y}*K*measure the ‘disturbing’ effect of each variable on the equilibrium of the other. More precisely, the quotient −

_{x}*H*/

_{y}*H*measures the effect of changes in

_{x}*y*on the equilibrium value of

*x*(

*x**). For example, if −

*H*/

_{y}*H*= 2, a small increase in

_{x}*y*will cause an increase in

*x** of twice the amount. Similarly, the quotient −

*K*/

_{x}*K*measures the effect of changes in

_{y}*x*on

*y**. Condition A6 in appendix A can be interpreted as requiring that the joint restoring force exceed the joint disturbing effect. Worked examples are provided in the supporting online materials.

The existence of the unstable equilibrium in figure 1*a* introduces bistability into the system dynamics. Depending on the initial conditions of *x* and *y*, evolution proceeds towards either a stable intermediate level of both production and exploitation, or towards no social interaction with both *x* and *y* driven to zero (a similar result is implicit in Gardner & West 2004, where the condition for maintenance of punishment and cooperation is less strict than the condition for invasion of punishment and cooperation). This ‘asocial’ boundary outcome is locally stable, since when a public good is not exploited (*y* = 0), there is no benefit to its production, and when there is no production of the public good (and it is not sufficiently generated by any independent process), there is no reward for investment in the mechanisms of exploitation. The more ‘social’ outcome of intermediate *x* and *y* is also locally stable because of the same synergistic interactions—when *x* is large, there are significant rewards to positive *y*, and *vice versa*. As a consequence of the bistability in figure 1*a* (and more generally, whenever any social equilibrium fails the stability test in condition A6), there will be regions of parameter space in which a small change in initial conditions (for example in either siderophore production or reception) can lead to a large change in the final evolutionary outcome—a highly social endpoint on the one hand, and an asocial one on the other.

Siderophore-mediated social interactions can have a particularly strong synergy owing to the obligate co-dependence of siderophore molecules and siderophore receptors (each are liabilities in the absence of the other—receptors are a particular liability in the absence of siderophores, as they then serve only to uptake toxic bacteriocins; Crosa *et al*. 2004). If, however, the public good is also generated in significant quantities by other independent forces, then the basin of attraction for the asocial endpoint can disappear (figure 1*b*)—for instance it may pay to place sheep on a pasture (positive exploitation), even if it has never been fertilized (zero production).

The stable and unstable evolutionary equilibria given high relatedness (*R* = 0.9) are illustrated in figure 1*a* by filled and open circles, respectively. In figure 2*a* we track these equilibria (*x**, *y**) as a function of relatedness *R*, varied from 0 to 1. As expected from an analysis of the *x* trait in isolation, increasing *R* leads to an increase in the stable equilibrium *x**. However, contrary to the analysis of *y* in isolation, increasing *R* also leads to an increase in the stable equilibrium level *y** of selfish exploitation; that is, we see maximal selfishness at high relatedness. The reason for this effect is again the synergistic coupling between *x* and *y*—less production at low *R* means less public goods *G* and therefore less benefit from investments in exploitation. As *R* decreases from 1, the stable intermediate equilibrium declines, the threshold separating asocial and social outcomes (dotted lines in figure 2*a*) rises, and the zone of attraction for the asocial outcome increases. At a critical value of *R* (near *R* = 0.75 in figure 2*a*), a catastrophic transition occurs as the attracting and repelling equilibria merge, and for lower *R* only the asocial outcome remains (e.g. figure 1*c*). Thus, under conditions of low relatedness we see no selfish behaviour (*y** = 0), as there is no constructive behaviour (*x** = 0) to socially parasitize. If independent sources of public goods provision are sufficiently high (figure 1*b*, e.g. the pasture can grow by itself), then the joint-stable selfish exploitation *y** can peak at intermediate or low relatedness (thus approaching the classical one-dimensional prediction), reflecting the weakening of the evolutionary feedback between *x* and *y* (figures 1*d* and 2*b*).

### (b) Synergistic and antagonistic interactions among social traits

In general, one-dimensional social analyses fail whenever there are fitness interactions among social traits. For social traits *x* and *y*, this is the case whenever the inclusive fitness effect of one trait is a function of the other (i.e. *H _{y}* ≠ 0 or

*K*≠ 0; see appendix A). In this case, Hamilton's cost and benefit terms (appendix A, equation A 3) become co-evolving dynamical variables, and thus neither of the one-trait Hamilton's rules (appendix A, equation A 4) can be properly assessed except in the context of the other. When an increase in one variable leads to an increase in the inclusive fitness effect of the other, we say we have evolutionary synergy (

_{x}*H*and

_{y}*K*are positive), and this is seen to be the case in figure 1. In contrast, when an increase in one variable leads to a decrease in the inclusive fitness effect of the other, we say we have evolutionary interference (

_{x}*H*and

_{y}*K*are negative).

_{x}Figure 3 provides an illustration of neutral, synergistic and interference forms of interaction among social traits, with the example of investment in two separate public goods. When the two public goods are entirely independent in both their costs and benefits (*H _{y}* and

*K*= 0; figure 3

_{x}*a*,

*d*), we see that the two-dimensional evolutionary equilibrium remains stable, and behaves entirely as expected from separate, one-dimensional treatments (the ESS level of public goods investment increases with relatedness). In contrast, when

*H*and

_{y}*K*are non-zero, more interesting dynamics occur. For instance, consider a scenario where the benefits of the public goods only accrue through their synergistic interaction (

_{x}*H*and

_{y}*K*> 0); when produced in isolation they confer only further costs. This scenario is consistent with the extracellular production of co-enzymes and is illustrated in figure 3

_{x}*b*,

*e*. As for figure 1, we see that synergistic interactions among social traits can produce bistability between an asocial boundary condition (investment in neither trait) and the mutual production of both traits (in this example, an unbounded runaway), illustrated in figure 3

*b*for

*R*= 0.8. In figure 3

*e*we then show how this unstable equilibrium (separating runaway processes to joint-zero or join-maximal public goods provision) varies with relatedness. At low relatedness, the positive runaway remains attainable, but the threshold to entering this runaway becomes increasingly high as relatedness decreases. Finally, we illustrate a case of interference (

*H*and

_{y}*K*< 0; figure 3

_{x}*c*,

*f*), where each public good is separately useful yet costly in combination. The resulting dynamics mirror the following treatment of interfering selfish traits (figure 4); however, here we see that for interfering altruistic traits, the effects of interference are increasing with relatedness, so that the bistability (convergence to investment only in

*x*, or only in

*y*, with the outcome dependent on initial conditions) emerges at high relatedness.

Our illustrative models so far have been ‘open’ in that we have chosen the value of relatedness independent of underlying demographic processes. Models in which relatedness emerges from demographic parameters such as deme size or dispersal rates (Taylor 1992; Rousset 2004; Lion & van Baalen 2008) are ‘closed’, and our final example is of this type. The open approach has allowed us a measure of flexibility in observing model behaviour with, for example, variable relatedness, but it is important to see the analysis at work in a particular population structure, and our final example provides a discussion of this. While the ‘open model’ approach sacrifices some ecological realism, it gains in terms of generality—allowing comparisons across diverse demographies—and in terms of tractability (Taylor & Frank 1996; Frank 1998; Gardner & West 2006). Clearly, the two approaches are highly complementary, and it remains to be seen what further insights can be generated via closed models of multiple social trait evolution under specific demographic regimes.

Our final worked example presents a two-dimensional generalization of Frank's (1994) model of the evolution of selfishness in an infinite island model with demes of size *n* and non-overlapping generations (Wright-Fisher demography). This well-known model traditionally works directly with fitness, assumed to have the form , where *x*_{0} is focal selfishness and *X* is average selfishness in the focal deme. In our structured version of this model it is more appropriate to suppose that behaviour affects only fecundity and let the population structure determine mortality through offspring recruitment. Thus, in a one-trait model, we would take fecundity *F* to have the classic form , with other components of fitness *W* determined by the population structure. In fact, because of the internal symmetry of the island model, a simple general result (Taylor 1992; P. D. Taylor, D. Cownden & T. Lillicrap 2009, personal communication) shows that the inclusive fitness effect *H*(*x*) of the behaviour *x* is independent of offspring dispersal patterns and relatedness, and has the same sign as the direct effect of focal behaviour on focal fecundity:
2.2
the second term providing the focal individual's share of the effect of average deme behaviour on focal fecundity. Although relatedness in this model is *not* equal to 1/*n*, but depends both on deme size *n* and offspring dispersal rate *d*, we see that 1/*n*, in formula (2.2), plays a role analogous to relatedness.

In our two-dimensional generalization of this model (figure 4), a common resource (e.g. space) is exploited in different ways for two distinct selfish purposes (e.g. acquisition of food and shelter). The most interesting behaviour occurs when cross-inhibition is more severe than self-inhibition (i.e. the social cost of *x* is more inhibiting to direct benefits of *y* than is the cost of *y* itself, and *vice versa*; figure 4*a*,*c*). Under these conditions of maximal trait interaction (here, interference) we again see the destabilization of equilibria (appendix A, equation A 6) that are independently stable in each constituent dimension.

However, now the destabilization occurs under conditions of low relatedness, which in this explicit demography can correspond to larger deme sizes. Specifically, we find that in very small demes both traits can coexist at low levels (figure 4*c*), but in larger demes one trait dominates (excluding the rival trait), with the possibility of bistability (e.g. *n* = 5; figure 4*a*). As the importance of cross-inhibition is reduced (figure 4*b*,*d*), we see greater equilibrium stability (e.g. *n* = 5; figure 4*b*), yet interference is still manifested in the maximization of selfish trait *y* at intermediate deme sizes (figure 4*d*) and intermediate relatedness (electronic supplementary material, fig. S2).

## 3. Discussion

Our analysis presents a simple and general framework for the study of the joint evolution of multiple social traits. We demonstrate the importance of analysing a complete set of stability conditions for multidimensional social dilemmas, as separate stability in each component dimension can conceal multidimensional instability. For the case of two interacting traits, we derive a two-dimensional stability criterion dependent on relatedness (appendix A, conditions A 6–A 7) that can be viewed as a multidimensional extension of Hamilton's rule. We offer a number of examples to illustrate the general stability condition. Our general theoretical framework now raises the vital empirical challenge of understanding the nature of interactions among multiple social traits.

The synergistic interaction between matching production and exploitation traits (figures 1 and 2) is likely to be mirrored across many interacting social traits. In the case of two synergistic productive traits, we can see an analogous threshold between mutual restraint and mutual production, illustrating again the potential for bistable dynamics given multiple social dimensions (figure 3*b*). The bistability between the presence of multiple social traits and the presence of none is consistent with the co-inheritance of multiple virulence (exoproduct) genes in pathogenicity islands (Hacker & Kaper 2000; Nogueira *et al*. in press) and the co-regulation of multiple public goods traits in quorum-sensing bacteria (Miller & Bassler 2001), highlighting the potential biomedical relevance of understanding multiple social trait evolution in bacteria (Brown *et al*. in press).

Our focal example of the joint evolution of a production and exploitation trait could be readily extended to consider further interacting traits. For instance, investments in the production and subsequent exploitation of a shared public good (figures 1 and 2) are likely to interact with investment in defence of the public good (in a microbial setting, investment in allelopathic traits such as bacteriocins; Brown *et al*. 2009). When there are more than two interacting traits, the system and the analysis becomes more complicated. Given a stable asocial boundary (all traits constrained by selection to zero), the addition of further social dimensions trivially diminishes the likelihood that the asocial limit remains stable (simply as there are more ways to be social). More subtly, as dimensions are added, the likelihood that any particular dimension becomes social (i.e. non-zero) increases, as evolution in an additional dimension can potentially favour the elaboration of the initially constrained focal trait (for instance, in figure 1*b*, the focal dimension *x* is constrained to zero when *y* = 0; however, unilateral elaboration of *y* from zero subsequently favours elaboration of *x*). This illustration echoes and generalizes Sober & Wilson's (1998) argument that any mechanism that leads to the emergence of affordable punishment will allow the elaboration of a more costly, and therefore previously constrained, cooperative trait (Sober & Wilson 1998, p. 144).

Following escape from the asocial boundary, predicting the ultimate system fate becomes more complex with increasing dimensionality. As the number of traits increases, the conditions for stability of any equilibrium become more severe, but the number of equilibrium points also generally increases. If the costs of large values of all traits are prohibitive, the dynamical system will stay bounded and has to go somewhere (converging to a stable point or cycle, or exhibiting chaotic behaviour) and the ecological systems that survive today are generally those that have arrived at a stable convergence. It is likely that such convergence has often wound up at the boundary on which one or more potential traits have been lost (although synergistic trait interactions will tend to favour trait coexistence), so that the interesting question becomes which behaviours are maintained in a particular system. In this study we have begun to build a framework in which this question might be studied.

## Acknowledgements

We thank Angus Buckling, Andy Gardner, Sébastien Lion, Craig Maclean, Daniel Rankin, François Taddei and Stuart West for helpful discussions and/or comments on an earlier draft. We thank the Wellcome Trust (S.P.B.) and the Leverhulme Foundation (P.D.T.) for financial support.

## Appendix A. General methodology

Let *W*(*x*_{0}*, y*_{0}*, X, Y*) be the fitness of an individual (*x*_{0}*, y*_{0},) in a neighbourhood (*X*, *Y*). Then the inclusive fitness effects of an *x*-increase and of a *y*-increase have the form:
A 1
and
A 2
where
A 3
all derivatives evaluated at *x*_{0} = *X* = *x*, and *y*_{0} = *Y* = *y* (Taylor & Frank 1996). The notation is chosen because, in models of cooperative behaviour, *C* and *B* can be regarded as a cost and a benefit. We allow the possibility that the *x-* and *y*-relatedness are different. For example, if *x* is a cooperative and *y* a selfish trait, the *x-*neighbourhood might be smaller than the *y-*neighbourhood. If *H*(*x*, *y*) (or *K*(*x*, *y*)) is positive, selection will favour higher values of *x* (or *y*). We write this as
A 4
and we obtain the form of Hamilton's (1964) classical rule for each trait separately. There is an assumption here of independence in *x* and *y*; that is, no genetic correlation between them (individuals with high or low values of one trait are neither more nor less likely to carry high or low values of the other). Given this assumption, the condition for a point (*x**, *y**) to be an *evolutionary equilibrium* is that *H*(*x*, *y*) and *K*(*x*, *y*) be zero.

We now turn to the question of the stability of such an equilibrium: what will happen to small departures from (*x**, *y**)? The classical condition of convergence stability (Christiansen 1991) is that *H _{x}* < 0 and

*K*< 0 at (

_{y}*x**,

*y**), where the subscript denotes partial derivative. In this case, both

*x*and

*y*will be separately stable with the other variable held fixed; that is, if the population-wide values of

*x*or

*y*depart from equilibrium, selection will favour those values closer to the equilibrium.

To account for the joint effect of departures in both traits, we consider the dynamical system
A 5
modelled after the standard adaptive dynamics (Leimar in press). Note that a stationary point of the system (*H* = *K* = 0) will be a point of evolutionary equilibrium. For such a point to be dynamically stable (Otto & Day 2007), we first require that *H _{x}* < 0 and

*K*< 0, and these are the conditions (above) of separate convergence stability, but we also require the additional condition

_{y}*H*>

_{x}K_{y}*H*that the linearized system have positive determinant. If this fails, departures from equilibrium in one variable can destabilize the other. This condition can be written A 6 where and (as above) the second subscript denotes partial derivative. When the two interaction neighbourhoods are the same,

_{y}K_{x}*R*=

_{x}= R_{y}*R*, condition (A 5) becomes A 7 where

*b*=

*b*+

_{x}*b*.

_{y}## Footnotes

- Received August 17, 2009.
- Accepted September 22, 2009.

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

- © 2009 The Royal Society