## Abstract

In addition to their nuclear genome, the vast majority of eukaryotes harbour cytoplasmic genomes, e.g. in mitochondria or chloroplasts. In the majority of cases, these cytoplasmic genomes are transmitted maternally only, leading to selective pressures divergent from those that act on nuclear genes. In particular, cytoplasmic genes, which reduce the fitness of males that carry them, but have no fitness effect in females, are believed to be selectively neutral. Here, we go a step further and argue that in outbreeding populations (i.e. populations with inbreeding avoidance), ‘spiteful’ cytoplasmic elements that reduce the number of offspring produced by males are in fact selected for. We study this process by means of a stochastic model, analysing both the probability of spread and the impact that such a spiteful cytotype can have on population dynamics. Our results demonstrate that the probability of spread of the spiteful cytotype can be several times higher in outbreeding than in panmictic populations. Spread and fixation of the spiteful cytotype can lead to different qualitative effects on the population dynamics, including extinction, decreased or increased stable population size. We discuss our results in respect to cytoplasmically induced male infertility and cytoplasmic incompatibility.

## 1. Introduction

According to their mode of inheritance, eukaryotic genomes can be divided into two fractions. Whereas the majority of genes are located in the nucleus and usually show Mendelian inheritance, a smaller fraction of genes are located in the cytoplasm and are passed on predominantly through the maternal line. These cytoplasmic genetic elements include mitochondria, chloroplasts and inherited micro-organisms.

The different modes of inheritance give rise to diverging selective pressures that act on nuclear and cytoplasmic genes: whereas nuclear genes are selected to produce healthy and sexually successful males, the production of males is wasteful from the cytoplasmic genes' ‘point of view’. This gives rise to various manifestations of intragenomic conflict (Cosmides & Tooby 1981), the empirical reality of which is corroborated by a wealth of phenotypes induced by cytoplasmic elements, including cytoplasmic male sterility induced by mitochondria in plants (Saumitou-Laprade *et al*. 1994), and cytoplasmic incompatibility (CI), male-killing and feminization induced by intracellular bacteria in arthropods (reviewed in Stouthamer *et al*. 1999).

While it has been recognized that cytoplasmic genetic elements with deleterious effects in males are selectively neutral (Frank & Hurst 1996), we here present the thesis that mutations with deleterious effects can even spread through natural selection. Consider an outbreeding population, that is, one in which individuals mate with their kin less often than expected by chance. Consider further a mutant cytoplasmic allele that causes its male carrier to produce fewer offspring than wild-type males. Mating of such a male with a female can then be expected to also reduce the number of offspring this female produces. Thus, the average offspring production of all females unrelated to the focal male is reduced, and therefore the relative fitness of the females closely related to that male is increased.

This argument can also be put in terms of ‘spite’, defined as an action that decreases other individuals' direct fitness while not increasing the actor's fitness. In theory, spiteful behaviour can increase the indirect fitness of the actor, and thus, be selectively favoured (Hamilton 1970). The crucial necessary condition for selection for spite is that on average, the recipient is less related to the actor than a randomly chosen member of the population, which is equivalent to a negative coefficient of relatedness *R* (e.g. Grafen 1985). Outbreeding provides a pre-existing system of interactions between negatively related partners that may enable spiteful cytoplasmic elements in males to evolve: by inducing spite towards females that have a higher probability of bearing the wild-type allele than close relatives, a mutant cytoplasmic allele can be expected to be selected for.

Outbreeding is defined as mating between partners that are less strongly related than randomly chosen members of the population. Outbreeding is often thought of as an adaptation to inbreeding depression and/or heterosis and can be the result of several mechanisms. For example, sex-biased dispersal before reproduction is commonly found in mammals and birds (e.g. Cockburn *et al*. 1985, 2003; Wolff *et al*. 1988; Clutton-Brock 1989; Wolff 1992). Behavioural avoidance of mating with kin is another trait that leads to outbreeding if mating is otherwise random. Behavioural inbreeding avoidance (including females not entering oestrous) is particularly common in mammals (Hoogland 1981, 1991; Dobson *et al*. 1997; Ishida *et al*. 2001; Tai *et al*. 2002), but has also been demonstrated in skinks (Stow & Sunnucks 2004), crickets (Simmons 1989, 1991) and mites (Enigl & Schausberger 2004). Finally, inbreeding avoidance may also occur after copulation. Examples include failure of females mated with close kin to fertilize their eggs, found in two cactophilic *Drosophila* species (reviewed in Markow 1997), as well as reduced sperm competitive ability of related compared to unrelated males reported in a lizard (Olssen *et al*. 1996), *Drosophila melanogaster* (Mack *et al*. 2002) and crickets (Stockley 1999; Bretman *et al*. 2004). Given that it is difficult to detect inbreeding avoidance, it may be inferred from these examples that outbreeding is not rare in natural populations.

Here, we develop an individual based stochastic model by means of which we study the spread of a spiteful cytoplasmic element (‘S-cytotype’). We assume that this S-cytotype, when present in males, results in increased mortality among the offspring of these males. First, we analyse how likely the invasion of a mutant S-cytotype is. We show that the S-cytotype can indeed be selected for and derive approximations for its selective advantage and its probability of fixation in a population with sibmating avoidance. Second, we analyse the impact that the spread of the S-cytotype has on the population dynamics. This is important to determine the conditions when spread of a S-cytotype leads to extinction of the population, but also reveals other interesting effects the S-cytotype may have on the population dynamics.

## 2. The model

Our approach is an individual based stochastic model with discrete, non-overlapping generations. We consider two different cytotypes, the ‘normal’, wild-type cytotype (N-cytotype) and the spiteful, mutant cytotype (S-cytotype). Within one generation, events occur as follows. Each female in the population mates either with a randomly chosen male (panmixis scenario), or with a randomly chosen male other than one of her brothers (sibmating avoidance scenario, outbreeding). Each female then produces *m* male and *m* female eggs, all of which inherit their mother's cytotype. While eggs with an N-cytotype father then develop into juveniles, eggs with an S-type father die with a probability *λ* due to a spiteful modification of their father's sperm. In the next step, juveniles die with a probability *μ*, regardless of their cytotype. Finally, frequency dependent mortality occurs: immature adults die with a probability *N*_{im}/*K*, where *N*_{im} is the total population size of immature adults, and *K* is the carrying capacity of the population.

## 3. Invasion of the S-cytotype

How likely is the invasion and fixation of a newly arisen S-cytotype in a population? We will first present some analytic considerations that can serve as approximations for the stochastic model. In a panmictic population, the S-cytotype is neither favoured nor disfavoured by natural selection, so that its frequency is solely determined by genetic drift. We denote by *N*_{f} the number of adult females in the population and assume that this number remains constant during the spread of the S-cytotype to keep the calculations simple (but see §4). The probability of spread and fixation of a newly mutated S-cytotype in a panmictic population is then given by *u*=1/*N*_{f} (Hartl & Clark 1997).

In a population with sibmating avoidance, the S-cytotype is selectively favoured. Consider a population with *N*_{m} males, a fraction *p* of which bear the S-cytotype. A female with the S-cytotype that has *k* brothers (among the *pN*_{m} males bearing the S-cytotype) will on average have a relative offspring production of(3.1)Likewise, an N-cytotype female with *k* brothers will produce a relative number of(3.2)offspring. Note that for both types of females the offspring production is reduced and decreases with increasing frequency *p* of the S-cytotype, because both types of females mate with S-cytotype bearing males. Also, while *w*_{S}(*k*, *p*) increases with increasing number *k* of brothers of the female, *w*_{N}(*k*, *p*) decreases with increasing *k*.

We will now assume that each reproducing female in the population has exactly one reproducing brother in the population, as can be expected on average with a 1 : 1 sex ratio. The selective advantage of the S-cytotype relative to the N-cytotype can then be calculated as(3.3)It can be seen from this formula that the selective advantage *s* of the S-cytotype decreases with increasing population size, which is a general result deriving from the theory of spite (Hamilton 1970). In addition, equation 3.3 shows that *s* increases with increasing frequency of the S-cytotype, yielding positive frequency dependence. Nevertheless, the S-cytotype is favoured over the whole range of frequencies, which becomes clear when considering .

We now apply theory developed by Kimura (1962) to obtain an approximation for the probability of fixation of an S-cytotype. We will assume that the frequency of the S-cytotype in males always equals the frequency in females and denote both by *p*. Also, we will assume *N*_{f}=*N*_{m}, but retain these two variables for more clarity. Note that both of these assumptions are in accord with what we can expect on average in the stochastic model.

In general, the probability of fixation of an allele with initial frequency *p*_{0} is estimated by(3.4)where(3.5)(Kimura 1962). In this formula, *M*_{δp} and *V*_{δp} denote the mean and the variance in the change of the frequency *p* of the S-cytotype. These quantities can be approximated by(3.6a)

(3.6b)(Note that the effective population size with respect to a cytoplasmic gene equals the number of reproducing females in the population.) After solving the integrals, we get(3.7)and(3.8)

Thus, for a newly arisen S-mutant in a population with sibmating avoidance we obtain a fixation probability of(3.9)

It should be emphasized that this result can only be a coarse approximation because it relies on several simplifying assumptions. Figure 1 shows how the estimates *u* for the panmictic and the sibmating avoidance scenario depend upon the population size. As can be seen, the fixation probability increases in both cases with decreasing population size.

Figure 1 also shows simulation results of the stochastic model. For each data point, 100 000 simulations have been run. Parameters and starting conditions were chosen such as to match the assumption of the above derivation of *u* as closely as possible. In particular, in each simulation, initially one sibling pair (sister and brother) had the S-cytotype so that selection commences in the first generation. Moreover, the parameters *m*, *μ* and *λ* chosen result in a relatively stable population size, even when the S-cytotype spreads (see §4). Comparing the proportions of simulations were the S-cytotype became fixed in the population, it can be seen that while the estimate for the fixation probability in a panmictic population matches closely the simulation results, the approximation for *u* in a population with sibmating avoidance slightly overestimates the simulation results. Nevertheless, it is notable that even for the moderate mortality among offspring from S-cytotype fathers chosen (*λ*=0.33), the probability of fixation is consistently about twice as high in the sibmating avoidance scenario than in the panmixis scenario.

For figure 2, the embryo mortality *λ* that ensues from a paternal S-cytotype was varied. In each simulation, we started with a population were all individuals had the N-cytotype; we assumed that the population was at its equilibrium size (see §4), with equal numbers of females and males. We then ‘mutated’ one of the females to an S-cytotype female and simulated until one of the cytotypes became fixed or the population went extinct. For each combination of parameters, we ran 10^{6} simulations. The plot shows that in an outbreeding population, the probability of invasion of the S-cytotype increases with increasing *λ*, i.e. with increasing spitefulness of the S-cytotype. Again, this is in accord with the above considerations on selection for the S-cytotype.

## 4. Population dynamics

Having discussed the likelihood of an invasion of the S-cytotype, we will now try to understand what impact such an invasion will have on the population dynamics. We would like to stress that due to the particularities of our model assumptions on life history and density dependence, the following treatment does not claim to make any quantitative predictions for real populations. Rather, it is meant to demonstrate qualitatively the variety of effects that are possible even under the simple assumptions made. We will first attempt to gain insight into the population dynamics of the model by interpreting the mortality probabilities as deterministic mortalities, and then proceed to results of computer simulations of the stochastic model.

Consider first the case where all individuals in the population have the N-cytotype. We denote by *N* the number of mature females in the population at a given generation. The expected number of mature females in the following generation is then given by(4.1)Note that this recursion equation is essentially the well-known logistic equation. Solving the equation for *N* yields the non-trivial equilibrium population size(4.2)Obviously, this equilibrium is positive for only, otherwise, the population will go extinct. Taking advantage of previous results on the logistic equation (May 1974), it can further be shown that for , is a globally stable equilibrium (i.e. when starting from any population size , the system will converge to ). For , the system will converge to cycles with periods 2^{i}, with *i* taking increasing integer values, while for , chaotic behaviour of the system will prevail.

Consider now the case where all individuals in the population bear the S-cytotype. All offspring then suffer increased mortality, and equation (4.1) becomes:(4.3)The equilibrium population size can now be calculated as:(4.4)and it can be seen that the new condition for this equilibrium to exist (i.e. be positive), can be considerably more restrictive than for the case that all individuals bear the N-cytotype. In other words, a population that can stably exist with the N-cytotype may go extinct when the S-cytotype is fixed.

When is fulfilled, the population can again converge to a stable equilibrium or a stable cycle, or exhibit chaotic behaviour. Interestingly, a population fixed for the N-cytotype that behaves chaotically may converge to stable cycles or a stable equilibrium when the S-cytotype becomes fixed. Also, the equilibrium population size may be higher when all individuals bear the S-cytotype than when all individuals bear the N-cytotype. These different outcomes of population dynamics depending on the parameters *m*, *μ* and *λ* are summarized in figure 3, which has been derived analytically using well-known results of the logistic equation (May 1974).

For selected parameter values, we have confirmed the above considerations in the stochastic model (figure 4). For each plot, we simulated a population fixed for the N-cytotype for 500 generations, after which we introduced a single female mutant with the S-cytotype. It can be seen that spread and fixation of the S-cytotype can lead to population extinction (figure 4*a*), decreased, but stable population size (figure 4*b*), increased stable population size (figure 4*c*) or stabilization of population size (figure 4*d*).

## 5. Discussion

We have developed a model that involved a spiteful cytoplasmic genetic element that is inherited maternally only, but induces increased mortality in the offspring of males that carry it. By combining analytical considerations and simulation results, we have demonstrated that such an element can be selected for in populations where individuals avoid mating with their siblings. We have also shown that while extinction of the population is a possible result of the invasion of the spiteful cytoplasmic element, it may also lead to an increased population size and can have a stabilizing effect on the population dynamics. Taking these results together, our model suggests two possibilities that may seem paradoxical on first sight: (i) a strongly deleterious genetic element can be selected for and (ii) while detrimental for each individual that carries it, it may increase the fitness of the population.

The action of the spiteful cytotype was modelled by assuming an increased embryonic mortality in offspring of males that carry the spiteful element. However, several other phenotypes in males induced by cytoplasmic genes can be expected to lead to the same results. For example, increased mortality among the offspring of fathers bearing the spiteful cytotype may also result from decreased or absent paternal care, or from infanticide. As another example, consider a cytotype that induces a decrease in the fertility of its male host. Females that mate with such a male will on average produce fewer offspring than if they mate with normal males. This is in particular true for monogamous (or polygynous) species. More precisely, the fewer males an average females mates with during her lifetime, the stronger we expect the selection for a cytoplasmic gene that reduces male fertility to be in an outbreeding population.

To date, a ‘spiteful cytotype’ as assumed in our model has not been found in any species. There are, however, at least two closely related phenomena that guide us to think that the notion of such cytotypes is realistic. First, increasing evidence has accumulated in recent years that in humans, mutations in mitochondrial DNA exist that have detrimental effects on male fertility (Ruiz-Pesini *et al*. 2000; O'Connell *et al*. 2002; Spiropoulos *et al*. 2002; St. John *et al*. 2005). While previously, such mutations were seen as selectively neutral (Frank & Hurst 1996), the model developed here, along with the above remarks on infertility in monogamous species, suggests that such mitochondria may in fact be weakly selectively advantageous.

Second, a phenotype very similar to that of the hypothesized S-cytotype termed cytoplasmic incompatibility (CI) is widespread in arthropods. CI is induced by intracellular bacteria of the genera *Wolbachia* and *Cardinium* (Yen & Barr 1971; Hunter *et al*. 2003). In CI, the sperm of infected males is modified by the bacteria in a detrimental way. As a result, the offspring of such males show an increased mortality (up to 100%) unless the same or a similar strain of bacteria is present in the egg, in which case the modification is ‘rescued’ and the offspring survive (Laven 1951; Hoffmann *et al*. 1986). CI has previously been discussed in the context of spite (Hurst 1991; Foster *et al*. 2001; Gardner & West 2004). In terms of the modification-rescue system, the S-cytotype can be viewed as a ‘mod-only’ strain, the existence of which has been hypothesized but not proven in *Wolbachia* (Charlat & Merçot 2001). Clearly, because of the additional rescue function, CI is a much more efficient way of enhancing the fitness of a cytoplasmic genetic element than mod-only. However, the S-cytotype considered in this paper can be expected to evolve more easily, and the extant CI-inducing bacteria may well have gone through a stage in which they were not able to rescue their own modification. Thus, our model also provides a hypothesis on the evolution of CI.

To keep the model as simple as possible, we have assumed complete avoidance of matings between siblings. It could be argued that sibmating avoidance is unlikely to be absolute in most species in which it occurs, and hence, that the selection for the spiteful cytotype is weaker than predicted by our model. However, it should be noted that partial inbreeding avoidance can also extend to more distantly related relatives as well as parents or offspring in age-structured populations, which may enhance outbreeding beyond the rate assumed in our model.

Another type of inbreeding avoidance is avoidance of selfing in hermaphrodite species. This is of great interest in particular because low selfing rates or even complete selfing avoidance are very commonly found in plants, and also in most hermaphrodite animal taxa (reviewed in Jarne & Charlesworth 1993). Avoidance of selfing can be expected to have the same effect in our model as sibmating avoidance in dioecious species. This is because from the cytoplasm's viewpoint, a hermaphrodite is essentially the same as a pair of siblings in the context of reproduction. Moreover, at least one of the mechanisms of selfing avoidance—self-incompatibility—can be expected to also lead to incompatibility between closely related individuals, thus resulting in even higher outbreeding rates than with mere selfing avoidance.

In his seminal paper, Hamilton (1970) argued that spite would not be common in natural populations for three reasons. First, ‘all actions do cost something’. This is certainly not true for the mechanism we have studied, because the direct fitness of a maternally inherited gene in males is zero anyway and cannot be further reduced (see also Hurst 1991; Foster *et al*. 2001). Second, organisms would not be able to sufficiently distinguish between positively and negatively related members of the populations, so that even when close kin can be recognized, the rest of the population would on average be only slightly negatively related to the actor. In our model, inbreeding avoidance provides a natural, pre-existing mechanism by which kin is distinguished from non-kin. For some of the mechanisms that can lead to outbreeding—e.g. dispersal of one sex and self-incompatibility in plants—no cognitive recognition mechanisms are required. However, it is true that selection for a spiteful cytotype will be weak, especially in large populations. Finally, Hamilton (1970) suggested that the small populations required for spite to be substantially selected for would readily go extinct once the gene causing spite spreads. This notion is not confirmed by the model studied in this paper. Rather, we demonstrated that the spread of the spiteful cytotype can stabilize the population and also lead to an increased population size. Also, instead of one small population, a large population subdivided in many small subpopulations may also favour the spread of the spiteful cytotype. As recent theoretical work has demonstrated, spite can evolve in such a scenario especially with soft selection, i.e. local regulation of subpopulation size (reviewed in Gardner & West 2004).

An important aspect that is likely to strongly reduce the incidence of spiteful cytoplasmically induced phenotypes is selection acting on nuclear genes. Since the spiteful cytoplasm reduces the fitness of males that carry it and of females that mate with such males, two types of resistance gene can be expected to evolve. First, nuclear genes will be selected, which suppress the phenotype of the spiteful cytoplasm in males. Second, genes will be selected for which ameliorate the negative effects caused by males bearing the spiteful cytotype. (Interestingly, cytoplasmic genes are also selected for amelioration, which may lead to a modification-rescue system like CI). Because of these selection pressures, many spiteful cytoplasmic elements may be fixed in natural populations, but the phenotype is not expressed because resistance genes are also fixed. Thus, similar to cytoplasmic male sterility, CI or meiotic drive, crosses between different populations or species may be a promising way of detecting spiteful cytotypes as hypothesized in this paper.

## Acknowledgements

We wish to thank Dr Greg Hurst, Lorenzo Zanette and Dr Andy Gardner for helpful comments on the manuscript, and acknowledge funding from the UCL Graduate School to J.E.

## Footnotes

- Received September 25, 2005.
- Accepted November 18, 2005.

- © 2006 The Royal Society