## Abstract

Costly signalling theory has become a common explanation for honest communication when interests conflict. In this paper, we provide an alternative explanation for partially honest communication that does not require significant signal costs. We show that this alternative is at least as plausible as traditional costly signalling, and we suggest a number of experiments that might be used to distinguish the two theories.

## 1. Introduction

Communication is ubiquitous in the biological world. When the interests of signaller and signal receiver are perfectly aligned, the evolutionary benefits of reliable communication are straightforward. But when interests are not aligned, signallers might be selected to manipulate signal receivers with misleading signals, and the signal receivers might evolve to disregard such communications. Why does communication not break down in situations that involve conflict?

Costly signalling theory provides one explanation. Communication is framed as a signalling game; with appropriate signal costs, honest communication is a Nash equilibrium of this game. If signalling is sufficiently costly so that lying is too costly to be worthwhile, but honest signals are not too costly to send, all signallers may choose to signal honestly at equilibrium [1–3].

However, the costly signalling explanation for honest communication is problematic for several reasons. On the theoretical side, costly signalling can be a very expensive mode of information exchange, and at equilibrium signals can be so costly that all involved would be better off simply not communicating [4]. From a dynamical perspective, the necessary level of cost may make it difficult to evolve signalling at all [5]. On the empirical side, researchers have not always been able to find substantive signal costs associated with putative costly signal systems—despite evidence that these systems do convey, at least, some information among individuals with conflicting interests [6–14]. What, then, are we to make of empirical situations in which signals appear to be informative even without the high costs required by costly signalling models?

One approach to resolving this problem is to recognize that—contra early work on the problem—the costly signalling mechanism does not require a cost to the signals that are actually sent in equilibrium. It requires only that out-of-equilibrium signals, i.e. ‘lies’, be too costly to be worthwhile [15,16]. In other words, it is not the cost of signalling, but rather the marginal cost of signalling, that ensures honesty [17].

In this paper, we offer an alternative explanation. We show that the paradigmatic costly signalling games from the animal behaviour literature allow a ‘hybrid’ signalling equilibrium, in which inexpensive signals are able to support a partially informative signalling equilibrium. While the existence of hybrid equilibria in economic models of signalling has been known for some time [18,19], the evolutionary significance of these equilibria is only just now being considered. Initial analysis focused on the Sir Philip Sidney game [5] and a signalling game from economics known as the Spence signalling game [20]. In this study, we demonstrate that hybrid equilibria in biological settings do not depend on the particular relatedness structure of the Sir Philip Sidney game, but rather are of general importance in signalling games of many types. We show that hybrid equilibria exist both in *differential cost models* such as models of mate advertisement [2], and in *differential benefit models* such as models of offspring begging [3,21]. Moreover, we demonstrate that under reasonable evolutionary dynamics, the hybrid equilibrium has a basin of attraction comparable in size with that of the classic costly signalling equilibrium. We conclude with a discussion of the implications of this work for empirical studies. We argue that most empirical studies taken as evidence of costly signalling theory fail to discriminate between the classical costly signalling equilibrium and the hybrid equilibrium presented here. However, these two different types of signalling equilibria make different predictions regarding equilibrium behaviour. This paves the way for future studies that may be able to empirically distinguish among these hypotheses.

## 2. Low-cost signals in costly signalling games

### (a) Differential costs and differential benefits

Figure 1 illustrates a canonical two-player signalling game in extended form. Player 1, the *sender* is one of two types, *T*_{1} or *T*_{2}. In the case of signalling between relatives, as exemplified by Maynard Smith's Sir Philip Sidney game, the types might represent the state of need of the receiver. In signalling between mates or signalling between predators and prey, these types might represent the quality of the signaller. Although the sender can condition her behaviour on her type, player 2, the *receiver*, cannot directly observe the signaller's type. Instead, the receiver can observe only whether a signal, designated *Sig*, is produced or not. On the basis of this observation, the receiver can choose one of two actions, *A*_{1} or *A*_{2}. In the Sir Philip Sidney game, the action *A*_{1} can be interpreted as transferring a resource and *A*_{2} can be interpreted as not transferring. In mate selection, *A*_{1} represents mating, whereas *A*_{2} represents not mating. In predator–prey signalling, *A*_{1} denotes declining pursuit and *A*_{2} denotes pursuit. This game features four pure (conditional) strategies for the sender, and four for the receiver. These are listed in table 1.

This game structure reveals that there is a common core to signalling among relatives, signalling to potential mates or signalling to predators. The pay-offs of this game result in a partial conflict of interest between signaller and receiver. When the signaller is type *T*_{1} (needy, a high-quality mate, or uncatchable as prey), both the sender and receiver prefer the same actions: both players prefer that the receiver take action *A*_{1}. However, when the sender is of type *T*_{2}, the sender and the receiver have divergent interests. In this case, the sender still prefers that the receiver take action *A*_{1}, but a fully informed receiver would prefer to take action *A*_{2}. This partial conflict of interest introduces the evolutionary conundrum: a *T*_{1} type individual benefits by advertising its true quality to the receiver, whereas a *T*_{2} type individual benefits by deceiving the receiver. The game illustrated in figure 1 has no cost associated with signalling. Without signal costs, this conflict cannot be resolved by honest communication. There is no stable set of strategies where the type is revealed by the signal. All the pure-strategy equilibria of this game are known as pooling equilibria, where both types send the same signal.

The central insight of costly signalling theory is that signal costs can stabilize honest communication. For this to work, different types of signallers must have different incentives to send signals. Such differing incentives can arise as a result of differential costs, or differential benefits, or both. In *differential cost* models, one type pays lower signalling costs than the other. This is the scenario originally described by Zahavi [1] in the context of signalling between mates; Grafen [2] treated this case in his landmark 1990 paper. Signalling from prey to predator also typically involves differential costs [22,23]. In *differential benefit* models, one type reaps a larger reward from the receiver's response than the other. Differential benefits drive honest signalling in Maynard Smith's Sir Philip Sidney game [3] and in Godfray's model of nestling begging [24]. In what follows, we treat differential cost and differential benefit models in turn.

### (b) Hybrid equilibria for differential cost games

Consider the game pictured in figure 2 and table 2. This game allows for the possibility of stable advertisement of type, because the signal imposes differential costs on senders of different types. Take the strategy where *T*_{1} signals and *T*_{2} does not. In such a case, it is obviously best for the receiver to take action *A*_{1} when the signal is observed and action *A*_{2} if not. So long as the *T*_{1} individuals pay a signal cost of *c*_{1} < 1, it is strictly in their interest to signal in order to ensure that the receiver takes action *A*_{1}. So long as the *T*_{2} individuals would pay a cost *c*_{2} > 1 if they were to signal, then it is strictly in their interest to refrain from signalling, because the cost of the signal outweighs the benefit obtained by inducing the receiver to take action *A*_{1}:
2.1
This is the familiar separating equilibrium in a differential costs model.

However, there is another equilibrium that allows for partial transfer of information. Consider the following mixed strategy pair: the signaller mixes between strategy *S*_{1} with probability (1 − *α*) and strategy *S*_{2} with probability *α*. The receiver mixes between strategy *R*_{1} with probability *β* and strategy *R*_{2} with probability (1 − *β*). The resulting behaviour is illustrated in figure 3. *T*_{1} individuals always send the signal, but *T*_{2} individuals play a mixed strategy, sending the signal with probability *α* < 1 and refraining from sending the signal with probability (1 − *α*). If the sender does not observe the signal, she takes action *A*_{2} with certainty. If she does observe the signal, she plays a mixed strategy, taking action *A*_{1} with probability *β* < 1 and taking action *A*_{2} with probability (1 − *β*). In plain English, this means that the sender sometimes ‘lies’ and is honest at other times, whereas the receiver only sometimes chooses the sender's favoured action.

When the players behave in this way, information transfer is imperfect, unlike the separating equilibrium; the signal does not indicate the type of the receiver with certainty. However, *the signaller does transmit some information to the receiver and the receiver does make some use of that information*: because *T*_{1} signals always and *T*_{2} signals only occasionally, observing a signal increases the likelihood that the signaller is a *T*_{1}-type. If the prior probability of *T*_{1} is *x*, then the probability that a signalling individual is of type *T*_{1} equals *x*/(*x*+*α*(1−*x*))>*x*.

When does this pair of mixed strategies constitute an equilibrium? First, consider the actions of the receiver. Because only *T*_{2} individuals fail to signal, receiving no signal informs the receiver that the signaller is a *T*_{2} individual and that the receiver does best to take action *A*_{2}. If at equilibrium, the receiver randomizes between *A*_{1} and *A*_{2} in response to receiving a signal, these two actions must yield the same expected pay-off. Supposing the sender's strategy conforms to the description just offered, the expected pay-off to the receiver of playing *A*_{1} given that a signal is observed is

The expected pay-off of playing *A*_{2} given that a signal is observed is

These expected pay-offs are equal when the signaller's mixed strategy uses strategy *S*_{2} with frequency *α* given by *α*=*x*/(1−*x*). This condition can be satisfied only when *T*_{1}-types are sufficiently rare. (The fact that in this model should not be taken as some deep fact about the possibility of hybrid equilibria. Rather, the threshold of arises because we have chosen a symmetric pay-off for the receiver's success when the sender is type *T*_{1} or type *T*_{2}. Had we made these pay-offs different, there would be a different constraint on *x*.)

Now consider the sender's strategy. *T*_{1} individuals always signal. For this to be an equilibrium behaviour, the pay-off from signalling must be at least as large as that from not signalling: , which obtains when *β* ≥ *c*_{1}. Individuals of type *T*_{2} mix between signalling and not signalling. Thus, these two behaviours must yield the same pay-off: , which reduces to *β* = *c*_{2}. Given these requirements, *β* can take on a value in (0,1) and thus represent a mixed strategy best response when 1 > *c*_{2} > 0 and when *c*_{2} ≥ *c*_{1}. Thus, the overall conditions for the hybrid equilibrium are
2.2
2.3
2.4

Comparing conditions (2.1) and (2.4), we see that both the separating equilibrium and the hybrid equilibrium require *c*_{2} ≥ *c*_{1}. Given condition (2.3), the hybrid equilibrium exists precisely when the cost *c*_{2} is not sufficiently high to maintain the separating equilibrium, i.e. *c*_{2} < 1. Differential cost is still necessary for the hybrid equilibrium, but the signal can be considerably cheaper than in the separating equilibrium. Figure 4*a* illustrates the regions of parameter space where the hybrid and separating equilibrium exist.

How do the two players fare at the different equilibria? At the traditional costly signalling equilibrium, the receiver gets a pay-off of 1, regardless of the state of the sender. At the hybrid equilibrium, this pay-off is less, because occasionally the receiver performs action *A*_{2} when the sender is of type *T*_{1} and sometimes the receiver performs *A*_{1} when the sender is of type *T*_{2}. From the perspective of the receiver, the traditional signalling equilibrium yields a higher pay-off.

Things are more complicated for the sender. Use *H* and *S* superscripts to the values *c*_{1} and *c*_{2} to refer to costs at the hybrid or separating equilibrium, respectively. Prior to learning her type, the sender's expected fitness in the traditional costly signalling equilibrium is . In the hybrid equilibrium, it is . Whether it is better for the sender to be in a signalling or hybrid equilibrium depends on *which* signalling and hybrid equilibrium we compare. This much can be said: for any signalling equilibrium, there exist hybrid equilibria where the sender fares better than in that signalling equilibrium.

### (c) Hybrid equilibrium for differential benefit games

Differential costs are not essential for a costly signalling equilibrium. Stable signalling can also arise when the two types of signallers face the same signal costs, but accrue different benefits from the receiver's response. This is the scenario modelled in Maynard Smith's Sir Philip Sidney game [3]. A differential benefits signalling game is illustrated in figure 5 and table 3.

In this differential benefits model, the two types pay the same cost *c* of signalling, but the benefit they receive from getting a receiver to take action *A*_{1} varies. The separating equilibrium requires first that the *T*_{1} individuals do best by signalling, namely that *a* > *c*. Second, it requires that the *T*_{2}-types do better by not signalling, i.e. that *c* > *b*. Thus, the benefit of *A*_{1} must be higher for the *T*_{1} individuals than it is for the *T*_{2} individuals, and the cost of signalling must separate the two:
2.5

There is also a hybrid equilibrium. Consider the same combination illustrated in figure 3: *T*_{1} individuals always signal, whereas *T*_{2} individuals mix between signalling and not signalling; receivers mix between *A*_{1} and *A*_{2} in response to a signal, and always select *A*_{2} in the absence of a signal. The condition on *α* remains the same as in the differential costs model: *α* = *x*/(1 − *x*).

The condition on the benefits is as follows: . The conditions are satisfied with *β* taking a value in (0,1) when . The overall conditions for a hybrid equilibrium are thus
2.6
2.7
2.8

Comparing conditions (2.5) and (2.7), we see that both the separating and hybrid equilibria require *a* ≥ *b*, i.e. that the benefit from *A*_{1} be greater for *T*_{1} individuals than for *T*_{2} individuals. However, the hybrid equilibrium exists precisely when the signal cost *c* is too low to sustain the separating equilibrium.^{1} Figure 4*b* illustrates the regions of parameter space in which each of these equilibria are found for differential cost models.

Again, we can compare the pay-offs obtained under the hybrid and signalling equilibria. As in the differential cost model, the receiver invariably gets a higher pay-off at the separating equilibrium. In the differential benefit model, the expected fitness of the sender is *x*(*a* − *c ^{S}*) at the separating equilibrium and

*xc*(

^{H}*a*/

*b*−1) at the hybrid equilibrium. As with the differential cost model, for every separating equilibrium, there exist hybrid equilibria at which the sender fares better. In both models, separating can be so costly that signallers do better with reduced information transfer. Because in this game—unlike in the Sir Philip Sidney game—the signal cost does not decrease the pay-off to the receiver, the receiver is always better off receiving more information.

Many biologically relevant interactions will feature both differential cost and differential benefit. Although we have divided this into two cases for the purposes of clarity, the same hybrid equilibrium exists in a model that combines both the differential cost and differential benefit. The conditions for its existence are more complex but essentially the same conditions as above.

## 3. Evolutionary implications

The mere existence of Nash equilibria does not guarantee their evolutionary significance. As is well known, many Nash equilibria are unlikely evolutionary outcomes in that they are unstable in one sense or another [25]. In this section, we will demonstrate the evolutionary significance of the hybrid equilibria characterized earlier by illustrating that these equilibria attract open sets of nearby states in the two-population replicator dynamics.

The two-population replicator dynamics captures the basic process of evolution by natural selection in asymmetric games [25]. Let *x _{i}* be the relative frequency of sender type

*i*and

*y*be the relative frequency of receiver type

_{j}*j*,

*i*,

*j*= 1, … ,4. This dynamics is given by: 3.1a and 3.1b The vectors and are the vectors of strategy frequencies for signaller and receiver, respectively. The expression gives the pay-off of

*i*against and gives the pay-off of

*j*against . The average pay-off in the sender population is given by and the average pay-off in the receiver population by . The two-population replicator dynamics is thus a simple formalization of the idea that, in each population, strategies with above-average fitness thrive, whereas strategies with below-average fitness decline.

We can now consider the evolutionary properties of both the signalling and hybrid equilibria. In both the differential cost game and the differential benefits game, the signalling equilibrium is a strict Nash equilibrium (when the listed strict inequalities are satisfied). Therefore, the signalling equilibrium is necessarily an attracting state in the two-population replicator dynamics [25]. Although this does not guarantee that it is a global attractor—meaning that we cannot be assured that the signalling equilibrium be reached from any ancestral state—we can conclude that a population at a signalling equilibrium will remain there if subject to only small perturbations.

The hybrid equilibria in both games are slightly more complex. On the plane defined by the four strategies (two sender and two receiver) that make up the hybrid equilibria, populations cycle around the hybrid equilibrium indefinitely. States near, but not on this plane, converge to the plane. So, the hybrid equilibrium characterizes a set of states that are also evolutionarily significant. In §3*a,b*, we demonstrate this for both games.

### (a) Differential cost games

There are two major claims that must be established. First, we will characterize trajectories on the plane composed by strategies *S*_{1}, *S*_{2}, *R*_{1} and *R*_{2}. Second, we will show that population states near the plane will converge to it.

We will begin by considering the game comprised by the four strategies listed earlier. This game produces the following two pay-off matrices:

By subtracting a (different) constant from each column, one obtains the following equivalent game:^{2}

The hybrid equilibrium corresponds exactly to the interior Nash equilibrium of this game (whenever it exists). The interior Nash equilibrium is either unstable for the replicator dynamics restricted to the plane, or there are closed orbits around it. Hofbauer & Sigmund [25] show that there are three necessary conditions that are required for the two-population replicator dynamics to produce the second case, i.e. cyclic behaviour. First, the non-zero entries in *S*^{′} must be of the same sign. Second, the non-zero entries in *R*^{′} must also be of the same sign. Finally, the non-zero entries in *S*^{′} and *R*^{′} must be of different signs from one another. The first condition holds on assumption that *c*_{2} < 1, which is required for the existence of the hybrid equilibrium. The second condition holds on condition that *x* < 1/2, again a requirement for the hybrid equilibrium. From these two requirements, the final condition follows. It is shown in Hofbauer & Sigmund [25] that when these conditions are satisfied, there exists a constant of motion. It follows that the replicator dynamics cycle around the hybrid equilibrium in closed orbits on the plane comprising the four strategies *S*_{1}, *S*_{2}, *R*_{1} and *R*_{2}.

What about states off this plane? Consider perturbations from the hybrid equilibrium into the interior of the space. At the hybrid equilibrium, the average pay-off for the sender is *x*(*c*_{2} − *c*_{1}) and (1 − *x*) for the receiver. The pay-off of *S*_{3} and *S*_{4} against the hybrid equilibrium is 0 in both cases; as a result, the motion in the interior close to the hybrid equilibrium will be in the direction of the hybrid equilibrium. The pay-off for strategies *R*_{3} and *R*_{4} against the hybrid equilibrium is *x*. Because the hybrid equilibrium exists only when , it follows that whenever the hybrid equilibrium exists, the motion of the system will be towards the hybrid equilibrium from the interior when the system is close to the hybrid equilibrium. These considerations show more formally that the so-called transversal eigenvalues of the hybrid equilibrium [25] are negative. Together with the results regarding the plane comprised of *S*_{1}, *S*_{2}, *R*_{1} and *R*_{2}, it follows that interior trajectories approach the hybrid equilibrium in a spiralling manner. The hybrid equilibrium is ‘strongly stable’ relative to outside states and ‘weakly stable’ towards states on the plane.^{3}

That stable cycles are observed on the face of the simplex is structurally unstable—small changes in the underlying dynamics (as might occur with the introduction of mutation or other stochastic effects) are likely to alter this prediction of our model. A number of different outcomes can occur depending on what changes are made to the underlying dynamical system [25]. We leave the details of these changes for future research. On the other hand, the stability of the plane itself against invasion from types off the plane is unlikely to be affected by small changes in the dynamics. While we are unsure whether cycles should be expected in natural populations, the stability of populations on the hybrid equilibrium plane to invasion by other types suggests that one should expect to find natural populations on or near this plane.

### (b) Differential benefit games

We will now complete the same analysis for the differential benefit version of the signalling game. *R*^{′} remains as before, but *S*^{′} is different:

Again, the three conditions are satisfied when the hybrid equilibrium exists, so we can conclude that the motion on the plane is cyclic. In the differential benefit game, the average pay-off for sender in the hybrid equilibrium is *xc*(*a*/*b*−1), which is positive so long as *a* > *b*. The pay-off for *S*_{3} and *S*_{4} are both 0, and so we can conclude that the motion in these directions is towards the hybrid equilibrium. The analysis for the receiver is identical as above.

### (c) Basins of attraction

Thus far, we have shown that a non-zero fraction of the state space will converge to the hybrid equilibrium when it exists. A similar result holds for signalling equilibria, provided that they exist. Figure 6 illustrates simulation results that capture the relative sizes of the basins of attraction for both signalling and hybrid equilibria. This graph reveals that in both the differential cost and differential benefit models, the hybrid equilibrium is as significant an evolutionary outcome as the signalling equilibrium.

In these graphs, the basin of attraction of the hybrid equilibrium is modelled by measuring how many initial states converge to the surface which only consists of strategies *S*_{1}, *S*_{2}, *R*_{1} and *R*_{2}. We have already illustrated that once on this face, populations will enter stable cycles around the hybrid equilibrium. This represents populations that are heterogeneous with respect to their willingness to signal honestly and respond to the signal.

Note that the analysis in this section deals with the situation in which mixed strategies are manifested by population-level polymorphism of pure strategists rather than a monomorphic population of pure strategists [26]. There is no straightforward way to characterize the dynamic stability of a hybrid equilibrium manifested as a monomorphic population of mixed strategists. The hybrid equilibrium is not a strict Nash equilibrium because mixed Nash equilibria are never strict. Nor is it an evolutionarily stable strategy (ESS), because by Selten's theorem [27], there are no ESSs in mixed strategies in role-asymmetric games.

## 4. Discussion

We have shown that hybrid equilibria are general features of the costly signalling games studied in evolutionary biology. They arise and facilitate low-cost, low-fidelity communication not only in differential benefit games used to model nestling begging, but also in the differential cost games used to model sexual signalling and prey-to-predator communication. Our findings suggest an important empirical avenue for investigation. Experiments that purportedly demonstrate costly signalling in animal populations have traditionally compared two hypotheses: the null hypothesis—that no communication is taking place—against an alternative hypothesis—that some communication is taking place. While these studies have conclusively rejected the hypothesis that no signalling is taking place, they were not designed to distinguish between the traditional signalling hypothesis and the hybrid equilibrium hypothesis presented here.

Empirical studies could distinguish between the hybrid equilibrium and the separating equilibrium. First, the hybrid equilibrium hypothesis predicts that signalling will be honest some, but not all of the time. Moreover, the deviation from honesty should take on a particular direction that distinguishes the hybrid equilibrium from a separating equilibrium muddied by noise or mistakes. At the hybrid equilibrium, ‘low’ type individuals (*T*_{2} in our model) should sometimes deviate and send the costly signal, whereas ‘high’ type individuals (*T*_{1}) should never deviate by failing to signal. Similarly, receivers should sometimes deviate and fail to respond positively to a signal, but they should never deviate by responding positively in the absence of a signal.

Second, our model predicts that the population would be heterogeneous with respect to both the sending and receiving strategies. Third, our model suggests that it is unlikely that the population would be in equilibrium, but rather would be observed cycling around the equilibrium. This last prediction is perhaps the most tenuous, because the cycling behaviour observed here will not be stable to small perturbations in the underlying dynamics.

Beyond the empirical significance of our results, we have demonstrated an alternative evolutionary explanation for (partially) honest communication in situations of conflict of interest, which can range from parent–offspring interactions to mating advertisement to predator–prey interactions. Although this equilibrium had been previously observed [5,20], it was not known if this phenomena was an artefact of the particular games or if it represented a general phenomenon common to many different signalling interactions. In this paper, we show that this type of signalling is at least as evolutionarily plausible as that offered by the traditional costly signalling models and may fit better with the observed data on signal costs. In this respect, it may represent a superior theory to traditional handicap theory.

## Acknowledgements

The authors thank the anonymous reviewers for helpful comments on earlier drafts of this paper. This material is based upon work supported by the National Science Foundation under grant no. EF-1038456. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

## Footnotes

↵1 Huttegger & Zollman [5] prove the existence of this equilibrium in the Sir Philip Sidney game, a model of signalling among relatives that involves relatedness between signaller and receiver, requiring additional parameters and a more byzantine form than the game considered here.

↵2 The game is equivalent in the sense of having the same equilibria.

↵3 See Huttegger & Zollman [5] for technical details in the case of the Sir Philip Sidney game; they easily carry over to this case. Further technical details are provided in Hofbauer & Sigmund [25].

- Received August 10, 2012.
- Accepted October 16, 2012.

- © 2012 The Author(s) Published by the Royal Society. All rights reserved.