Royal Society Publishing

Upstream reciprocity and the evolution of gratitude

Martin A Nowak, Sébastien Roch


If someone is nice to you, you feel good and may be inclined to be nice to somebody else. This every day experience is borne out by experimental games: the recipients of an act of kindness are more likely to help in turn, even if the person who benefits from their generosity is somebody else. This behaviour, which has been called ‘upstream reciprocity’, appears to be a misdirected act of gratitude: you help somebody because somebody else has helped you. Does this make any sense from an evolutionary or a game theoretic perspective? In this paper, we show that upstream reciprocity alone does not lead to the evolution of cooperation, but it can evolve and increase the level of cooperation if it is linked to either direct or spatial reciprocity. We calculate the random walks of altruistic acts that are induced by upstream reciprocity. Our analysis shows that gratitude and other positive emotions, which increase the willingness to help others, can evolve in the competitive world of natural selection.


1. Introduction

The evolutionary analysis of altruistic behaviour began with kin selection (Hamilton 1967; Queller 1992; Frank 1998; West et al. 2002; Foster et al. 2006) and group selection (Levin & Kilmer 1974; Wilson 1975; Keller 1999; Michod 1999; Boyd et al. 2003; Traulsen & Nowak 2006). Direct reciprocity is the idea that cooperation emerges in repeated encounters between the same two individuals according to the principle ‘I help you and you help me’ (Trivers 1971; Axelrod & Hamilton 1981; Axelrod 1984; Fudenberg & Maskin 1990; Nowak & Sigmund 1992; Imhof et al. 2005). In contrast, indirect reciprocity embodies the concept ‘I help you and somebody else helps me’ (Alexander 1987; Nowak & Sigmund 1998; Nowak & Sigmund 2005). Indirect reciprocity comes in two flavours. Downstream indirect reciprocity means that a person who has helped in the past has a higher chance of receiving help (Nowak & Sigmund 1998; Wedekind & Milinski 2000; Engelmann & Fischbacher 2002; Milinski et al. 2002; Fishman 2003; Brandt & Sigmund 2004; Ohtsuki & Iwasa 2004, 2005; Brandt & Sigmund 2005; Chalub et al. 2006). Upstream indirect reciprocity means that a person who has just received help has an (unreasonable) urge to help someone too. Although observed in experimental settings (Dufwenberg et al. 2001; Güth et al. 2001; Greiner & Levati 2003; Bartlett & DeSteno 2006), upstream reciprocity is harder to understand from an evolutionary perspective (Boyd & Richerson 1989; Pfeiffer et al. 2005). Here, we assume that the recipient of an altruistic act experiences gratitude and is more likely to help either the donor or another person. Therefore, gratitude is an emotion which can lead to upstream reciprocity. We show that upstream reciprocity alone does not allow evolution of cooperation, but it can evolve if it is linked to a mechanism for the evolution of cooperation. In particular, we study the links to direct reciprocity and spatial reciprocity (Nowak & May 1992; Nowak 2006; Ohtsuki & Nowak 2006a,b; Ohtsuki et al. 2006; Santos et al. 2006).

2. Random walks of upstream reciprocity

Each altruistic act involves a cost, c, for the donor and a benefit, b, for the recipient. We assume that b>c, otherwise cooperation cannot lead to an overall benefit. Consider a large population of players in a game of upstream reciprocity. One player initiates a chain of altruism. The second player passes it on, with a certain probability, to a third player, who in turn might help a fourth player and so on. After a number of steps, the chain dies out. Upstream reciprocity leads to random walks on the set of all players (figure 1).

Figure 1

Upstream reciprocity. (a) Somebody who has just received help is elevated, happy and grateful and thus more likely to help someone else. This idea is called ‘upstream reciprocity’. (b) Upstream reciprocity leads to random walks of altruism. One player initiates the chain, and then each recipient passes on with probability p. The average length of a random walk is 1/(1−p), which is the number of people who benefit from an initial act of kindness. Upstream reciprocity alone is not a mechanism for the evolution of cooperation, but it can evolve and enhance the level of cooperation if it is linked to such a mechanism.

In the simplest formulation of the game, each player is characterized by two parameters: q denotes the probability to initiate and p the probability to pass on. If everybody in the population uses strategy S(p, q), then the average length of an altruistic random walk is 1/(1−p). Each player initiates random walks at the rate q. Therefore, the ‘cooperativity’ of a strategy is given by s=q/(1−p), which is the expected number of secondary altruistic acts induced by a single player per time-step. In the limit p→1, the cooperativity tends to infinity (if q>0), because any one initiation generates a line of altruism which is never to break.

Defectors, S(0, 0), never initiate and never pass on. Classical cooperators, S(0, 1), are spontaneous altruists: they initiate, but do not pass on. They are cooperative, but not grateful. In contrast, the strategy S(1, 0) never initiates, but always passes on, and the strategy S(1, 1) always initiates and always passes on. These four strategies represent the corner points of our strategy space which is given by the unit square, [0,1]2. Each point in this square represents a pure, probabilistic strategy. Strategies with p>0 are gratitude-related altruists: they have a positive probability to help someone if they have just received help themselves.

Let us now consider a population of players using different strategies of upstream reciprocity. We can show that natural selection always reduces the cooperativity, s. A strategy with lower cooperativity always outcompetes a strategy with higher cooperativity. Defectors dominate all strategies with positive cooperativity, s>0. We conclude that upstream reciprocity alone does not facilitate the evolution of cooperation.

3. Upstream reciprocity can evolve as a by-product of direct reciprocity

The situation changes dramatically if we allow some level of direct reciprocity. Denote by r the probability that help is immediately returned to the donor. Thus, a recipient reciprocates to the donor with probability r and helps another random player with probability (1−r)p. A strategy is now given by three parameters, S(p, q, r).

In order to calculate the payoff for strategy Si=S(pi, qi, ri) versus Sj=S(pj, qj, rj), we need to evaluate all random walks in a large population where the frequencies (relative abundances) of Si and Sj are given by x and 1−x, respectively. The calculation leads to a payoff function that is nonlinear in x. But we can derive a simple matrix game which captures all evolutionary properties.

Consider the interaction between two strategies, S1 and S2. Let x denote the frequency of S1. The frequency of S2 is 1−x. The fitnesses of S1 and S2 are given by f1(x) and f2(x), respectively. In the electronic supplementary material, we show that the fitness difference is given byEmbedded Image(3.1)whereEmbedded Image(3.2)Therefore, in the standard framework of evolutionary game dynamics, the replicator equation (Taylor & Jonker 1978; Weibull 1995; Hofbauer & Sigmund 1998; Nowak & Sigmund 2004), the frequency of S1 changes asEmbedded Image(3.3)Note that f(x) is a nonlinear function of x, but the nonlinearity is benign. As long as both p1 and p2 are less than one, the denominator γ is always greater than 0. It is easy to check that equation (3.3) with f(x) given by equations (3.1) and (3.2) is equivalent (up to a change in speed) to a standard replicator equation for the payoff matrixEmbedded Image(3.4)Here, si=qi/[(1−ri)(1−pi)] is the cooperativity of strategy Si and uj=(brjc)/(1−rj) is the responsiveness of strategy Sj. This is the characteristic payoff matrix of the game. All invasion criteria, equilibrium points and their stability, as well as adaptive dynamics can be directly derived from this matrix. Therefore, the simplified payoff function for Si versus Sj is given by A(Si,Sj)=siuj.

Let us consider the game between an arbitrary strategy S=S(p, q, r) and defectors S′=S(0, 0, 0). Denote by x the frequency of players who use strategy S. Selection favours S, if Embedded Image. The critical frequency, Embedded Image, denotes the invasion barrier of S to replace defectors: if the frequency of S in the population is greater than Embedded Image, then selection favours S, otherwise selection favours defectors. Observe that Embedded Image requires r>c/b. Thus, only a strategy that fulfils this inequality can replace defectors by overcoming an invasion barrier.

We can calculate the evolutionary game dynamics between two strategies S1=S(p1, q1, r1) and S2=S(p2, q2, r2). If both r1 and r2 exceed c/b, then selection favours the strategy with higher cooperativity. If r1>c/b>r2 and s1<s2, then there is stable equilibrium between the two strategies. If r1>c/b>r2 and s1>s2, then both strategies are best replies to themselves. Finally, if both r1 and r2 are less than c/b, then selection favours the strategy with lower cooperativity.

From the simplified payoff function, we can see immediately that a strategy, S(p, q, r), is evolutionarily stable against invasion by another strategy with lower cooperativity if r>c/b. The probability to reflect the random walk to the donor, r, has to exceed the cost-to-benefit ratio, c/b, of the altruistic act. In this case, direct reciprocity allows the evolution of cooperation, and upstream reciprocity can hitch-hike on direct reciprocity. The interpretation of this finding is: if there is direct reciprocity in a population, then upstream reciprocity will evolve too (figure 2).

Figure 2

Upstream reciprocity evolves if there is direct reciprocity. (a) Suppose there is a certain probability, r, that the recipient of an altruistic act will immediately reciprocate to the donor. With probability (1−r)p the recipient will help someone else. The probabilities p and r denote the levels of upstream and direct reciprocity, respectively. If r exceeds the cost-to-benefit ratio of the altruistic act, r>c/b, then direct reciprocity allows the evolution of cooperation. But in this case, upstream reciprocity will evolve too, because selection will tend to increase the parameter p. (b) In order to calculate the payoff for one strategy (light blue) versus another (dark blue), we consider a large mixed population and average over all possible random walks initiated by each player (see the electronic supplementary material). Three random walks are shown (red, green and yellow arrows).

If r>c/b, then selection tends to increase all of the three parameters p, q and r, and evolution will lead to more upstream reciprocity, more initiation of random walks and more direct reciprocity. Adaptive dynamics (Nowak & Sigmund 1990; Dieckmann et al. 1995; Metz et al. 1996; Hofbauer & Sigmund 2003; Doebeli et al. 2004) are described in Appendix A. If the parameter r evolves very close to 1, then upstream reciprocity becomes less relevant. For many social interactions, however, it is natural to assume an upper bound rmax: a direct reciprocation is not possible or meaningful if there is no opportunity or need to help the donor immediately. In this case, there is a continuous selection pressure leading to more and more upstream reciprocity (p→1).

The parameter q characterizes spontaneous altruism, while p denotes the gratitude-related altruism of upstream reciprocity. The total amount of altruism in a population increases linearly with q, but hyperbolically with p. Therefore, gratitude-related altruism can lead to much higher levels of cooperativity. Moreover, if r>c/b, then spontaneous cooperators (with p=0) are outcompeted by gratitude-related cooperators (p>0).

4. The synergistic interaction between upstream and spatial reciprocity

As a second model, we investigate the interaction between upstream and spatial reciprocity. Imagine a cluster of cooperators surrounded by defectors. Altruistic acts are initiated by cooperators and lead to random walks of cooperation that die within the cluster or after hitting a defector. We need to calculate the expected payoff of all individuals by summing over all possible random walks. For the evolutionary dynamics, let us consider ‘imitation updating’ (Ohtsuki et al. 2006): at each time-step, an individual is chosen at random to update its strategy. It will compare its own payoff to the payoff of its immediate neighbours and then choose one of the strategies proportional to fitness. We can perform an exact mathematical analysis if we study the simplest possible geometry: all individuals are aligned in a one-dimensional array (Ellison 1993; Nakamaru et al. 1997; Lieberman et al. 2005; Ohtsuki & Nowak 2006a). The details of the calculation are shown in the electronic supplementary material. Here, we only state the results.

As before, we consider strategies S(p, q), where p denotes the probability to pass on and q the probability to initiate. It turns out that spontaneous altruists, S(0, 1), can outcompete defectors, S(0, 0), if b/c>4. In contrast, gratitude-related altruists, S(1, 1), can outcompete defectors if b/c>10/7. Therefore, upstream reciprocity working together with spatial reciprocity makes evolution of cooperation easier (figure 3).

Figure 3

There is a synergistic interaction between upstream and spatial reciprocity. Spatial reciprocity means that cooperators win against defectors by forming clusters (Nowak & May 1992; Ohtsuki et al. 2006). The simplest spatial geometry is a one-dimensional array. (a) Spatial reciprocity alone: if there is no upstream reciprocity, then a cluster of cooperators (blue) expands against a cluster of defectors (yellow) if b/c>4, assuming imitation updating (Ohtsuki & Nowak 2006a). In this case, selection favours cooperation by spatial reciprocity alone. (b) Spatial and upstream reciprocity together: if there is upstream reciprocity with probability p, then cooperators win against defectors if b/c>h(p). For p=1, we simply obtain b/c>10/7. Therefore, upstream reciprocity lowers the critical benefit-to-cost ratio that is needed for the evolution of cooperation. If p<1, then a random walk of altruism can end either within the cluster of cooperators (grey arrows) or after reaching a defector (red arrows).

More precisely, the game is determined by the following two functions:Embedded ImageIf b/c>h(p), then upstream cooperators, S(p, 1), win against both the defectors, S(0, 0), and the classical cooperators, S(0, 1). If b/c>g(p), then natural selection favours small increases in p and q, thereby enhancing the cooperativity, s=q/(1−p). Note that h(0)=g(0)=4, while h(1)=10/7 and g(1)=1. Moreover, we have h(p)≥g(p) for all probabilities p. All these results are derived in the electronic supplementary material.

5. Conclusion

In summary, we conclude that upstream reciprocity alone does not select for cooperation, but can promote cooperation if it is linked to a mechanism for the evolution of cooperation. Here, we have studied the links to direct and spatial (or network) reciprocity. If direct reciprocity allows cooperation (r>c/b), then upstream reciprocity will evolve too. In the model with spatial reciprocity, we find that evolution of cooperation is easier with upstream reciprocity (b/c>10/7) than without upstream reciprocity (b/c>4). In both models, spontaneous cooperators are outcompeted by gratitude-related upstream cooperators. Moreover, upstream reciprocity greatly enhances the level of altruism in a population.

Our models are psychologically plausible. Experiments with human subjects demonstrate that gratitude, which is the positive emotion one feels after having received something of value, fosters prosocial behaviour (Carlson et al. 1988; Emmons & McCullough 2004; Bartlett & DeSteno 2006): the recipient of a favour is more likely to help both the donor (direct reciprocity) and a stranger (upstream reciprocity). Thus, gratitude may be the key to understanding upstream reciprocity. Our analysis demonstrates that gratitude and other positive emotions, which enhance the willingness to help (given by the parameter p in our model), can evolve by natural selection. It is also conceivable that upstream reciprocity is a by-product of emotions that have primarily evolved to facilitate direct reciprocity.

This paper is a first step towards studying the enormous consequences of upstream reciprocity for human and animal behaviour. We expect that upstream reciprocity will also work in synergy with kin selection, group selection and downstream indirect reciprocity, which is based on reputation. It will be of particular interest to study upstream reciprocity as a random walk on general social networks. Another promising extension is the idea that a person, who has just received help, may not only help one other person (as considered here) but also several other people. This can lead to an ‘epidemiology of altruism’ resulting in an explosive increase of altruistic acts. For a change, this is a pandemic which would be welcomed by all of us.


Support from the John Templeton Foundation, the NSF/NIH joint program in Mathematical Biology (NIH grant 1R01GM078986-01), NSERC (Canada) and FQRNT (Quebec, Canada) is gratefully acknowledged. The Program for Evolutionary Dynamics at Harvard University is sponsored by Jeffrey Epstein.



Notice of correction

    The corresponding author's affiliation is now correct.

    21 December 2006

View Abstract