Foraging under conditions of short-term exploitative competition: the case of stock traders

Serguei Saavedra, R. Dean Malmgren, Nicholas Switanek, Brian Uzzi

Abstract

Theory purports that animal foraging choices evolve to maximize returns, such as net energy intake. Empirical research in both human and non-human animals reveals that individuals often attend to the foraging choices of their competitors while making their own foraging choices. Owing to the complications of gathering field data or constructing experiments, however, broad facts relating theoretically optimal and empirically realized foraging choices are only now emerging. Here, we analyse foraging choices of a cohort of professional day traders who must choose between trading the same stock multiple times in a row—patch exploitation—or switching to a different stock—patch exploration—with potentially higher returns. We measure the difference between a trader's resource intake and the competitors' expected intake within a short period of time—a difference we call short-term comparative returns. We find that traders' choices can be explained by foraging heuristics that maximize their daily short-term comparative returns. However, we find no one-best relationship between different trading choices and net income intake. This suggests that traders' choices can be short-term win oriented and, paradoxically, maybe maladaptive for absolute market returns.

1. Introduction

Animals face a recurring alternative between continuing to forage in a patch or gambling on switching to a different patch with possibly better returns [13]. Optimal foraging theory purports that animal foraging choices have been shaped by natural selection and should maximize absolute fitness [47]. Similarly, optimal foraging theory considers that both human and non-human animals can take into account the foraging choices of their competitors while making their own choices [818]. Thus, interactions among competitors are increasingly important to understanding how real foraging choices can be shaped as animals compete for resources [1926]. Competitive interactions are typically of two types: exploitative competition, when different animals consume common limited resources (e.g. two predators hunting the same prey) [1921]; and interference competition, when direct interactions such as territoriality negatively affect the foraging of other animals [222]. Yet, broad empirical facts on the link between optimal and real foraging choices are scarce due to the complication of gathering field data or constructing experiments [27].

Importantly, biological and socio-economic systems share many common features in terms of distributed resources and competition, and thus financial systems have provided a fruitful and intriguing setting to test biological theories of behaviour because of their high-quality quantifiable and dynamic behavioural data [20,2832]. As far as we know, however, financial traders have not been examined from the perspective of foraging. Day traders face the classical foraging trade-off of trading the same stock multiple times in a row—patch exploitation—or switching to a different stock—patch exploration. For instance, each trader can trade multiple stocks within a class of stocks she has expertise in (e.g. technology stocks, banks stocks, transportation stocks, etc.), and is faced with the foraging choice of buying and selling the same stock multiple times in a row (e.g. buy a stock at a low price and vice versa for selling) or switching their trading to a different stock where returns are potentially higher (figure 1). By analogy to foraging in a physical habitat where energy is invested in travelling and hunting, traders either exploit the returns related to one stock (i.e. a patch) or explore a different patch while potentially experiencing cognitive costs for switching between patches [3336]. Moreover, the returns in each patch are shaped by exploitative competition, where the foraging choices of other traders, even within a short period of time, can increase or decrease the quality and availability of resources as they choose to buy or sell their stocks [37]. Thus, if a trader is willing to buy and the majority of traders are also buying then the stock price increases, in turn, the trader's return will be reduced. In this paper we investigated the extent to which professional traders' exploration and exploitation choices can be explained by foraging heuristics that respond to short-term competition with other traders. Additionally, we analysed whether traders' trading choices are associated with their net income intake. A significant relationship would mean a real correspondence between trading choices and absolute returns; whereas a lack of relationship would suggest a maladaptive behaviour for absolute market returns.

Figure 1.

Foraging choices. The figure shows an illustrative example of a trading activity from a single trader. Grey boxes correspond to the different trading patches (Roman numerals) of sequential transactions of the same stock. Note that patches are separated when two consecutive transactions belong to a different stock. Arabic numerals represent the exploitation index of each transaction within its trading patch. The upper grey region is a zoom to transaction three within trading patch II. This transaction took place at 12.37.23 h, where the trader bought GOOG stock at 317.05 USD. The stock's market price during the 5-min window between 12.35 and 12.40 h was 〈316.14〉 ± 0.105 (s.d.). Therefore, the short-term comparative return (see text) of this transaction can be calculated as Embedded Image. This suggests that this was a negative short-term comparative return for that time and choice in specific. (Online version in colour.)

2. Material and methods

We studied the second-by-second trading decisions of day traders at a typical small-to-medium-sized trading firm from 1 January 2007 to 31 December 2008. We recorded when a trader begins to trade a stock, how much he subsequently traded the same stock, and when he switched to explore a different stock. In our data, traders typically (greater than 90% of the time) made more than 10 transactions, and more than 3 switches, per day (figure 2). These novel data cover more than 300 000 trades made on approximately 3000 different stocks across a very wide range of sectors and on various exchanges, mostly from NYSE, the ‘blue chip’ exchange, and NASDAQ, the exchange known for high tech and volatile stocks. In particular, the stocks include high technology firms, diversified financials, shipping, natural resources, construction, chemicals, insurance, steel, etc. The top five stocks traded at the firm over our time period in terms of number of trades and volume were JP Morgan Chase & Co., Mechel Steel Group Oao, Goldman Sachs Group, Apple Inc. and Potash Corporation of Saskatchewan Inc.

Figure 2.

Individual trading activity. (a,b) The cumulative distribution of the number of total transactions and number of patches visited each day by each trader. In our data, most of the traders typically (greater than 90% of the time) made more than 10 transactions and made more than three switches per day. (Online version in colour.)

A typical small-to-medium day trading firm invests the money of the owners of the firm in stocks and hires traders to make the firm's investments. Day traders make only intraday trades; they typically do not hold inventories of stocks beyond a single day. Rather, they enter and exit positions each day during normal trading hours of 9.30 and 16.00 (EST). Our day traders are ‘point-and-clickers’. They make trades in real time 98 per cent of the time (the 1.2% of the trades done algorithmically were omitted and did not affect the results). Though they sit in the same firm, day traders typically trade different stocks from each other and trade independently of each other. Trading different stocks diversifies the firm's holdings, exploits specialized trading knowledge, and avoids accidentally trading against each other's positions. These dynamics mean that traders have little incentive to mimic each other's trades, information gathering behaviour, or trading decisions. The firm was located in the USA.

Our sample of day traders under study was 30. This sample of 30 traders was the full number of traders for which there was complete data on all decisions and behaviours measured over our observation period. By contrast, the other traders at the firm (n = 36) all worked for truncated interludes or worked erratically, which made their measurement unbalanced and unsystematic, and vulnerable to selection and small sample size biases [38]. All traders at the firm were men of an average age of 35 years old and a range between 22 and 50 years of age. They used the same technology to trade, had access to the same public information sources, and were subject to an equivalent incentive scheme. Traders were paid a base salary plus commissions on trades. The firm did not share with us their commission formula. They did indicate that like typical firms, the commission was based on end of the day earnings over a range of time to remove as possible chance fluctuations.

At the time of observation, our sample of traders traded about half of the stocks available on these exchanges on average. It is likely that the specific company stocks that were not traded were ones that lent themselves to holding long-term positions rather than trading on intraday shifts in price. All trading related data were automatically captured by the firm's trading system, which is specially designed for accuracy in recording, and used by most other firms in the industry. This automated and electronic capture system works unobtrusively to avoid interference with trading. The capture system fulfills US Securities and Exchange Commission requirements that all trades be recorded and archived for up to 7 years. The net income data were calculated by the firm using standard industry metrics. In our study, we analysed all the trades of all the stocks of all the traders in our sample. The study conforms to Institutional Review Board criteria. There was no subject interaction, all data were 100 per cent archival, and the firm and the subjects were anonymized. Legally, all data used in the study are owned by the company. All traders at the firm know the firm owns the data and that their communications and trading behaviour is recorded by law. We received written permission from the firm to use these data for research purposes and publishing contingent on identifying characteristics of the firm and its traders remaining confidential and anonymous.

3. Results

(a) Short-term comparative returns

To measure the extent to which traders' exploration and exploitation choices can be shaped by the foraging choices of their competitors, we introduced a novel measure that captures the difference between a trader's resource intake and competitors' expected intake over a short period of time—what we called short-term comparative return—and tested whether foraging choices can be explained by traders trying to maximize their daily short-term comparative returns.

The short-term comparative return associated with each transaction was calculated as the difference between actual traded prices decided on by each trader and the average prices in the market within a relevant time window. Since the anticipation of and response to the actions of competitors can be manifested by acting before them or by waiting and acting after them [3942], we followed theory and defined context limits according to the smallest time window (5 min) where it has been shown that individual transactions can impact the returns of others in the market [37]. For each trader i and each of his transactions j on day t, we defined the short-term comparative return as zijt = (γ)(Tijt−〈Tst*〉)/σTst*〉, where Tijt is the traded price and 〈Tst*〉 and σTst* are the average and standard deviation of stock s's price on day t within a five-minute interval, and γ = 1 (γ =−1) for selling (buying) transactions (figure 1). The stock's average price 〈T*st〉 and standard deviation σT*st are a mirror of the foraging choices of competitors, since prices move according to the stock's consumption or demand [28,37]. These price statistics are computed using the WRDS database (http://wrds-web.wharton.upenn.edu/wrds/), which has all the recorded transactions made around the world for each stock. Thus, zijt > 0 and zijt < 0 always indicate, respectively, a positive and negative short-term comparative return relative to the actions of competitors at that time.

To test how well the time window captures the changing foraging choices and depletion of resources over a period of time, we calculated lagged and leading short-term comparative returns zijt(Δ) using the stock's average price 〈T*st(Δ)〉 and s.d. σT*st(Δ) within 5 min intervals 5 min before Δ and, respectively, 5 min after Δ+ the observed 5 min interval of each transaction. Again, we used the WRDS database (http://wrds-web.wharton.upenn.edu/wrds/) to calculate these values. If the distribution of lagged short-term comparative returns is similar to the actual distribution of short-term comparative returns then it would suggest that the prices within the actual time window are, in fact, representative of the actions of others over a recent short period of time and not simply artefacts of the 5 min interval. Using Wilcoxon signed-rank test for testing paired and non-normally distributed distributions, we found that in 28 out of 30 traders the actual and lagged short-term comparative returns were significantly similar (table 1), which confirms that the actual time window is a reasonable context to use. Additionally, we repeated the same analysis but with lags (Δ and Δ+) greater than 1 h and found in all traders the actual and lagged returns were significantly different (table 1), meaning that these prices are representative of the actions of others only in the short-term.

View this table:
Table 1.

Traders' detailed information. For each trader, the table shows the total number transactions made n and the mean number of daily transactions 〈Nit〉 over the observation period. The Wilcoxon signed-rank tests z5 min and z1 h for lags of 5 mins and 1 h, respectively. Note that values of z >|2| are considered statistically significant. The coefficients B0, B1 and B2 taken from the multivariate regression model that takes the form zijt = β0+β1bijt + β2qijt+ε. Embedded Image corresponds to s.e. The correlation values ρrel and ρabs correspond, respectively, to the association of daily short-term comparative returns Rit with actual relative returns Ait and net income intake Iit (see text). Calculations are performed with software Stata.

To know whether traders' short-term comparative returns are associated with their foraging choices, we divided the total number Nit of transactions j of each trader i in day t according to their exploration index bijt, or trading patch, and their exploitation index qijt, or position within the patch. Figure 1 presents an illustrative example of how we divided the number of transactions. This example shows that a trader i in a day t had a total of 14 transactions (green bars) allocated in four different patches (grey regions). Regarding the exploration index, the first two transactions were characterized by bijt = 1 for j = {1,2}, the next four transactions by bijt = 2 for j = {3,4,5,6}, the next six transactions by bijt = 3 for j = {7,8,9,10,11,12} and the last two transactions by bijt = 4 for j = {13,14}. Additionally, these transactions were characterized by their exploitation indices qi1t = 1, qi2t = 2, qi3t = 1, qi4t = 2, qi5t = 3, qi6t = 4, qi7t = 1, qi8t = 2, qi9t = 3, qi10t = 4, qi11t = 5, qi12t = 6, qi1t = 1 and qi14t = 2. Note that each time the trader visits a new patch, the exploitation index is reset to 1.

For each trader, we modelled short-term comparative returns as a function of the importance of exploration β1 and exploitation β2 using a multivariate regression model that takes the form zijt = β0 + β1bijt + β2qijt + ε. Table 1 indicates that both exploration and exploitation are negatively associated with short-term comparative returns. In line with optimal foraging theory, these results reveal diminishing payoffs per resource, i.e. daily comparative returns Embedded Image can decrease in proportion of the number of stocks exploited or explored.

To illustrate this point, we used β1 = −0.002 and β2 = −0.02 of one single trader, and assuming that in one particular day that trader made 65 transactions exploring 65 different patches, the trader would have Embedded Image, where Embedded Image are the predicted short-term comparative returns from the regression model without considering β0, i.e. this decline is relative to the trader's average returns over the same transactions. In contrast, if the trader would have explored one single patch, the total returns would have changed to Embedded Image. If one multiplies Embedded Image by say the average difference between traded price and average price in the market ($0.13) times the average volume of stocks per transaction (300) in our data, Embedded Image translates into a relative loss compared with the trader's average performance over the same transactions of $−218.01 and $−1678.17, respectively. Note that this negative return is a relative measure of performance and should not be interpreted as the actual payoff. Instead, it reflects the possibility of different expected outcomes [43].

This resulting relative loss indicates that when foraging is compounded over many choices of exploitation and exploration, different activity patterns can impact the daily short-term comparative returns of traders. Similarly, the relative loss can also be examined by quantifying the decline in Embedded Image generated by exploitation and exploration patterns separately when considering a constant number of transactions per patch Qit. Figure 3 shows that when considering exploration only, the lower the value of Qit the higher the decline of Embedded Image (dashed line); and the opposite behaviour is observed when considering exploitation only (solid line). Importantly, the relationship between exploitation and exploration patterns reveals that an optimal pattern for jointly maximizing traders' Embedded Image exists, i.e. the intersection between the two curves.

Figure 3.

Importance of foraging choices over daily short-term returns. The figure shows an illustrative example of the relationship between daily or daily short-term comparative returns Embedded Image and different exploration and exploitation choices as given by a constant number of transactions per patch Qit. In the example, we considered a trader with Nit = 65, β1 = 0.002 and β2 = 0.02. Embedded Image, where Embedded Image are the predicted short-term comparative returns from the regression model (see text). When considering exploration only, the lower the value of Qit the higher the decline of Embedded Image (dashed line); and the opposite behaviour is observed when considering exploitation only (solid line). Importantly, this reveals that an optimal pattern for jointly maximizing traders' Embedded Image exists, i.e. the intersection between the two curves. (Online version in colour.)

(b) Optimal short-term comparative returns

To test whether traders' foraging choices respond to maximize their daily short-term comparative returns, we measured the extent to which the observed number of transactions per patch Qit agreed with the optimal transactions per patch Qit*. To find Qit*, we used the equality of returns from the exploration and exploitation curves to describe the intersection point of the curves in order to then estimate the expected optimal number of transactions. Mathematically, we calculated the value that maximizes Embedded Image given by (Qit*/2)(Qit*+1)(〈Nitβ2/Qit*) = (〈Nit〉/2Qit*)(〈Nit〉/Qit*+1) (Qit*β1), where 〈Nit〉 is the mean number of total transaction of trader i, and β1 and β2 are, respectively, the importance of exploration and exploitation taken from the multivariate regression model for each trader separately (table 1). Thus, the expected optimal number of transaction per patch is the positive root of (Qit*)2+Qit* (1−β1/β2)−〈Nitβ1/β2.

Interestingly, we found that exploration and exploitation choices can, in fact, be explained by traders trying to maximize their daily short-term comparative returns. We measured the deviation between the optimal Qij* and the distribution of actual values of Qij using the normalized model error (NME) for each individual case [44]. Here, the NME was computed as the difference between Qij* and the observed median value of Qij divided by the difference between the observed median value and the observed value of Qij at the 2.5 or 97.5 per cent quantiles, depending on whether the optimal value is lower or larger than the observed median value. The NME makes no particular assumption about the distribution of observed values. NME values between (−1,1) can be taken as cases where the optimal value is significantly similar to the observed values [44]. We found only three cases with NME values greater than 1 (figure 4). Importantly, this number of cases falls within the number of rejections [0,4] that one would expect with 95% confidence from a Binomial model B(30,0.05) [45]. Thus, one cannot reject the hypothesis that this model is a good approximation to the observed exploration and exploitation choices of traders. Broadly, our findings reveal that traders' choices can be explained by foraging heuristics that maximize their daily short-term comparative returns.

Figure 4.

Optimal versus real foraging choices. For each trader, the figure shows the NMEs between the optimal number of transactions per patch Qij* and the distribution of actual values of Qij. Here, the NME was computed as the difference between Qij* and the observed median value of Qij divided by the difference between the observed median value and the observed value of Qij at the 2.5 or 97.5% quantiles, depending on whether the optimal value is lower or larger than the observed median value. The NME makes no particular assumption about the distribution of observed values. NME values between (−1,1) can be taken as cases where the optimal value is significantly similar to the observed values [44]. We found only three cases with NME values greater than 1 (figure 4). Importantly, this number of cases falls within the number of rejections [0,4] that one would expect with 95% confidence from a Binomial model B(30,0.05) [45]. Thus, one cannot reject the hypothesis that this model is a good approximation to the observed exploration and exploitation choices of traders. (Online version in colour.)

Finally, to test whether traders' choices are associated with their net income intake, we introduced two additional return metrics. The first metric, which we called actual relative return Ait, provides information about the amount of money made by traders relative to the expected amount made by competitors. It is calculated similar to the short-term comparative returns measure except that it does not take into account the standard deviation; and instead, it multiplies returns by the number of stocks sold or bought. The second metric, which we called net income intake, Iit, is simply the amount made by traders; it is does not compare it with competitors. Figure 5a shows a significant and positive association between short-term comparative returns and actual relative returns, confirming that traders' choices respond to short-term competition with other traders. By contrast, figure 5b shows no association between comparative returns and net income intake, revealing a significant deviation between traders' short-term returns and their absolute returns. This suggest that traders' potential focus on short-term competition may come at the cost of missing net income optimizing opportunities.

Figure 5.

Association of daily short-term comparative returns with actual relative returns and net income intake. For illustrative purposes, (a) The positive and significant association between daily short-term comparative returns and actual relative returns for one single trader and (b) the association between daily short-term comparative returns and net income intake for the same trader. Correlation values for all traders are reported in table 1. Daily short-term comparative returns are given by Embedded Image. Actual relative returns Ait are calculated similar to the short-term comparative returns measure except that they do not take into account the standard deviation; and instead they multiply returns by the number of stocks sold or bought. Net income intake, Iit, is simply the amount of money made by traders; it does not compare it with competitors. Each symbol corresponds to one trading day. (Online version in colour.)

4. Discussion

Optimal foraging theory has proved useful for understanding how the fitness and survivability of animals depends on the trade-off between effort expended and absolute resources gained. It has further been shown that human and non-human animals rarely make the core foraging trade-off independently: their foraging choices are influenced by the choices their competitors make. Nonetheless, the study of the relationship between optimal and real foraging choices remains nascent. Here, we investigated whether the exploration and exploitation choices of day traders can be explained by short-term exploitative competition. Traders' foraging choices may be more abstract, stochastic and rapid than foraging choices in physical environments, yet the same mechanisms may underpin the allocation of vast financial and material resources under competition [32,4649].

Our study analysed the investing choices made by a cohort of 30 day traders at one firm. By analogy to foraging in the physical world, these traders sought to find the most beneficial compromise between the costs and benefits of continued foraging within a patch (i.e. consecutively buying and selling of the same stock) or switching to forage in a new patch (i.e. trading a different stock), where the returns to trading are affected by the foraging choices made by competitors. We measured traders' short-term comparative returns as the marginal difference between their actual returns to trading a stock and the mean returns possible based on the competitors' foraging choices in the market within a relevant period of time. We found that traders' short-term comparative returns are subject to an important trade-off between exploration and exploitation. We could not reject the hypothesis that traders' exploration and exploitation choices can be explained by traders following short-term choices that focus on maximizing their daily short-term comparative returns. While a complete determination of the drivers of these choices is beyond our analysis, one possible account for the observed behaviour is that traders first visit the patch in which they do best, then next best and so on. Thus, traders may choose patches that descend in worth, assuring at least early success, while limiting exposure to unpredictable shifts in competition in a patch that might create losses for the trader [43]. Such trading choices, however, may be different under new algorithmic trading where price transactions are previously fixed [37].

Foraging animals appear to optimally decide what patch of resources will offer the best returns to their efforts and how long to stay in a patch before moving onto the next best patch. Remarkably, our findings revealed that stock traders' trading choices can be explained by similar foraging heuristics that respond to short-term competition with other traders. However, there were important differences too. We found no one-best relationship between different trading choices and net income intake, suggesting that traders' choices can be short-term win oriented and, paradoxically, maybe maladaptive for absolute market returns [50,51]. This implies that traders' net income intake might be more strongly associated with global outcomes, social contagion or sporadic big losses and wins [28,29,32]. While the same problem is not true of animal foraging since the resources gained from each patch are also the net payoffs, it would be interesting to investigate whether maladaptive foraging behaviour can arise under rapid changing environments. In financial settings, it remains to see the extent to which this deviation between short-term choices and net income intake can influence the instability of markets.

Acknowledgements

We thank Alex Bentley, Esteban Freidin, Cristián Huepe, Rudolf Rohr and Michael Schnabel for useful comments on a previous draft. Funding was provided by the Kellogg School of Management, Northwestern University, the Northwestern University Institute on Complex Systems (NICO) and the Army Research Laboratory under Cooperative Agreement W911NF-09-2-0053. NSF VOSS grant no. (OCI-0838564). S.S. also thanks CONACYT.

  • Received December 4, 2012.
  • Accepted January 7, 2013.

References

View Abstract