## Abstract

Aggregation is a common life-history trait in open-water taxa. Qualitative understanding of how aggregation by prey influences their encounter rates with predators is critical for understanding pelagic predator–prey interactions and trophic webs. We extend a recently developed theory on underwater visibility to predict the consequences of grouping in open-water species in terms of increased visual detection of groups by predators. Our model suggests that enhanced visibility will be relatively modest, with maximum detection distance typically only doubling for a 100-fold increase in the number of prey in a group. This result suggests that although larger groups are more easily detected, this cost to aggregation will in many cases be dominated by benefits, especially through risk dilution in situations where predators cannot consume all members of a discovered group. This, in turn, helps to explain the ubiquity of grouping across a great variety of open-water taxa.

## 1. Introduction

Aggregation is a dominant feature of the life-histories of many organisms (e.g. flocks of birds, shoals of fish, herds of ungulates, clusters of insect eggs). Of the many selective benefits that group living can confer, those related to reducing predation risk (e.g. through collective vigilance, collective defence, predator confusion and risk dilution) seem the most ubiquitous (see [1,2] for reviews). However, these benefits to group living will be moderated or even nullified if predators (and other antagonists—such as parasitoids) can detect groups at greater distances than they can detect single individuals. There is a paucity of current theory and empirical exploration related to the ability of predators to detect groups of prey, and how that might be affected by different traits of that group. This is a significant handicap to understanding the ecological consequences stemming from aggregation as a widespread anti-predatory defence. Firstly, aggregation plays a facilitating role in human harvesting of natural populations; with many species targeted only because their tendency to aggregate makes harvesting economically viable [3]. Secondly, as species introductions, range changes and extinctions alter ecosystems, our ability to predict consequences will critically depend on our understanding of interspecific interactions (especially predation, given its ubiquity). Finally, the detectability of aggregations is also important in ecological contexts other than predator–prey or host–parasite interactions. For example, pollinators are attracted to larger aggregations of flowers from a greater distance away, and this has been linked to increased detectability (e.g. [4,5]). A similar effect has also been reported for attraction of seed-dispersers to aggregations of fruits (e.g. [6]). An improved quantitative understanding of how aggregation affects visual detection is vital in these contexts too.

### (a) Existing theory on visual detection of groups of targets

The most commonly cited theory on the issue of group visibility remains that of Vine [7,8]. Vine [7] considered a tightly packed chain of *n* individuals, each of width *l* and height *h*, which form a straight line of length *nl* and height *h* when viewed from the side. He argued that in terms of visual acuity the important dimension is the minimum one (*h* in this case), and thus the length of the line formed by the *n* individuals was irrelevant to its ease of detection, and detection rate would be independent of group size. In Vine [8], he admitted that a body of evidence existed suggesting that humans could more easily detect horizontal lines than points of the same height. He thus revised his argument to suggest that the maximum distance (*r*) at which a chain of individuals could be detected would take the form
where *A* and *B* were empirically determined constants. He suggested on the basis of experiments with human observers that *B* seemed to be of the order 0.40–0.45. A consequence of these values is that *r* initially rises steeply (*r* doubling as *n* increases from 1 to 5), but this effect quickly wanes in strength at larger group sizes (with highly reduced differences in *r* above *n* = 50). Vine pointed out that the consequences of having less-packed individuals so that there are gaps in the viewed aggregation were unknown, and this remains true. To this we would add that the contrast of individuals with the background will strongly influence detectability and is unexplored in this theory. Further, the ecological applicability of this work is limited, as only a small fraction of natural aggregations involve long chains of individuals.

Turner & Pitcher [9] provided the most influential theoretical work for the overall effect of aggregation on not just detection but prey capture rates. Their theory explored two simple alternative assumptions for the effect of group size on detection rates: assuming that the probability per unit time of a group of prey being detected by a nearby predator is either independent of group size (*n*) or increases linearly with *n*. The authors argued that these two alternatives likely bracket reality for most natural systems. However, the distribution of real-world cases within this very wide bracket remains unclear. Ioannou *et al.* [10] argued that detectability of a group should increase with the visual angle subtended by the group. However, they did not speculate on how increasing the visual angle will translate theoretically into ecologically relevant measures such as rate of detection. Treisman [11], on the basis of unpublished experiments with humans, argued that the probability of target detection increases linearly with increasing angular area of the target until a critical area is reached, after which further increasing area brings no further improvement.

### (b) Motivation for our work

Modelling vision is considerably more tractable in pelagic environments than other habitats, because the background against which objects are viewed is simple and predictable, and the detection range is strongly influenced by well-characterized patterns of absorption and scattering of light. Hence, there have been a number of theoretical predictions of pelagic visual detection (e.g. [12–14] and references therein). However, no previous study has explicitly explored the detection of a group of individuals. Recently, Nilsson *et al.* [15,16] have offered a general theory for the visual detection of objects in this environment. Here we build on that framework, extending it to the situation where an approximately spherical group of targets (such as a tightly packed school of fish called a bait ball) is detected visually by something with a camera eye (e.g. a cetacean or predatory fish) viewing it horizontally.

An understanding of the anti-predatory effectiveness of grouping is particularly important in the pelagic realm, the largest habitat on the planet. Grouping is a common life-history trait in this environment. With no physical structures to offer protection, prey aggregation is an important and common anti-predatory strategy. The consequences of aggregation for rates of discovery by predators are critical for understanding pelagic predator–prey interactions, and trophic webs. In the next section, we develop a new theory for the effects of group size and various ecological factors on maximum detection distance in an open-water environment. A key part of this theory is the attenuation of light as it passes through water, which is substantially greater than in air. Thus, the application of our theory is currently limited to aquatic systems, although it could be adapted to terrestrial and aerial situations where the background against which prey are viewed is relatively simple (e.g. snowfields, the sky, mudflats).

## 2. Material and methods

### (a) General theory

We model detection of a compact spherical school of fish (henceforth called a bait ball). We begin with the following definitions:

- Number of photons collected by the retina in one integration time from the target bait ball. We assume that the eye employs spatial summation to collect all photons from the ball in one big ‘pixel’. This is known as optimal summation [16], which maximizes detection range, and thus provides an upper bound for the effect of aggregation on visibility.
- Number of photons from the background water (over a pixel the same angular size as the bait ball pixel).
- Number of photons scattered into the path between the viewer and the bait ball as viewing distance increases. This is typically referred to as ‘pathlight’.
*C*_{0}- Inherent Weber contrast of the bait ball against the background water. Given by
*N*_{t}(0)/*N*_{b}− 1. For simplicity, we assume that the bait ball consists of enough individuals that it appears as a solid wall of opaque fish. Thus, the contrast of the ball equals the contrast of the individual fish. This contrast attenuates with distance*r*following , where*c*and*K*are the beam and diffuse attenuation coefficients of the water, and*θ*is the viewing angle of the predator (0° for looking directly upwards and 180° for looking directly downwards) [17].

The pelagic light field is approximately monochromatic at viewing angles greater than 48° from vertical (i.e. outside Snell's window) even at relatively shallow depths below the surface, and at all viewing angles at depths greater than approximately 100 m [18]. In these situations, the beam and diffuse attenuation coefficients of the water (*c* and *K*) can be considered to be approximately constant and equal to the values at the wavelength of peak penetration (480 nm in this study). In this case, the four terms defined above are related as
2.1and
2.2The first part of (2.1) is obtained from solving the Weber contrast equation; the second (exponential) part is obtained from the contrast attenuation equation given above. Equation (2.2) is from the pathlight equation for horizontal viewing [17,19]. Now, from [16]
2.3at the maximum sighting distance, where *R* is the reliability coefficient. The photoreceptor noise term introduced by Nilsson *et al.* [16] is negligible at the euphotic depths examined in this study (less than 200 m) and thus excluded. Substituting (2.1) and (2.2) into (2.3) gives
2.4

Combining terms gives
2.5As mentioned above, is the apparent contrast of the bait ball at viewing distance *r* and thus is much less than two at the sighting distance unless the light levels are extremely low, so (2.5) is well approximated as
2.6

From [16]
2.7where *T* is the diameter of the bait ball. For the viewing organism *A*, *q*, *τ* and Δ*t* are the diameter of the pupil, the quantum efficiency of the photoreceptors, the ocular transmittance and the integration time of the photoreceptors, respectively. The parameters *k* and *l* are the absorption coefficient and the length of the photoreceptors, respectively. *L*_{b}(*λ*) is the spectral radiance of the background light and *R*(*λ*) is the normalized absorbance spectrum of the photoreceptors. We define
2.8which is the number of photons absorbed by a pixel that views a region 1 sr in angular area. This can be thought of as the product of the sensitivity of the eye and the amount of light available for vision. Since the terms cannot be separated, due to the weighted integral, they are considered as one. Substituting equation (2.8) into equation (2.5) gives
2.9Now *T*, which is the diameter of the spherical bait ball, is related to the number of fish in the target group *n* via
2.10awhere *V*_{0} is the volume each fish occupies in the bait ball (including the fish and the surrounding water). This volume varies by species and swimming speed, but is approximately the cube of the body length of the fish for a school larger than 50 individuals [20]. Substituting (2.10*a*) into (2.9) and rearranging gives
2.10b

Gathering the constants and setting the reliability coefficient *R* to 1.96 (the value for 95% confidence of detection), we get
2.11awhich can be solved for *r* as
2.11b
2.11cfor horizontal viewing. *W*(*x*) is the Lambert *W* function (the inverse of ), which can be calculated using Matlab, Maple, Mathmatica and other computational packages.

### (b) Specific example parameter values

For the visual system of an Atlantic blue marlin predator (*Makaira nigricans*), representative values are: pupil diameter *A* = 0.019 m, integration time Δ*t* = 0.017 s, ocular transmittance *τ* = 0.8 and quantum efficiency *q* = 0.34. The photoreceptors of the marlin have a peak absorbance at 480 nm, an absorption coefficient of 0.035 µm^{−1}, and a length of 57 µm [21].

The background radiance spectra (*L*_{b}(*λ*)) were modelled using measured profiles of inherent optical properties and commercial radiative transfer software (HydroLight 5.1, Sequoia Scientific). The ability of radiative transfer theory to accurately model oceanic radiance distributions has been validated by *in situ* measurements of selected radiances and irradiances in multiple studies (e.g. [22,23]). The agreement between modelled and measured spectral radiances is particularly good in oceanic waters, which are easily characterized (reviewed by [18]).

Depth profiles of inherent optical properties and chlorophyll *a* concentration from tropical oceanic water (approx. Jerlov oceanic type I; [24]) needed for the radiative transfer software were obtained from Drs Andrew Barnard, Scott Pegau and Ronald Zaneveld (College of Oceanic and Atmospheric Sciences, Oregon State University, Corvallis, OR, USA), who collected them using a dual path, multiband absorption and attenuation meter (ac-9, WETLabs) and fluorometer in the Equatorial Pacific (0°0′ N 177°21′ W). Absorption and beam attenuation coefficients (at 412, 440, 488, 510, 532, 555, 650 and 676 nm) were measured to a depth of 199 m and chlorophyll *a* concentration was measured at 1 m intervals to a depth of 110 m (figure 1*a*).

Underwater radiance distributions were calculated from 400 to 700 nm at 10 nm intervals and the surface to 200 m depth at 10 m intervals. The sky was assumed to be cloudless, the wind to be 5 m s^{−1} and the sun at the zenith. The sky irradiance was calculated using the Radtran model [25], and the sky radiance angular distribution was calculated using the semi-empirical model given in [26]. Both models account for atmospheric effects, such as the reddening of the sun as it approaches the horizon and are well established. Pure water absorption was taken from [27], and the particle scattering phase function was an average-particle phase function based on measurements by Petzold [28]; tabulated values are given by Mobley ([18], table 3.10). Chlorophyll fluorescence was calculated from the measured chlorophyll *a* concentration using a modelled phytoplankton absorption spectrum taken from [29] and a fluorescence efficiency of 0.02 that was independent of excitation wavelength. Raman scattering by the water molecules was also included [30]. These values were used to calculate estimates of the number of photons captured per steradian per integration time (*N*_{0}). Figure 1*b* shows the values for three oceanic predators as a function of depth.

## 3. Results

In figure 2, we plot maximum horizontal sighting distance (*r*) as a function of the number of fish in the group (*n*, varying over three orders of magnitude from 10 to 10 000) and depth (from 0 at the surface to 200 m depth) for three values of the inherent Weber contrast between the fish and the background (0.25, 0.5 and 1). For reference, a contrast of 0.25 would be found in a relatively cryptic silvery fish, such as a herring or sardine, a value of 0.5 would be for a typical reef fish, and a value of one would be for a black fish. We solve equation (2.11) for the maximum distance (*r*) at which the group of individuals of approximate individual lengths of 10 cm can be detected. Our key results are, however, qualitatively unchanged for different sized fish. For example, by inspection of equations (2.9) and (2.11), we can see (unsurprisingly) that we predict that larger individual size of fish leads to longer sighting distances, but this effect is relatively modest, with the rate of increase being much slower than linear.

Our first key prediction concerns the inherent visual contrast of the prey against the background. It is unsurprising that *r* increases with increasing inherent contrast (*C*_{0}) of the prey. What is less obvious is that this effect is nonlinear: having a four-times greater contrast does not increase sighting distance fourfold. This is because sighting distance is related to a function of the natural logarithm of the contrast (the effect of change, by contrast, can be seen in greatest detail in figure 3). Also less obvious is that there is no strong effect of contrast on the shape of the *r*–*n* relationship, and so we would not expect the visibility costs of grouping to be inherently different for prey of different contrasts against the background. By inspection of equation (2.11), all three terms on the right will be the same in this regard: inherent contrast, the square root of photons (the weighted product of the intensity of illumination and sensitivity of the viewer) and the cube root of the bait ball volume (and so the length of the individual prey fish) all affect the relative sighting distance in a similar way. As a rule of thumb, the number of photons drops by a factor of 10 every 70 m in clear oceanic water [18], so the square root drops by a factor of approximately 3. Thus, as an example, cutting contrast to a third of what it was has the same effect as moving the bait ball 70 m deeper or cutting the number of individuals by a factor of 27. This line of argument may explain why schooling pelagic fish nearly always invest in mirrored scales that reflect much of the incident light to drop their inherent contrast considerably [31].

Our next key result is the effect of depth, with sighting distances being maximized at around 100 m, above that the dominant factor is higher attenuation of horizontally travelling light (high *c*) caused by suspended particles (e.g. phytoplankton), and below that the dominant factor is low incident light levels (leading to low *N*_{0}) caused by attenuation of sunlight as it passes through the surface waters above.

Our primary interest has been in predicting the relationship between maximum sighting distance (*r*) and group number (*n*). Unsurprisingly, *r* increases with *n* under all circumstances. It is also perhaps unsurprising that the *r*–*n* relationship flattens as *n* increases, but what is less obvious is that (even over the broad range of *n* considered) there is no saturation of the curve. That is, we can still see appreciable increase in *r* as *n* changes from 1000 to 10 000, for example. This is due to optimal summation, which allows the fish to make its visual pixel the same size as the bait ball. Thus, at least over the situations we model, there is no ceiling effect whereby after a group reaches a certain size, further increases in size do not increase the ease of detection of the group. However, of most interest is the relatively modest effect of increasing *n*: under all the circumstances that we explored, increasing the group size by two orders of magnitude (i.e. multiplying *n* by a factor of 100) causes *r* to rise by less than a factor of 2. Even this is likely an overestimate, since not all animals employ optimal summation. By assuming optimal summation we are finding the longest possible sighting range. The relatively modest costs of grouping in terms of increased visual detection may be relatively easily outweighed by benefits through risk dilution, collective vigilance and/or confusion effects, all of which have been demonstrated to increase rapidly with increasing group size (see Discussion and [1]). If maximum sighting distance doubles, then this would suggest that the volume of space over which the prey can be detected increases by a factor of 8. Thus, our model predicts that (as would be expected) a group of 5000 pelagic prey is more obvious to predators than a group of 50, and this should increase the rate at which the larger group is discovered by predators, but only by a factor of 8 or less, the exact number depending on the details of the predator's foraging strategy.

In figure 3, we show a greater range of values of contrast for three levels of *N*_{0}, corresponding (for our Blue Marlin viewer) to depths of about 50, 100 and 200 m (beam attenuation coefficient *c* is considered to be at a constant value of 0.1 for all three situations). This emphasizes that the inevitable rise in sighting distance with increasing group size can be counteracted by a decrease in the inherent contrast, leading to our prediction that the larger the characteristic shoal size of fish the stronger the selection pressure should be for morphological adaptations (most obviously mirrored scales) that reduce contrast.

It is important to note that our visual model allows the minimum contrast threshold of the viewer to go well below levels that have actually been measured. It may very well be that natural predators do achieve these low levels, but so far this has not been demonstrated in large pelagic predators. The lowest threshold measured for fish (and indeed for any animal) is 0.005 [32]. Figure 4 shows the sighting distance versus depth and group size using the same procedures as used to generate figure 2 but with the additional constraint that the contrast threshold of the viewer cannot go below 0.005. As can be seen by comparing figures 2 and 4, this added constraint does not change any of our qualitative conclusions. Interestingly though, group size does not affect sighting distance at all at shallower depths under this constraint. For the viewer to get any advantage when viewing larger groups at these depths, its contrast threshold would have to be very low indeed. One thing that is obvious from both figures 2 and 4 is that water clarity has the biggest influence on sighting distance, because it is the only variable outside the (very slowly increasing) Lambert *W* function. This is why the schools can be seen at greater distance at depth despite it being darker, so long as the water is clearer, an effect commonly experienced by scuba divers as they drop below the murky surface layer to the darker but clearer depths.

## 4. Discussion

The main prediction of our model is that, in general, a 100-fold increase in the number of individuals in a group will only lead to at most a doubling in the range at which prey are visible to a predator and so (in a worst case, where individuals could be detected from all angles) the larger group might be detected eight times as frequently as the smaller. We now consider the anti-predatory benefits of grouping for comparative purposes.

The benefits of risk dilution can sometimes be substantial. If the predator is relatively small in comparison with the prey and not as fleet as the prey, then it may only be able to capture a single individual from a group. In this case, the dilution benefits of being in a group a hundred times larger (and having the risk of being the selected individual reduced by a factor of 100) will far outweigh the eightfold increase in frequency of encounter of the group with a predator. However, at the opposite extreme where the predator is large (or hunts in packs) and fleet compared with the prey, then all of a discovered group may be consumed and there is no dilution benefit to grouping. In general, available empirical evidence (reviewed in [2]) suggests both these extreme situations are commonplace, and ecologically and taxonomically widespread. We can conclude that in cases where a single attack captures only a single individual or small fraction of the prey group, and a predator cannot repeatedly attack a discovered group, then dilution benefits will exceed the visibility costs estimated here.

Although predator confusion resulting in a reduced ability to capture prey when faced with larger moving groups has often been demonstrated (see Beauchamp [2], for a review), the effect of group size has rarely been quantified, and current theory does not allow strength of confusion and prey survival to be quantitatively linked [33]. However, the confusion effect can be strong. In the most thorough study of the effects of group size, Landeau & Terborgh [34] demonstrated that predatory bass were always successful in quickly capturing a single minnow when both were in an experimental arena together. By contrast, this success rate (for capturing every single minnow) dropped to 11% despite an extended time for interaction when the prey group size was increased to 15. Our model suggests that such an effect could again dominate the cost of increased ease of detection of larger groups.

If we turn to increased vigilance as another anti-predatory benefit of grouping, the most relevant data from the recent extensive review of Beauchamp [2] is that of Kenward [35] on the characteristic distance at which flocks of woodpigeons reacted to apparent attacks by a trained goshawk. Single pigeons reacted on average when the goshawk was only 4 m away, this distance increased fourfold for flocks of between two and 10 birds and 10-fold for flocks of more than 50. Interpretation of such data is complicated because there may be a lag between detection of the approaching predator and flight response, but this is likely to be low in this system where predators are much more successful if they can pin prey to the ground, and in any case such a lag is likely to be bigger for large flocks where risk dilution will be substantial. However, as with confusion, it is difficult to quantify the relationship between early predator detection and prey survival. Clearly, there is a dearth of data quantifying how vigilance benefits of aggregation change with aggregation size, but given the modest costs of increased detection estimated here there is at the very least no reason to reject the possibility of vigilance benefits outstripping these costs. However, vigilance for predators is particularly beneficial in situations where forewarned prey can flee to a place of safety, and this option is generally not available in pelagic environments.

Finally, another factor that mitigates the cost of larger groups being detectable at a greater distance is that, for finite prey populations, increases in group size correspond to a decrease in the total number of groups in the environment. This reduction in the density of groups means at any one time a predator will be a greater distance on average from the nearest group [9,10]. However, evaluation of the consequences of this for predators and prey would require consideration of how such aggregation changed not just average distance from prey but also predator activity budgets and search strategies (i.e. in terms of speed and direction of travel during searching). This is an open but tractable problem theoretically, which (in common with all work on predation) should benefit from a step change in our ability to collect data on free-living animals through miniaturization of on-board data-loggers [36]

Finally, we also note that the benefits of remaining in a group often appear to hold across a broad range of group sizes, and after attack on the group has begun. Observation of bait balls suggests that the tenancy to aggregate remains even as the ball of fish is whittled away by a group of predators [37].

In summary, we have been able to offer an estimate of the likely consequences of grouping in open-water species in terms of increased visibility of groups to predators. Our model suggests that such enhanced visibility will be relatively modest, with maximum detection distance only doubling for a 100-fold increase in the number of individuals in the group. This suggests that although larger groups will probably be detected and attacked more often by predators, the costs of grouping will in many cases be outweighed by the benefits through (some or all of) risk dilution, predator confusion and enhanced collective detection of approaching predators. This helps to explain the ubiquity of grouping across a great variety of open-water taxa—the greatest predation cost to this behaviour is likely to be dominated by expected benefits.

## Competing interests

We declare we have no competing interests.

## Funding

We received no funding for this study.

## Acknowledgements

We thank Eleanor Caves, Julia Notar, Sarah Solie and Kate Thomas; Drs Nicholas Brandley, Robert Fitak, Daniel Speiser and Eric Warrant; and two anonymous reviewers for valuable comments.

- Received June 29, 2016.
- Accepted August 26, 2016.

- © 2016 The Author(s)

Published by the Royal Society. All rights reserved.