Theoretical investigations of competitive dynamics have noted that numbers of predator and prey influence each other. However, few empirical studies have demonstrated how a life-history trait of the prey (such as fecundity) can be affected simultaneously by its own density and the density of predators. For instance, density dependence can reduce fecundity with increasing number of prey, while inverse density dependence or Allee effects may occur especially when the prey is a social organism. Here we analysed an intraguild predator–prey system of two seabird species at a large spatio-temporal scale. As expected, we found that fecundity of prey was negatively affected by predator density. Nevertheless, fecundity of prey also increased nonlinearly with its own density and strikingly with the prey–predator ratio. Small groups of prey were probably not able to defend their nests especially against large number of predators. At the highest prey densities (i.e. when anti-predator strategies should be most efficient), prey fecundity also lowered, suggesting the appearance of density dependence mediated by food competition. Allee effects and density dependence occurred across a broad range of population sizes of both the prey and the predator at several local populations facing different ecological environments.