The question of why territorial residents usually win asymmetrical owner-intruder contests is critical to our understanding of animal contest evolution. Game theory suggests that, under certain conditions, residency could be used as an arbitrary means of contest settlement in a manner analogous to tossing a coin. Key empirical support for this idea is provided by a study on the speckled wood butterfly (Pararge aegeria); however, this result has proven controversial. We show conclusively that residency does not serve as an arbitrary cue for contest settlement in this species. By means of a series of manipulative experiments, conducted on two phenotypically divergent populations of P. aegeria, we also rule out the recently presented alternative that contests are settled due to resource-correlated asymmetries in thoracic temperature. Our results instead suggest that more intrinsically aggressive males accumulate as residents and continue to win due to the self-reinforcing effect of prior winning experience. Truly arbitrary contest settlement may be rare or non-existent in the wild.