Group selection was decisively rejected on theoretical grounds, according to the patriotic history of individual selection theory. Richard Dawkins declared in 1982 that group selection had "soaked up more theoretical ingenuity than its biological interest warrants" and compared further inquiry to the futile search for a perpetual motion machine. Richard Alexander stated in 1987 that "a great deal of convincing theory suggests that any such view [the beneficence that evolves by group selection] will eventually be judged false" (see T&R V for more).
Blustery statements like these have a bad smell about them. In truth, the models that led to the rejection of group selection were little more than sketches on the backs of napkins. Models supporting the plausibility of group selection began to appear as early as the 1970s, but that did not stop the patriotic propaganda machine from rolling onward.
Before I support my claim, a word about theoretical modeling is in order. Some models attempt to approximate reality in as much detail as possible, but most evolutionary models are more like caricatures. Just as a good artist can capture the likeness of a person with a few lines, a good modeler attempts to capture the essence of a problem such as group selection with the fewest possible assumptions. This kind of model tells us what can happen, given the assumptions of the model. It says nothing about what might happen, given other assumptions. Many models are required to support the sweeping claim that "between-group selection is invariably weak, compared to within-group selection."
Against this background, let's take a look at the most influential model that led to the rejection of group selection, which is affectionately known as the haystack model. The year was 1964. Two years earlier, V.C. Wynne-Edwards had published his book Animal Dispersion in Relation to Social Behavior, which made sweeping claims about group selection (see T&R III). One year earlier, a British graduate student named William D. Hamilton published a note in the journal American Naturalist outlining how altruism can evolve among individuals that share the same genes. John Maynard Smith, one of the top evolutionists in the UK, was familiar with Hamilton's work and thought that it provided an alternative to group selection. He, therefore, wrote a critique of Wynne-Edwards in a letter to the prestigious journal Nature titled "Group Selection and Kin Selection". The letter was only three pages long and included a brief model, which was truly little more than a sketch on the back of a napkin.
Like Dr. Seuss, Maynard Smith asked the reader to imagine a cartoon world in which a species of mouse lives entirely in haystacks. Each haystack is colonized by a single fertilized female, whose progeny reproduces for a number of generations. At some point, all of the mice from all of the haystacks disperse, mate on a population-wide basis, and the cycle is repeated for a new set of haystacks. There are two genes in the population, one coding for aggressive behavior and the other coding for docile behavior. Aggressive mice outcompete docile mice within each haystack, but they also decrease the growth rate of the group. Specifically, every haystack is colonized by four genes; two from the female and two from the male who fertilized her. Maynard Smith assumed that if one or more of these genes is the aggressive gene, then the aggressive gene completely replaces the docile gene by the time dispersal from the haystack occurs. When a haystack is colonized by four docile genes, however, more mice are produced by the time dispersal occurs than from the other haystacks. Imagine those docile mice streaming away from their haystacks singing "Kumbaya", while the aggressive mice limp forth from their haystacks nursing their wounds!
Let me praise the haystack model before presenting and critiquing the results. The essence of the group selection controversy -- what I called "the original problem" in T&R II -- is whether a trait that is "for the good of the group" can evolve in the total population, despite being selectively disadvantageous within groups. The haystack model elegantly captures the essence of the original problem. In this sense, it is a good cartoon. Docility is for the good of the group. Aggressiveness is selectively advantageous within each group. Which level of selection prevails?
Maynard Smith concluded on the basis of his model that aggressiveness almost always wins. Group selection is an evolutionary force, but it just can't prevail against the opposing force of selection within groups. In contrast, Maynard Smith felt that Hamilton's theory provided a more plausible explanation for the evolution of altruism. It was Maynard Smith, not Hamilton, who coined the term "kin selection" to contrast it with "group selection".
The haystack model had an enormous impact on the rejection of group selection and the conceptualization of kin selection as an alternative to group selection. Nevertheless, at least one of its many assumptions is fatally biased. Nobody noticed at the time. Can you figure it out in hindsight?