Sunday, October 28, 2012

Steven Pinker and archaeology

Traveling in a hammock, Belgian Congo
National Museum of African Art
Smithsonian Institution
This is my second post on Steven Pinker’s Better Angels of Our Nature (the first is here). This book is such a steaming pile, I need to vent but really don't want to spend time on it.  So I am going to end this particular foray into the colonial mindset and finish up on Chapter 2, which is the main archaeological and anthropological section. In this chapter Pinker argues that there has been a significant decrease in violence as a result of the development of the state. He presents his main evidence for this in a chart—Figure 2-2 in the book, but the same data is presented in two separate charts in an Edge talk he gave. The archaeological charts is below, but it is probably best to look at the chart in the book.

In my earlier post on the topic, I went over some of the more general conceptual issues Pinker faces in comparing non-state/state violence.  In this post I am concentrating on the actual quality of his evidence.  I am just going to talk about the archaeological data, otherwise things are going to get totally unmanageable, and I will never get to stop writing about the book. 

Pinker's evidence that the development of the state led to a significant decrease in violence is gleaned from a grab-bag of archaeological and ethnographic studies--studies that focus on human violence. He uses different datasets at each stage of his chart, none of which I think he really understands. His data for prehistory is the percentage of skeletons showing signs of fatal violence within each site, or group of sites.  The data for modern hunter-gatherer and horticulturalist societies is for the most part ethnographic data gathered through oral histories and reports.  The data for states is probably the most reliable (other than the Ancient Mexico figure). A big problem is here is that, although Pinker intends the chart to compare violent deaths in state and non-state societies, the only actual “state” included in the chart is the US (in 2005). The data are all harvested from secondary sources, some of which are highly questionable, and none of which were trying to answer Pinker’s basic question (actually it is an assumption, but let’s be charitable) —”is there a decline in violence from non-state to state societies?”

Even to those without an archaeological, anthropological, or historical background Pinker’s Figure 2-2 must appear dubious.  The big issue is figuring out exactly what is being compared here; what are the categories on the X axis and what is being measured on the Y axis?  The entities being compared are very different.  We start off with prehistory being represented by cemetery sites, or clusters of cemetery sites, leap to 19th/20th century hunter-gatherer/horticulturalist societies, then we come to “states.”  Notice that of the eight categories here, only one, the “U.S.  2005” is actually a state.  The rest are geographical regions (Ancient Mexico, Europe, and the World).  Pinker has made no effort to maintain an even roughly consistent scale of comparison.  This is important since the bigger the sample, the more variation will wash out.

The time frames represented by each category also fluctuate wildly, from days at one end up to centuries at the other.  What sense does it make to compare a mass-burial from a one-day massacre (the Crow Creek Site) to the planet Earth in the entire 20th century?  What possible conclusion can one draw from such a comparison?  

In the following table, I’ve listed the archaeological sites he uses in his argument.  I reviewed his original sources and, where I could figure it out, included the number of individuals in each cemetery (data that he really should have included).  There are some issues with his data that I need to address before proceeding.  A relatively minor, but still  significant, problem with this figure is how poorly it is cited.  One has to go through a pile of references hoping to find a match.  Fortunately Pinker used only three sources for his archaeological and ethnographic data—Lawrence Keeley’s War Before Civilization: The Myth of the Peaceful Savage (1996), an article in Science by  Samuel Bowles ("Did Warfare Among Ancestral Hunter-Gatherers Affect the Evolution of Human Social Behaviors?"), and Azar Gat’s 20008 War in Human Civilization.  Even so, I couldn't find all the sites in these books, so the chart may not be fully cited.

# burials
deaths by warfare
Nubia, nr. Site 117
12000-10000 BCE
Djebel Sahaba Nubia (Site 117)
2000-10000 BCE
Calumnata, Algeria
6300BCE-5300 BCE
Gobero, Niger
14000-6200 BCE
Volos’ke Ukraine
7500 BCE
Vasiliv’ka Ukraine
9000 BCE
Boggebakken Denmark
4300-3800 CE
Vedbaek Denmark
4100 CE
Skateholm I, Sweden
4100 BCE
6000 BCE
Ile Teviec France
4600 BCE
British Columbia 30 sites
3500BCE-1674 CE
Central California
1500BCE-1500 CE
Central California
1400BCE-235 CE
Central California (2 sites)
240CE-1770 CE (?)
S. California 28 sites
1400 BCE-235 CE
Kentucky (2750 BCE)
2750 BCE
Crow Creek, South Dakota
1325 CE
1300 CE.
Northeast Plains
1435 CE
Sara Nahar Rai, India
2140-850 BCE

I've grouped my carping about the archaeological evidence into three main categories: (1) Pinker's understanding of the data (2) Pinker's sample size, and (3) the manner in which Pinker selected his sample.

1) Poorly understood data

Djebel Sahaba (Site 117) and the “nr. Site 117” Site.  These sites should be grouped.  they probably belong to the same community.  Pinker uses grouped data in other cases, so he probably should here.  Or Djebel Sahaba should be dropped as representing something exceptional, either the remains of a battle or massacre, or the community buried people who died by violence separately.  Speaking the sites that might represent exceptional events…

Crow Creek, South Dakota.  This is a notorious site.  It is a mass grave, the remains of a massacre.  Using this site as just automatically representative of prehistoric mortality is senseless, especially when your sample is already small and biased.  There are no mass graves from Europe in the 20th century?  What would Pinker’s state-level death rates look like if he used cemetery data (as he should have)?

Vedbaek, Denmark, and Boggebakken, Denmark.  This is bad.  Unless there are two Mesolithic sites with 17 burials in Vedbaek, these are the same site.  Keeley calls it Vedbaek; Bowles calls it Boggebakken. The site is sometimes called Vedbaek-Boggebakken (e.g., Jochim 2011:127).  If these are the same site  (as they certainly seem to be), this is probably the most egregious error in Pinker’s data.  The rest of the errors can be attributed to lack of understanding of the evidence and appropriate methods.  But, this, THIS, is sheer laziness. He made no effort to clean his data.

As a side note, this mistake shows the importance of not just dumping other people’s numbers in to a hopper—“Vedbaek” (from Keeley) is 14% mortality, while “Boggebakken” (from Bowles) is 12%-- different numbers for the same site. This is probably because Bowles was only counting adults.  If you are going to use numbers from different studies, make sure the numbers mean the same thing. 

2) Sample sizes

Another issue to be aware of, especially if one quantifying and comparing data, is sample sizes.  Pinker doesn’t include how many individuals are represented by each cemetery in his sample, so I trawled through the sources to recover that information (information that should have damn well been in the book).  The problem here is the smaller the number of individuals in the cemetery, the greater the statistical impact of an individual showing evidence of violent death.  If there are only ten individuals in a cemetery, the percentage will jump by 10% increments.  Those are big jumps.   Note Sara Nahar Rai has a total count of only 10.  Ile Teviec only 16.  Boggebakken has a count of 17 (as does Vedbaek, for obvious reasons :) ).  Volos’ke has 18.  We need to consider whether these are adequate samples, especially when the comparative cases are continents and the entire world.  For Pinker’s analysis, which boils down to visual comparisons of percentages of uncorrected data (with the prehistoric data being summarized as a mean without error bars), I would be hesitant to use anything less than 50, simply because the statistical impact of single deaths is going to be exaggerated. 

3) Sample selection

This is probably the single largest problem with Pinker’s evidence.  In any kind of quantitative study, a first consideration is sample selection.  Is the sample biased?  Is it representative or has the author cherry-picked only the data that supports their position?  What is immediately obvious in reading Pinker’s discussion of his data is the absence of any kind of methodological discussion, and how few sites are presented as evidence.  Those are two big, bright red flags.  Pinker presents his methods for sample selections as being simply the sites “that he knew of”.  The criteria of selection is simply that he knew of the sites, more precisely he knew of these sites because they were given in tables in Keeley (1996) and Bowles (2009)—two sources dealing with violence in prehistory. Keeley was interested in demonstrating the existence of violence in prehistory, not its extent.  For this goal it was adequate to simply find sites where violence existed.  It is basically a collection of anecdotes.  We cannot use his collection of sites to infer the extent of violence because he only picked sites that showed evidence of violence.  The sample is fine for Keeley’s argument, but not for Pinker’s.  Bowles, beyond noting that he “studied all available archaeological and ethnographic sources that present (or are cited as presenting) relevant data”, does not discuss his sample selection at all, which is surprising for a Science article.  The emphasis of his article is on intergroup violence, and his sample is certainly skewed towards cemeteries that show signs of violence.  

If these are the only sites that Pinker “knew of”, he did not do enough research.  What he needed here was some kind of sampling design.  In relying solely on Keeley and Bowles, (as well as Gat for the ethnographic data), Pinker guaranteed, whether he realised it or not, that he would have a dataset consisting of practically nothing but cemeteries that show evidence of violence.  I don’t know how many prehistoric cemeteries have been excavated, and I don’t how many show no evidence of violence (hundreds at least, maybe thousands). Pinker needed a sampling program that would take  into account a population of all known cemeteries, not just those that showed evidence of violent death.  Given Pinker’s selection of the data, it is a given that non-state societies will be more violent, but this is the result of cherry-picked evidence, not necessarily reality.

I think, since WWII, we have seen an overall decline in death by violence.  Does this 60 years translate to “states are less violent than non-state societies.”  Of course not.  I am not sure it is even a meaningful question due to the different ways in which states and non-state societies deliver violence, and due to historical variation.  It may be possible to answer this question, or at least weigh in on it, but Pinker hasn’t done so.  He is not even wrong. You cannot pull something like this off without a great deal of historical understanding. If Pinker has such an understanding, it is not evident in Better Angels.  Instead he approaches the historical disciplines as factoid mines, carefully picking out the numbers he wants, and tossing them into an ideological hopper.  Better Angels fails as science and as history at a very basic level.   

References cited

Bowles, Samuel
2009     Did Warfare Among Ancestral Hunter-Gatherers Affect the Evolution of Human Social Behaviors? Science 324(5932):1293–1298.

Gat, Azar
2008     War in Human Civilization. Oxford, UK, Oxford University Press.

Jochim, Michael
2011     The Mesolithic. In European Prehistory: A Survey, Sarunas Milisauskas, editor, pp. 125–152. New York, Springer.

Pinker, Steven
2011a     A History Of Violence Edge Master Class 2011. Edge.

2011b     The Better Angels of Our Nature: Why Violence Has Declined. First edition  New York, NY, Viking

No comments:

Post a Comment