On Aug. 26, the fatal shooting of two journalists during a live broadcast in Virginia sparked a new wave of debate about gun violence and a flurry of reportage on the alarming frequency of mass shootings in the United States. The Washington Post most notably reported that, with 248 mass shootings having occurred in the 238 days of this year, we are now averaging more than one mass shooting per day.
This figure has certainly helped to emphasize an issue in need of address. After all, of the 12 deadliest mass shootings in the U.S., half have occurred in the past eight years—yet in the year after 26 people, including 20 children, were killed in Newtown, Conn., U.S. states passed 70 laws loosening gun restrictions.
But as powerful and shocking as “more than one mass shooting per day” may be, it also serves to illustrate the need for maintaining a certain level of consistency and specificity when examining data, particularly regarding how terms and events are defined.
A single definition can shape an entire narrative. The Washington Post’s statistic is based on records compiled by a GunsAreCool (GrC)-crowdsourced mass shooting tracker, which describes “mass shooting” as an incident in which four or more people are shot in a single spree or setting.
The official definition used by the FBI, however, is much more stringent, stipulating that an event is a mass shooting only if three or more people are killed.
In fact, when the FBI’s definition is used, the number of mass shootings that have taken place this year drops to 46—an average of just under one mass shooting every five days. (Out of the 248 shootings cited by the Post, 101 resulted in no deaths, 77 in one death, and 24 in two deaths.)
And what happens when we use a definition somewhere between the two—broader than the FBI’s but slightly less inclusive than the one GrC gave us? Say we only consider shootings in which either three or more people were killed OR at least five people were shot. Now only 138 events are considered mass shootings; this represents triple the number identified using FBI criteria and approximately 56% of the Post’s statistic.
The results in all three cases are varied enough to affect how we can analyze trends in gun violence. Consider the number of mass shootings in 2014, between Jan. 1 and Aug. 26. GrC would argue that there were 223 mass shootings, meaning that in the past year, we have seen an 11% increase in the frequency of mass shootings.
The FBI, on the other hand, would count 35, which may be a lower number but ends up pointing to a much more significant increase in frequency: 31%. The stricter definition ultimately calls attention to a more “sensational” trend, even though the more inclusive definition yields data that seems on the surface to have higher shock value.
Why should we prefer the stricter definition? While the FBI’s definition may not cast as wide a net as GrC’s, it does perform more rigorous data analysis by identifying mass shootings based on behavioral and situational trends and not just statistics. For instance, a study released by the FBI last September, which examined 160 shootings between 2000 and 2013, found that an average of 16.4 mass shootings occurred annually from 2007 to 2013—a sharp increase from an average of 6.4 shootings each year between 2000 and 2006. This comparison was made possible by the fact that the study differentiated between mass shootings and shootings involving domestic disputes or gang violence (although this study comes with its own share of criticism).
On the other hand, it is difficult to draw comparisons across years using the lists generated by GrC—not only because there is not yet enough data (the site started tracking shootings in 2013), but also because the motivations behind the shootings have not been differentiated. An upward trend determined by GrC data could mean an increase in domestic or gang violence, but not necessarily in “mass shootings” as they are understood behaviorally.
The ramifications of such over-counting do not pertain solely to event frequency. A September 2014 report published by the Crime Prevention Research Center (CPRC) shows how mass shooting data that is too inclusive can skew conclusions about the percentage of mass shootings that occur in gun-free zones and correlations between mass shootings and previous diagnoses of mental illness.
The problem with overly-inclusive definitions is one we’ve seen many times before. Last year, for instance, the Los Angeles Times found that claims of increasing deportation rates under President Obama relied on misleading statistics. The misinterpretation stemmed largely from an inconsistent and overgeneralized definition of “deportation.” By making the distinction between the deportations of people who had already found residences and jobs in the U.S. and the deportations of people who had just crossed the border, the LA Times was able to map more accurate trends in U.S. immigration policy. The inclusion of recent border-crossers in the definition of “deportation”—when older statistics did not take those cases into account—had skewed the data and inflated previously reported rates of deportation.
Meanwhile, by studying each type of deportation separately, it is possible to draw more accurate—and often more interesting—conclusions about the effects of certain laws and policies. It turns out, for example, that while the overall number of deportations actually decreased under Obama, deportation policies became more punitive (for instance, there was an increase in the number of deported immigrants who faced longer bans from re-entry).
Using consistent and specific definitions is important in measuring almost any trend, from unemployment rates (the definition of “unemployment” is not fixed, and can include individuals who work irregularly or exclude individuals who are not actively seeking employment) to international comparisons of test scores (scores are often dependent on socioeconomic status and therefore cannot be analyzed in a vacuum).
To return to mass shootings, then—yes, an average of more than one mass shooting per day is certainly a shocking statistic, and one that has the power to emphasize the gravity of the problem at hand. But it is also misleading. We can see that if used in further analysis, these numbers can actually conceal important information (such as the correlation between extended background checks and a decrease in gun violence, or between stricter gun legislation and lower crime rates) that is just as vital for addressing the issue of gun safety in the U.S.