Limited Data, the NRA Annual Meeting, and Ignoring Intent: The Latest Study

Have you heard the latest breaking news? 

“A study pokes holes in the idea that experienced firearm users are less likely to injure themselves.”

“To see gun injury drop, hold an NRA Meeting.”

“Gun injuries fall during NRA conventions.”

A Harvard doctor and a Columbia grad student published a letter in the New England Journal of Medicine this week that found gun injuries drop 20% during the NRA’s Annual Meeting. The researchers used more than 75 million patient observations over a nine-year span, using the three weeks before and after the Annual Meetings as a control.  The argument coming from this letter, as noted in Scientific American:

If guns were perfectly safe in the hands of trained NRA members, Jena and Olenski reasoned, they should have found no differences between gun injury rates on convention days versus other days. Yet injury rates were, on average, 20 percent lower on meeting days. “We believe this is due to brief reductions in gun use during the dates of these meetings,” Jena says.

In other words, the researchers are using NRA’s focus on safety and training as part of a natural experiment. They believe that Annual Meeting attendees – all of them NRA members committed to safety, training, and responsible firearms ownership – abstaining from using firearms reduces the number of firearms-related injuries. 

Usually, one would look at the data behind the analysis. This may be an unusual case, and not just because we don’t have access to the subscribers-only database the researchers used. Basic logic and math calls their findings into serious question. 

Start with their premise: that firearms injury would decline during a period of firearms abstinence – the Annual Meeting. The researchers obviously have not attended an Annual Meeting or read much about the event; if they had, they would know that there is no prohibition on carrying firearms at the Annual Meeting. NRA actively schedules the Annual Meeting in cities and venues that respect the right to keep and bear arms.

So, we’ve established that the people at Annual Meeting may be armed. What about the magnitude of the crowd size? About 80,000 people attend in a given year. There are about 100 million or so gun owners in the country, so the researchers are claiming that less than one-tenth of one percent of firearm owners are responsible for a 20% drop in the firearms-related injury rate nationwide. Whatever nonsense they conducted with the data and their methods, this finding flies in the face of common sense and logic.  It would be laughable if not so completely absurd. 

The authors suggest that there is a trickle-down effect of sorts. They believe that going to a shooting range, hunting, plinking, or carrying a firearm for self-defense doesn’t occur during the Annual Meeting.  Again, this is absurd.    

But even more extreme, these researchers actually hypothesized that the level of “non-engagement” with firearms and the shooting sports would be widespread enough to contribute to a measurable difference in the firearms-related injury rate nationwide. 

 even beyond common sense, the methodology used by these researchers is very unusual

This brings up an interesting point. While the researchers use some mysterious variable to account for state firearms ownership rates (a number that doesn’t actually exist), they don’t control for firearms usage at all. They see 80,000 Annual Meeting attendees and work from there, but not all of those attendees are firearms owners. Some attend with their family or significant other. And among the firearm owners attending, many are carrying concealed.  That is, they remain actively engaged in the use of firearms while they attend. There is no “stoppage” of folks exercising their Second Amendment rights.  Quite the contrary.

But even beyond common sense, the methodology used by these researchers is very unusual. Instead of publicly available injury data from reliable government sources, they use a proprietary database of emergency department visits and hospitalizations among private insured patients. Guess what?  This means not everyone in the country.  Not even close. 

They acknowledge that privately insured patients account for only about a third of all unintentional firearms-related injuries (though they use all injuries regardless of intent). Their population is skewed female and southern, and the western U.S. is underrepresented. Their study period also covers the Great Recession, in which a significant number of people lost their jobs and, presumably, their privately held insurance. 

Oh, and one last point.  Which “injuries” counted for these researchers?  You’d think that true firearm-related accidents, but you’d be wrong on this. In addition to accidents, they included legal intervention and terrorism.  If legal intervention sounds like self-defense to you, it does to us too, as that phrase is commonly used in this way.  It is hard to tell because the authors are not forthcoming about why they included these injury codes in the analysis, so we can only guess. And why injuries from terrorism were included, again, is anyone’s guess.

Oh, and by the way, the Annual Meeting has actually been found to reduce crime in the host city. Louisville serves as a great example.