Explore The NRA Universe Of Websites

APPEARS IN News

Gun Control Research Methods: Close Enough

Monday, November 16, 2020

Gun Control Research Methods: Close Enough

Social science methods are not perfect, but that has never stopped anti-gun researchers from making outlandish claims. Most recently, a team of researchers led by Doctor Apurva Bhatt from the University of Missouri Kansas City sought to evaluate the effect of changes in Missouri firearms laws on suicides among teenagers and young adults. Their analysis is burdened by a fundamental disregard of applicable firearms laws and their findings are so suspect that even uninvolved anti-gun researchers have formally acknowledged the problems. 

Let’s begin with the results as claimed by the researchers in the first-page summary of their findings:

“Repeal of the PTP [permit to purchase] law was associated with a 21.8% increase in firearm suicide rates in young adults aged 19 to 24 years in Missouri. Lowering the minimum age of concealed carry to 19 years in Missouri was associated with a 32.0% increase in firearm suicide rates and a 29.7% increase in nonfirearm suicide rates in adolescents aged 14 to 18 years, and a 7.2% increase in firearm suicide rates in young adults aged 19 to 24 years.”

We’ll unpack these findings a bit but – spoiler alert – they didn’t include all of their results in that summary. Surely it was a simple oversight, but we’ll get to these other findings.  

Bhatt and company here claim that the 2007 repeal of the requirement that a law-abiding adult obtain a permit to purchase a firearm is associated with a large increase in firearm suicide rates among people between the ages of 19 and 24. This, on the surface, aligns with what anti-gun activists and researchers insist must be true: that less onerous requirements to obtain firearms must lead to more deaths involving a firearm. The key to this study is what Bhatt and team have intentionally excluded from this summary. You see, according to the researchers in Kansas City, the repeal of this law is also estimated with a decrease in firearms-involved suicides among teenagers aged 14 to 18. If the law were really related to the suicide rate, one would expect similar changes among both populations.

But don’t take it from us. Take it from widely published researcher and new CDC grant recipient Doctor Ali Rowhani-Rahbar. He said, “This finding should be interpreted with caution, especially considering the lack of traditional measures of uncertainty when using the synthetic control approach. If not explained entirely by chance, one would want to understand why the consequences of repealing the PTP law might be qualitatively in different directions for individuals aged 14 to 18 years compared with those aged 19 to 24 years. This repeal does not seem to coincide with additional safety measures at home for families with youth, questioning the mechanism behind these seemingly opposing associations.”

Like, Dr. Rowhani-Rahbar, we’ll also question that mechanism. Bahr and company used a synthetic control methodology which, in short, means they used a statistical analysis program to build a version of Missouri comprised of other states with similar laws and other similarities across a very limited number of additional variables. The “synthetic” version of Missouri used to model teenage firearm suicides was 46% New York. 

The second aspect of this new study concerns the twice-lowered age threshold for concealed carry permits in Missouri. The age requirement was lowered from 23 to 21 years old in 2011 and later lowered to 19 years old in 2014. According to the researchers, lowering the age for concealed carry permits to 21 is associated with a decrease in the firearm suicide rate among teenagers aged 14 to 18 AND among adults aged 19 to 24.  This reduction is also associated with a decrease in the non-firearm suicide rate in the older of these two populations but an increase in the non-firearm rate among the younger cohort. Again, differing affects by age cohort. 

Don’t remember reading these in the results summary? Of course not. The researchers left it out. Instead, they only mentioned the findings from the second age reduction. Supposedly, reducing the age for obtaining a concealed carry permit to 19 years old is associated with a 32% increase in the firearm suicide rate among teenagers 14 to 18 and a 7% increase in firearm suicide rate among adults aged 19 to 24. According to their analysis, this reduction is also associated with a nearly 30% increase in the non-firearm suicide rate among teenagers in Missouri but a 19% decrease in non-firearm suicide rate among adults age 19 to 24. 

That…just does not make sense. Suicide prevention researchers believe that changing access to one type of means for suicide would also have a corresponding effect on other types of suicide. The overly simplified version of this would be an increase in non-firearm suicide if all firearms just disappeared. But for the third time in this study, the researchers have effects moving in opposing directions for the two age groups. If the change in the law was associated with changes in the suicide rate in the real world, we would expect to see offsetting changes between firearm and non-firearm suicide rates. That is not present in this study, so we have some questions about this mechanism as well. Maybe their data and coding system is flawed?

Missouri instituted permitless carry at the very tail-end of the researchers’ study period (in 2016), so they could maybe – maybe - be excused for not addressing that in their analysis IF one was desperate to believe in this work. They could have just excluded those years from their study period. What cannot be excused is the glaring oversight in the concealed carry laws in the states used to create the synthetic Missouri for these models. 

“For the concealed carry analysis, states were only included in the donor pool if they had an existing concealed carry law and no change in that law during the study period (January 2008 to December 2018).” 

States used to build “synthetic” Missouri included Alaska (2003), Arizona (2010), Arkansas (2013), Kansas (2015), Maine (2015), Mississippi (2013), New Hampshire (2017), West Virginia (2016), and Wyoming (2011). 

The years in parentheses are the year in which each of these states enacted permitless carry. Some of these states set the age threshold for permitless carry at 18, some at 21. The point is that there is a very fundamental mistreatment of the states and laws at the heart of this analysis. Their “synthetic” Missouri also includes may-issue states that allow local official to deny even a qualified person a permit for arbitrary reasons. California, Hawaii, Massachusetts, New York, and New Jersey are all may-issue states that were used in the concealed carry permit model to build a synthetic Missouri.

The authors do not appear to have built a reliable “synthetic” Missouri. The charts on page 7 of their study reveal that synthetic Missouri and “real” Missouri did not have similar rates of firearm suicide or non-firearm suicide. The trend lines for the two Missouris are plotted on the same chart, and one can simply see that the two lines do not match up before the law was changed.  This, of course, indicates that their synthetic control is not really approximate the relevant characteristics of Missouri.

There was no effort to control for mental health treatment or other laws or state programs that may have impacted suicide rates. The researchers offer ham-handed excuses for the discrepancies within their findings and the limitations of their study, and Rowhani-Rahbar tries to make the case that this “research” is still relevant and important because it contributes to the body of knowledge about firearm policies and suicide.

It does not contribute to the body of knowledge – it muddies what is known about suicide and firearms policy by promoting claims drawn from fatally flawed analyses.

This paper, like other examples of poorly done research – see here and here, seems to have prioritized blaming firearms instead of using a sound methodology.

Surprised?

IN THIS ARTICLE
Missouri gun control
TRENDING NOW
U.S. House Passes Reconciliation Bill, Removing Suppressors from the National Firearms Act

News  

Second Amendment  

Thursday, May 22, 2025

U.S. House Passes Reconciliation Bill, Removing Suppressors from the National Firearms Act

Earlier today, the U.S. House of Representatives passed H.R.1 the One Big Beautiful Bill Act, which included Section 2 of the Hearing Protection Act, completely removing suppressors from the National Firearms Act (NFA).

NRA and 2A Allies Announce NFA Lawsuit

Monday, July 7, 2025

NRA and 2A Allies Announce NFA Lawsuit

Following the passage of the “One Big Beautiful Bill”—which eliminates the National Firearms Act of 1934’s (NFA) excise tax on suppressors, short-barreled rifles, short-barreled shotguns, and AOWs—the National Rifle Association issued a joint statement along with the ...

Congress Passes the “One Big Beautiful Bill,” Now Headed to President Trump

News  

Thursday, July 3, 2025

Congress Passes the “One Big Beautiful Bill,” Now Headed to President Trump

Earlier today the U.S. House of Representatives passed the “One Big Beautiful Bill.” This bill contained a provision that would, among other things, eliminate the burdensome $200 excise tax imposed by federal law on suppressors, short-barreled firearms, ...

United Nation’s Attack on Ammunition Formally Begins

News  

Monday, July 7, 2025

United Nation’s Attack on Ammunition Formally Begins

The recently concluded negotiations on the United Nations’ Global Framework on Through-Life Conventional Ammunition Management (Framework) should be of grave concern to anyone who values the constitutional protections afforded by the Second Amendment. 

DOJ Declines to Seek Supreme Court Review of Decision Striking Down Federal Laws Prohibiting FFLs From Selling Handguns to 18-to-20-Year-Olds

Thursday, July 10, 2025

DOJ Declines to Seek Supreme Court Review of Decision Striking Down Federal Laws Prohibiting FFLs From Selling Handguns to 18-to-20-Year-Olds

In Reese v. ATF, the Fifth Circuit held that 18 U.S.C. §§ 922(b)(1) and (c)(1)—which together forbid Federal Firearms Licensees from selling handguns to 18-to-20-year-olds—violate the Second Amendment.

U.K. Embarks on Fresh Knife Amnesty, Reminds Americans Why 1776 was a Good Idea

News  

Monday, July 7, 2025

U.K. Embarks on Fresh Knife Amnesty, Reminds Americans Why 1776 was a Good Idea

It has been a while since we’ve looked at weapon news from across the pond, but with Americans everywhere having just celebrated the Glorious Fourth and our independence from British monarchy, the timing seems particular ...

President Trump Signs the “One Big Beautiful Bill Act” into Law

News  

Friday, July 4, 2025

President Trump Signs the “One Big Beautiful Bill Act” into Law

Earlier today, on the 4th of July, a day on which our Founding Fathers declared their intent for a free nation, the President of the United State of America, Donald Trump, signed the “One Big ...

NRA-ILA July 2025 Litigation Update

Thursday, July 10, 2025

NRA-ILA July 2025 Litigation Update

In the second quarter of 2025, the National Rifle Association filed two cert petitions in the U.S. Supreme Court and five amicus briefs, while continuing to litigate dozens of ongoing lawsuits across the country.

Florida: Second Amendment Sales Tax Holiday Signed by Governor

Monday, July 7, 2025

Florida: Second Amendment Sales Tax Holiday Signed by Governor

Governor Ron DeSantis recently signed the Florida Budget for Fiscal Year 2025–2026, which includes a Second Amendment sales tax holiday from September 8 through December 31, 2025. The NRA is thankful for Governor DeSantis’ strong ...

North Carolina: Update on Gun Bills Moving through the General Assembly

Tuesday, June 24, 2025

North Carolina: Update on Gun Bills Moving through the General Assembly

Recently, House Bill 193 (H193) was reported favorably out of both the Senate Judiciary Committee and the Senate Rules Committee, with amendments.

MORE TRENDING +
LESS TRENDING -

More Like This From Around The NRA

NRA ILA

Established in 1975, the Institute for Legislative Action (ILA) is the "lobbying" arm of the National Rifle Association of America. ILA is responsible for preserving the right of all law-abiding individuals in the legislative, political, and legal arenas, to purchase, possess and use firearms for legitimate purposes as guaranteed by the Second Amendment to the U.S. Constitution.