Contents

1 Copyright

This document is copyright © 2006 January 02 by Howard Gibson. You may copy this document onto bulletin boards, web pages and other computer media, as long as the article is complete, unaltered, and the distribution is not-for-profit. All other rights are reserved.

2 Introduction

I was watching a TV show on ABC (Sunday, 1998 Aug 09) about guns and crime. On this program, anti-gun activists quoted the statistic that a gun in the house is forty three times more likely to be used against a family member or acquaintance than it is to be used against a criminal from outside. On the same program, pro-gun activists claimed that in any given year, 2.5 million people use guns to defend themselves from attack by criminals. According to the World Wide Web Gun Defense clock [5], 15.7% of these defenders would “almost certainly” have been killed, and another 14.2% would “probably” have been killed.

These numbers don’t seem to come from the same reality. Even if one does not hold strong opinions about gun control, this sort of thing is worth looking into because so much public discussion is done by quoting statistics that “prove” one’s opinion.

3 Kleck and Gertz

3.1 Gun Defense Use Survey

The Kleck/Gertz study was a follow-up to previous studies which were showing an increasing number of Defensive Gun Uses, refered to as DGUs. Gary Kleck, a professor of Criminology at the University of Florida, assumed that reported DGUs were low because the respondents (called “Rs” in Kleck’s report) to the survey were reluctant to admit to them. From reference [2]…

Kleck decided to do an anonymous survey. He called approximately 5000 randomly selected phone numbers from the lower forty eight states. A team of surveyors interviewed the Rs without determining their identities. Approximately 220 people reported defending themselves with a gun, against a human, sometime within the last five years. Approximately fifty reported a gun defense within the year. This works out to about 1% of the population, or approximately 2.5 million each year. Many accounts quoting this report state that the rate of DGUs is between 1 million and 2.5 million per year[4].

Since Kleck’s survey was anonymous, there was no way to corroborate stories told them by their Rs. Kleck claimed that his surveyors could detect lying, and that he did eliminate some responses because he did not believe them.

3.2 Analysis of the Survey Statistics

We can do a validity check on Kleck’s numbers by using them to calculate some real-world condition. Let’s look at the Rs who reported a DGU. In any given year, there are 2.5 million of them. 15.7% estimated that they “almost certainly” would have been killed if they had not resisted with a gun. Another 14.2% said “probably” and 16.2% said they “might have” been killed. They had the additional choices of “probably not” and “almost certainly not”. If the surveyors and the Rs are honest and accurate, these numbers will have some similarity to real crime statistics.

Assume that all of the “almost certainlys”, half of the “probablys” and none of the “might haves” were killed, for a total of 15.7+8.1=23.8%, or, approximately 20%. Apparently, something like half of all American households have guns. Let us assume that half of all Americans are armed and ready to defend themselves. This is probably an overestimate. This means that another 2.5 million Americans experienced a criminal act that they could have defended themselves from if they had been armed. According to the statistics above, something like 20% of them ought to have died. That works out to 500,000 homicides. The rate of successful homicides in the U.S. was something like 20,000¹ at the time of Kleck’s survey. Kleck’s lethality estimates are out by well over an order of magnitude.

Note that I rounded down my percentage killed, and almost certainly, I overestimated the percentage of Americans armed and ready to defend themselves.

On the other hand, the rate of crimes committed against unarmed citizens is probably less than that against gun owners. Some people arm for self defense in response to a legitimate threat. These people probably would not arm in the absense of that threat. A lot of unarmed people probably would acquire a gun if they faced significantly increased danger. Kleck excluded in his survey police, security guards and the military, for this reason. He did not exclude clerks in late night convenience stores and gas bars. In Canada, circa 1998, these were the people most likely to be murdered at work.

Kleck’s Rs grossly exaggerated the danger they were in. We don’t know why they did this or to what extent they did it. There is a possibility that the Rs responded accurately about everything except the homicidal intentions of their assailants. There is also a possibility that many of the accounts were wildly exaggerated in their entirety, with the Rs maximizing the danger presented by the assailants, and their criminal intentions, their own innocence, and their own activity and heroism.

Kleck estimated that his figures show 400,000 people a year avoiding death with a DGU. He recognized this as excessive, however, he speculated that if even 10% of these claims were true, then DGUs were having a significant effect on homicide rates. This reasoning is wrong. When the numbers are this far out, you cannot make estimates based on them.

3.3 Marginal Statistics

A big problem with Kleck’s statistics is well understood by statisticians outside the gun community.

If you do a survey of any kind, some of your responses are wrong. In a binary survey (yes/no) some of your responses either are false positives or false negatives. If both choices are frequent, these false answers tend to balance each other out, and they can be ignored. If your answers are overwhelmingly one sided, your errors are overwhelmingly one sided.

In Kleck’s survey, 95% of the respondents indicated that they did not use a firearm in self defense against another human being in the last five years. The vast majority of responses are negative, therefore, the vast majority of incorrect responses are false positives.

The 5% of respondents who claim to have defended themselves is consistent with a 100% accurate survey in which 5% of the respondents defended themselves. It is also consistent with a 95% accurate survey in which no one defended themselves.

3.4 Conclusion

Kleck’s survey is meaningless. Kleck’s concern about people concealing gun defense uses is valid, however, his survey grossly overestimates the number of potentially lethal encounters of his respondents. The remainder of his statistics cannot be considered reliable.

4 Arthur L. Kellermann

4.1 Warning – Odds Ratios

Anti-gun activists like to state that gun owners are 2.7 times as likely to die from guns as non-gun owners. This statistic comes from Arthur L. Kellermann [1].

The number 2.7 is an odds ratio. An Odds ratio indicates correlation within a study. It is not a factor of probability or risk, and gun owners are not 2.7 times as likely to be murdered. Odds ratios are explained in quite a bit of detail further on in this document.

An earlier Kellermann study is frequently quoted to show that gun owners, or their acquaintances were 43 times more likely to be murdered with the gun, than were burglars or other intruders. Critics point out that 37 of these were suicides, and that the word “acquaintance” can cover a wide variety of people, some of them criminals [8]. Also, a successful gun defense does not necessarily involve the shooting of the attacker. I haven’t seen this study, and I don’t know if 43 is a probability or an odd.

4.2 Homicide in the Home

The Kellermann study I read [1] was an investigation into homicides that took place in the victim’s homes. Kellermann compared the victims with a control group of persons of the same sex, race, age and socioeconomic status, living in the same neighborhood.

Kellermann made the following conclusions, quoted from his abstract…

The above conclusions are in declining order of importance. Illicit drugs and a history of violence were stronger factors than gun ownership in the backgrounds of the murder victims.

Kellerman worked out a list of variables to be evaluated with odds ratios, and he produced two sets of data.

• “Crude” odds ratios were worked out from his total population of murder victims and controls, using the Mantel-Haenszel chi-square analysis for matched pairs. The odds ratio for persons with handguns in the home was 1.9, with a 95% confidence interval of 1.4 to 2.7. In other words, there was a 95% certainty that the real odds ratio falls somewhere between 1.4 to 2.7.
• The first analysis was used to identify characteristics that homicide victims tended to have. Kellerman found six. For each of these variables, Kellerman analyzed each case where the other five variables matched between the murder victim and control. In other words, the violent, alcoholic victim was compared to the violent alcoholic control, the home renting, arrestee victim was compared to the home renting, arrestee control, etc. This leaves us with an odds ratio for one variable that is not affected by the other variables.

The “adjusted” odds ratio for gun ownership by murder victims is 2.7. Kellermann listed a 95% confidence interval of 1.6 to 4.4. In other words, there was a 95% probability that the real odds ratio fell somewhere between 1.6 to 4.4.This reflects the small sample Kellerman was working with.

The crude odds ratios for shotgun and rifle owners were 0.5 to 1.1 and 0.5 to 1.3, respectively. Crossing over 1.0 like this means that there is no correlation. Kellerman’s survey indicates that shotguns and rifles in the home are not factors in homicide, at least, not to a 95% certainty.

Here is Table 4 from Kellermann’s report entitled “Variables Included in the Final Conditional Logistic-Regression Model Derived from Data on 316 Matched Pairs of Case Subjects and Controls”

Note how the 95% confidence level varies, indicating the varyingly small size of the corrected samples.

Kellermann admitted to several problems with his statistics. It could not be assumed that the controls were accurately reporting problems with violence, alcohol or other drugs, or accurately reporting their ownership of drugs. The study was restricted to guns used in the home, and does not provide a guide to guns in other locations such as bars. The study was done is areas with small Hispanic populations, so it does not take Hispanics into account. There is the possibility of “reverse causation”, in which the victim buys a gun in repsonse to an accurately perceived threat. Perhaps the sort of people who acquire handguns are more violent than people who do not acquire handguns.

That last point is important. People do not merely decide to buy guns. They decide to participate in a lifestyle that requires guns. This lifestyle decision could be to move to Texas and participate in the drug trade. It could be to take up target shooting or hunting. The decision to acquire a gun for self defense could be from a perception, valid or otherwise, of a high crime rate, or it could be made because someone is hanging out with violent people.

I have an additional issue with the stratified odds. These control other homicide factors such as prior violence and drug abuse. They do not control for their non-presence. Kellerman does not provide a control for competent, responsible gun owners.

4.3 Conclusion

The 2.7 odds ratio indicates a correlation. A correlation does not guarantee a causal relationship. If there is a causal relationship, it may be that ownership of guns causes homicide, or it may be that the possibility of homicide causes gun ownership.

Gun studies are politically relevant. Researchers doing politically relevant work should expect their work to be quoted by people who do not understand the technical terms. It would have helped a bit if Kellerman had explained odds ratios, or at least had explained that the odds ratios were not probabilities.

5 Statistical Analysis

I was having problems with the way Kellerman gets an odds ratio of 2.7 out of the actual gun ownership ratios. We must take a close look at Kellerman’s statistical methods.

5.1 Odds Ratios

An odd, as opposed to a probability, is the number of times something happened, divided by the number of times it didn’t happen.

Eg.[9] on average 51 boys are born in every 100 births, so the odds of any randomly chosen delivery being that of a boy is:

The squiggly equals sign (≈) means approximately equals, as opposed to exactly equals.

The probability of the baby being a boy is

In medical literature, the terms probability and risk are interchangeable, although I see no reason to apply the latter term to the example above. :)

The actual calculations we do are somewhat clumsy. We are interested in the odds and the probability of the murder victims being gun owners, as opposed to the odds and probabilities of the unmurdered controls being gun owners.

These are Kellerman’s figures for guns kept primarily for self defense. Out of 388 murder victims, 125 kept guns for self defense. Out of 388 controls, 86 kept guns for defense

The odds a murder victim owns a gun

The risk a murder victim owns a gun

The odds a control owns a gun

The risk a control owns a gun

Odds Ratio for ownership of handguns…

(1)

Relative Risk for ownership of handguns…

(2)

Kellerman quite properly rounds his odds ratio to 1.7 on Table 3 of his article. My value is too accurate for the data. I am merely demonstrating the mathematics.

In my 1998Nov01 article I claimed that odds ratios and relative risk are equivalent. My source for this was A Dictionary of Medical Terms, Fifth Edition by F. H. C. Marriot, Burnt Mill. This is wrong. They can be approximately the same for small values of odds and risk, but these are much smaller than any numbers we encounter here. Kellerman’s article is based entirely on odds ratios.

The term “risk” is confusing. The risk in this study is that murder victims and controls own handguns. The risk we actually are interested in the that of gun owners being murdered, versus gun non-owners being murdered.

In fact, risk is not a useful mathematical concept for the purposes of matched pairs analyses such as Kellerman’s survey. Exactly half of Kellerman’s subjects have been murdered, making the homicide risk of a surveyee 50%. This is an artifact of how the study was constructed.

The derived values for risk, R_g and R_c have no relationship to the national population outside the study. For that matter, neither do the odds ratios. They are relevant only to show correlations within the study.

According to Jon Deeks[9], odds ratios are controversial. The advantages of odds ratios are discussed in Deeks’ description of them. They are especially useful for evaluating samples like Kellerman’s, where the mathematical concept of risk doesn’t mean anything.

5.2 Chi Squared Distribution

Figure 1: Normal Curve – Green area is the 95% confidence interval

Take a look at the Normal curve in the figure. This comes from a spreadsheet model of a coin flip experiment. The probability of heads is 50%. If you are determined to read up on this stuff, you should have no problems finding a book on statistics. I will describe only the results.

A characteristic of normal curves is that there is approximately a 95% chance that any given result will fall within the interval defined by the average value plus and minus two times the standard deviation. The 95% confidence interval can be fairly easily calculated, and is shown on this graph, shaded in green.

When you are working with a Normal curve, you have the additional options of a 68% confidence level, and of a 99.7% confidence level. Medical researchers have arbitrarily standardized on 95% for medical research results.

The figure shows us that there is a 95% chance that an experiment of 100 coin flips will come up heads somewhere between 40 and 60 times.

6 General Observations

Neither of the above articles have provided statistics complete and accurate enough to prove their point. This may not even be possible. Both surveys treat gun ownership as a factor that directly affects crime rates. This disregards factors leading to gun ownership. A gun acquired for late night convenience store robbery, directly affects the crime rate. A gun acquired by a farmer for shooting varmints, probably has no effect on crime.

The press almost always addresses the gun control issue by describing horrific incidents. The ABC documentary I saw described how someone ran out with his gun to try to stop a car thief, who wrestled the gun away and shot him. Gunners usually respond with the story about the woman who survived the shooting at Luby’s in Texas and who claims that her parents would have survived too if she hadn’t left her gun in her pickup truck.

These anecdotal accounts are irrelevant. The truth about guns lies in statistical evidence that compares fatality rates from accidents, impulsive use and easy availability to criminals, with fatalities prevented by self defense. These studies must take into account the sort of people exposed to criminal attack, since a high proportion of these are themselves criminals. They must also take into account the attitude of the gunner. Most gun problems occur because people either are criminals, or they are bad tempered, sloppy and irresponsible. The case for total gun control (as opposed to selective gun control) is weaker when it is applied to responsible gun owners.

A more general point is the validity of claims backed up by statistical “factoids”. I have come to the conclusion that the quoting of technical specifications and other such numerical data are nothing more than rhetoric. Most points of view can be reinforced by selecting and quoting numbers that may well be accurate. There is no reason to question the honesty of Kleck’s and Kellermann’s surveys. The problem is with the relevance of these numbers, and with the fact that the numerical data provides an incomplete picture of the events.

References