DNA Straw Poll
Suppose you are on a jury in a murder trial, and the case depends entirely on DNA evidence. A sample of the perp’s DNA was found at the scene, but was severely degraded and could only be matched at only 5 1/2 loci rather than 13. Expert witnesses advise you that the odds of someone who isn’t the perp or a blood relative matching at 5 1/2 loci are 1.1 million to one, meaning that there are about 9 people in your state or area who could have done it. You are not told how the police came across this guy or identified him as a potential suspect, and are forbidden to speculate on it.
Based on this evidence alone, what are the odds that this guy is guilty?
- 1.1 million to 1, in favor of guilt.
- 1.1 million to 1, in favor of innocence.
- 1.1 million to 9, in favor of guilt.
- 1.1 million to 9, in favor of innocence.
- 1.1 million to 8, in favor of guilt.
- 1.1 million to 8, in favor of innocence.
- 9 to 1, in favor of guilt.
- 9 to 1, in favor of innocence.
- 8 to 1, in favor of guilt.
- 8 to 1, in favor of innocence.
- 3 to 1, in favor of guilt.
- 3 to 1, in favor of innocence.
- Other (specify)
BONUS ROUND: Having given me your best shot at the answer, which of the above 13 do you think your fellow jurors are most likely to believe? If it’s a different answer from yours, why?
August 21st, 2008 at 10:47 pm
8 to 1, in favor of innocence, assuming that he was picked because he matched an entry in a database of randomly selected state/area residents, and there is no physical evidence linking him to the crime.
I would be forced to assume those things, as a juror, because I have to draw inferences in favor of the accused where the prosecution isn’t putting on a case. I can’t convict a man based on rank speculation.
Since an 11% chance of guilt is nowhere near “beyond a reasonable doubt,” I would have to vote not guilty. In fact, at that level, a judge should not allow the case to even get in front of a jury.
August 21st, 2008 at 10:58 pm
Oops. I made a stupid error. It would be 9 to 1 odds, not 8 to 1.
The fact that this man has a rare DNA sequence (1 in 1.1M) does NOT make it any less likely that anyone else in the state would have the same rare DNA sequence. So we have to assume that there would be about NINE or more other people in the state with similar DNA, on average, not eight.
Another thing I overlooked: if the perp is a young/middle aged male, that would make him more likely to be guilty, because young/middle aged males commit most murders. So if only 1 in 4 state residents are young/middle-aged males, then the odds are closer to 2-to-1 in favor of innocence, or 3-to-1.
That’s still obviously not enough to get beyond a reasonable doubt.
August 22nd, 2008 at 12:35 am
None of the above. For all I know all 4,500 people who match that profile (if I believe those calculations in the first place) live in that neighborhod and had the motive, means and opportunity to commit the crime. Saying one in 1.1 million is the prosecutor’s fallacy. Saying that he is one out nine possibly guilty person is the defense fallacy.
August 22nd, 2008 at 12:52 am
13: I don’t think you can give a statistic of guilt or innocence.
August 22nd, 2008 at 8:42 am
What’s the answer, Jeff?
August 22nd, 2008 at 8:51 am
Correction to my #3,
*4,500* should be *5,455″
August 22nd, 2008 at 9:57 am
Daryl: interesting thoughts about sex, age, etc., but we’re only looking at DNA evidence, not other factors that may further adjust the likelihood of guilt vs. innocence. At this stage, assume a dead 98 year old woman, or a baby who had yet to be born had as good a chance of being the perp as anybody else. [Or, if you prefer, assume that the estimate of 9 has already been pared down to adjust for these implausble scenarios.]
Patterico, why not? I’m not asking for the final odds of guilt vs. innocence, which obviously would require other evidence to be considered as well. I’m asking what the odds of guilt vs. innocence are, based on DNA evidence alone. It’s a simple question based on a simple computation.
NK, I’m not sure you understand the question. It’s not a question of whether you, NK, personally believe that the 1 in 1.1 million figure is valid, or whether you believe the pool of potential perps has been limited to 9. Assume that you, as a juror, have been presented with whatever evidence it takes to be convinced that both numbers were in fact correct. Given that, which of the 13 would you pick?
I’ve responded privately to Pigilito with the “correct” answer. However, the point of this exercise is not to find out what the truly correct answer is – I know it, and so does Patterico – but rather, to gauge the likelihood that a juror would get it right, or at least get something close enough that it doesn’t matter all that much.
August 22nd, 2008 at 10:20 am
There is a problem with assuming that only people in the state/area could have done it, because someone could have come in from a distance (in the Puckett case, for example, he was visiting SF from SoCal).
Suppose SFPD found an SF resident with matching DNA (probability-wise, there’s probably at least 1 man in SF with those 5.5 loci). If you assume an area resident must have done it, then you rule out Puckett (who we now know is most likely the true culprit). So you may end up convicting an innocent person.
There is also a problem with assuming that, as a matter of randomness, every single person in the state with that DNA would be equally likely to do it/have an equal opportunity to do it. If you know nothing else about them, I suppose you have to assume that.
We also have to assume that the police have checked out the suspect’s relatives, because his/her relatives are more likely to share the same 1-in-1.1M DNA profile. It could be that his brother is the true culprit.
There are too many assumptions the jury is being asked to make. The odds are impossible to calculate unless you fill in the gaps–and as a juror, you would have to fill them in in a way that is fair to the defendant, as a matter of due process.
The truth is, if I show up in court and the police say “X is guilty,” and they’ve taken it to trial, there’s an 80-90 percent chance, at least, that X is in fact guilty. But that’s a prohibited inference for jurors to use, because it would destroy due process of law.
August 22nd, 2008 at 10:58 am
Daryl, Daryl, Daryl…
I’m not asking whether the numbers are correct. They may or may not be, but that’s another discussion entirely. The question here is simple: if they are correct, which of the above odds best represents the odds based on DNA alone that the defendant is guilty? Are they 8 to 1 in favor of innocence (your original guess), 9 to 1 in favor of innocence (the starting point of your second guess), or something else?
August 22nd, 2008 at 12:52 pm
Fair enough. Other. It seems to me like a lineup where the witness says, “It could be him. It kind of looks like him”. “Misidentification” has implications I want to avoid. Not sufficient identication? Not a unique enough identification?
And the probability that a genetic profile will match either *another particular person* or *any person* is not a probability of guilt.
August 22nd, 2008 at 1:09 pm
13: Definite reasonable doubt.
August 22nd, 2008 at 3:00 pm
Thanks for the email explanation. For what it’s worth (I hate statistics the way way my sister in law hates conservatives – a lot), I agree.
August 22nd, 2008 at 9:26 pm
Can we assume that there is no blood relative with no alibi within a reasonable distance?
I expect that most jurors would see it as 1.1 million:1 in favor of guilt. That’s the interesting tidbit, and unless there was evidence. Expert witnesses sound like impartial experts, and that makes you think that whatever they emphasize is the most relevant part. Things coming first are the emphasis. The phrasing you used, at least assuming that there are a mere ~9.9 million people in the state or area, makes it rather clear how likely the perp matches the DNA sample. That puts the odds of the DNA being the accused at 1 to 9, falling heavily toward the guilt of the accused — after all, this is in coincidence with the accused’s arrest, opportunity, and ability, as well as the odds that the other 9 individuals in the area could have their own alibi. We don’t know what these numbers due to the odds (they probably make things several thousand times better than 1 to 9, but not at 1 to 1.1 mill), but we can put a gut-level check on them.
Technically, it’s also correct, as the odds of the match being accurate remain the same regardless of how or why the search takes place. If the assumption for murderer maximum commute distances are correct, there are only (a statistical sample centering on, with a fairly small normal distribution) 9 non-blood relative individuals in the area who match the DNA sample.
But we’re not given enough evidence to know if the DNA match is enough to demonstrate guilt. If there were, for example, 1.1 million people in a database somewhere, there are good (but not certain) odds that one of them would have a DNA match with this sample even with a perfect alibi and innocence (as well as a smaller chance of multiple or zero individuals matching). If the man was founding trawling through such a database, the odds of a match is rather high, but we have no idea what the relevance of the match is toward the odds of the found man being guilty. While the odds of the DNA sample being the matched man’s remains the same one to nine, most of the old modifiers don’t attach. There’s enough doubt to stall me.
So my vote would be 13: we don’t know; there is not enough information. If I were on the jury, I’d ask whether the man when the man’s DNA sample dated from or if the man was on the DNA list before his arrest, but if there was nothing else stated I’d say the prosecution left (or produced) several shadows of doubt.
August 22nd, 2008 at 10:56 pm
[...] P.P.S. Xrlq is taking a poll about a statistical question. Feel free to weigh in here. [...]
August 22nd, 2008 at 11:39 pm
10: 8 to 1, in favor of innocence. Knowing nothing else, the chance of him being guilty are about 1 in 9, which means 8-1 against.
August 23rd, 2008 at 12:11 am
I’ve not yet looked at any of the other answers.
Assuming that you have described the entire universe of potential perpetrators, I would say that the answer is no. 10: 8 to 1, in favor of innocence.
I HOPE I could convince any my fellow jurors who hadn’t come to the same conclusion on his or her own to come around to the side of the Force.
August 23rd, 2008 at 12:24 am
The crux of this is the instruction: *forbidden to speculate on it.* Complying with jury instructions would, for me, necessitate 9 to 1 in favor of innocence, as the underlying facts of what I was told was there were 9 possible matches in this area.
August 23rd, 2008 at 12:25 am
Oops. 8 to 1 in favor of innocence.
August 23rd, 2008 at 1:59 am
You can only say 8-to-1 if you know he was plucked at random from a giant DB.
If he was chosen based on his proximity to the crime scene, for example, then we would expect about 9 other State/Area residents to have his DNA. Of course, if he was chosen for his proximity to the crime scene, the odds of his guilt are not 1-to-9, they are much higher!
As a juror, I don’t even know that much. So I would have to assume 8-to-1 odds of innocence, because I’m not permitted to draw an inference against the defendant that is based purely on rank speculation.
Further, that is assuming that the police have checked out the suspect’s family members, and that the police have NOT checked out the other people in the State/Area that showed up as hits in the DB (or THEIR family members).
For example, if EVERY State/Area Resident is in the DNA database, and the police check out everyone, and the only person with a DNA match and no alibi is the suspect, then you can’t say there is a 8-to-1 chance of innocence. The chance of guilt goes up. But again, as a juror, I could not presume that. It would be unfair to the defendant.
Further, there’s a chance that the suspect’s DNA was planted at the scene, by the culprit and/or the police or someone else. THAT is a probability that I have no way of calculating, especially if I don’t know any other facts of the case. There’s also a chance that the DNA got there by innocent means.
(This is why, when asking this question, statisticians don’t ask the chance he’s innocent/guilty, but ask what is the chance that the police have accurately identified the DNA as belonging to him)
The juror honestly just can’t know the odds of guilt based on what you’ve told me. At most, he can make a series of reasonable inferences most favorable to the defendant and come to the conclusion that there is reasonable doubt. I say reasonable inferences because I don’t think jurors should assume police corruption without any evidence of it. I would not infer police corruption, even though that would be most “fair” to the defendant. I think it’s more speculativey to infer corruption than not.
August 23rd, 2008 at 2:15 am
There is another thing I would like to address:
If the suspect was chosen from a database, I think 8-to-1 understates the chance of innocence. Even if, on average, you would expect 9 State/Area residents to have a rare trait like this, that doesn’t mean that you can expect that a rare trait that shows up at a crime scene is likely to be found in only 9 of the State/Area Residents.
Allow me to explain, with my infamous “Supermarket” hypothetical:
Assume: 1 Supermarket (M); 10 Checkout Lanes (C); 100 Shoppers (S)
Assume all shoppers are in one checkout lane or another.
QUESTION ONE: For each checkout lane, what is the average number of shoppers?
ANSWER ONE: ten.
QUESTION TWO: For each shopper, on average how many shoppers are there total in the same lane?
ANSWER TWO: ten or more.
If the 100 shoppers are spread perfectly evenly across the lanes, the answer to Q1 and Q2 is 10.
BUT if all 100 shoppers are in the same shopping lane, then the answer to Q2 is 100, even though the answer to Q1 is still 10.
If they are distributed randomly, you might end up with something like this:
8 8 9 12 8 12 13 11 7 12
Then the answer is:
Q1: 10
Q2: 10.44
The answer to Q1 is: (8 + 8 + 9 + 12 + 8 + 12 + 13 + 11 + 7 + 12)/10
The answer to Q2 is: (8^2 + 8^2 + 9^2 + 12^2 + 8^2 + 13^2 + 11^2 + 7^2 + 12^2)/100
The average shopper is in a lane with 9.44 other shoppers. This is because crowded lanes end up counting more.
Suppose some shopper has a heart attack. This shopper is almost twice as likely to be in the 13-person lane as in the 7-person lane, assuming random distribution of heart attacks.
If it does hit the 13-person lane, you can’t say that each person in the lane has a 10% chance of having a heart attack!
* * *
Now, assume random distribution of sex offenses: the trait shared by 13 people is almost twice as likely to be found at the scene of a sex crime as a trait shared by 7 people, even if the FBI’s statistics predict an equal number of people, ON AVERAGE, in a given State/Area of similar size to the State/Area at issue, will share those same traits.
It’s not enough to know that ON AVERAGE some trait is shared by 1-in-1.1M people. Maybe it’s shared by a larger fraction of people in the State/Area–in fact, statistically, it probably is!
August 23rd, 2008 at 3:17 am
Simple enough.
Given the constrictions in the thought experiment we know that there are nine people who could’ve done it. One of them is on trial.
9 to 1 in favor of innocence.
The odds that your average collection of 12 idiots deliberately selected by the jury selection process for lack of IQ will get it?
About three trillion to one.
August 23rd, 2008 at 4:15 am
If I know nothing else then I vote to acquit based on there being other people who could reasonably have done the deed (8 others in the state/area) for 8-1 in favor of innocents, which is very much a reasonable doubt.
However, if there is a shred of evidence placing him within, say, 5 blocks of the crime, the odds change markedly.
Note that what this says is that you cannot prove guilt solely by DNA evidence. Nor can you prove it solely by a fingerprint, even though fingerprints are thought to be unique.
The other jury members will probably believe whoever spoke last, but I’ll go with 1.1 million to one. Unless one is a reporter in which case it will be 1.1 million to 9 in favor of whatever they already believe.
August 23rd, 2008 at 5:31 am
Statistical answer:
Nine people in my state or area who could have done it; if we assume that others outside my state or area could not have done it, the right answer is 8-1 in favor of innocence.
This is not a hard problem.
There are nine people who fit the DNA profile and could have done it. One did. That’s 8-1, folks.
As far as we know, this guy has been picked at random. We are forbidden from taking into account why he got picked.
What my fellow jurors believe going into the jury room will depend on what the arguments from the lawyers are. What my fellow jurors will believe when we come back with the verdict forms is, “That JRM is a pain in the ass, but he’s right.”
If-this-happened-in-real-life-answer:
Presumably, all of the other evidence was suppressed, and the guy is guilty. However, certainty is far from clear; a prosecutor who takes this case forward on this evidence is an idiot, and a court which doesn’t throw this case out with only this evidence is also…. not good at math. This does not give me warm feelings toward anyone involved’s ability to evaluate evidence.
The jury should still acquit, obviously, but several people have some splainin’ to do.
–JRM
August 23rd, 2008 at 11:27 am
Hopefully, the jurors will believe that there is one chance in nine that he is guilty and eight chances out of nine that he is innocent. If I were defending him, I would put eight white marbles and one black marble in a bag, one to be taken out without looking, to illustrate the defense argument.
August 23rd, 2008 at 12:32 pm
Suppose the perp left his social security number, but the last digit was obscured. (Pretend for a minute that SSNs are issued sequentially).
Isn’t this the same problem?
If the cops rounded up all 10 people that fit the criteria and did eenie meenie minie moe to charge someone, then it is 9 to 1 on innocence.
August 23rd, 2008 at 1:48 pm
My answer should not be read as saying that the identity of a perpetrator can never be proved with DNA evidence.
It’s just that, while I sometimes participate in these games where we conflate “guilt” with “the DNA at the crime scene came from the defendant,” they are not identical concepts. So sometimes I register a protest. Technically, you can’t calculate “odds” of guilt. You can only calculate odds that a particular profile has a particular random match probability.
Generally there will be some evidence of a crime having committed, even if there is no other evidence of identity. Thus, the case as a whole will consist of more than the DNA evidence, even if the DNA evidence is the only evidence of identity.
I would have no trouble convicting if identity were shown by a full DNA match.
Obviously, people should not lose sight of the fact that the odds here are very rare, in that they leave open the reasonable possibility that other people might have committed the crime. That is usually not the case.
August 23rd, 2008 at 3:15 pm
As I understand it, time also degrades DNA. So not only are we looking at one suspect out of nine possibles, according to the problem, we may also be looking at the wrong time period.
August 23rd, 2008 at 10:14 pm
Put down the bong everyone, you are way too paranoid and adding in all kinds of weird things.
The science alone says #2 – It’s actually stated in the question.
BONUS: #13 1:1
People who sit on juries are either too stupid to get out of doing or want to see criminals get justice.
August 23rd, 2008 at 11:48 pm
Also, I assume everyone here will say that a jury will decide the issue more stupidly than they — even with the benefit of expert testimony. Isn’t that always how people feel: they are smarter than others?
August 24th, 2008 at 12:27 pm
[Without reading prior comments, since this is a straw poll.]
> You are not told how the police came across this guy or identified him as a potential suspect…
This clause makes it impossible to figure the odds on the guy.
If the cops discovered him and developed the case against him and only then figured out the DNA connection, then the odds are 1.1 million to one in favor of guilt, assuming there’s no case against any of the other +/- 9 local people, and also assuming that the suspect doesn’t have a convincing alibi. But that’s a lot of assumptions to pile atop a precise numerical estimate like “1.1 million to one.”
If the cops identified this guy through the DNA connection (e.g. via a database search), the picture’s different. His name was going to be one of the +/- 10 local names to be tagged by the computer. So, assuming (that word, again) that only a local could have committed the crime, the odds would be 1 in 10.
Bonus question — This story has more holes in it than the Albert Hall. Why was nothing presented at trial other than DNA evidence? Why mustn’t we speculate? Why did the prosecution present the impressive-sounding “1.1 million to one” expert witness testimony, without adding appropriate qualifiers? Sounds like a story that could have a lot of hidden sloppy police work to it. I think many juries would balk at a conviction on the basis of the facts presented in the post.
August 24th, 2008 at 12:40 pm
Having now read through the comments, as a juror, I’d go with JRM’s #23 (odds of 1 in ~9 that the DNA at the crime scene came from the suspect). Not that that is necessarily the correct answer–rather, “1 in 9″ is all I could conclude as a juror, under the ground rules of the thought experiment.
August 24th, 2008 at 7:10 pm
I did some new research on the crowding problem, and found that no matter what your numbers are, you have to add +1 to the number of people. I will post/email my source code on request.
If there are 11,000,000 people, with 1,100,000 unique, randomly-distributed DNA patterns, you might think on average there would be 10 people per unique DNA pattern. You would be right.
BUT if you count per PERSON, the average number of PEOPLE who share their DNA pattern, the average goes up to 11.
I don’t know why it’s always a boost of exactly +1, but I get the same answer whether I use very big or very small numbers.
* * *
Further, in the above example, approximately 8.3% of all people will have DNA patterns shared by more than 15 people. So in 8.3% of your cases, the odds are actually more like 15-to-1 in favor the DNA not being a true match. That’s a significant fraction of cases in which using the FBI’s base figures results in significantly underestimating the likelihood it’s not a true match.
August 24th, 2008 at 8:21 pm
The +1 factor is the perp himself. He wasn’t chosen based on matching or not-matching any particular genetic makeup. His test was the “no wrong answers” round, akin to betting on heads vs. tails, but waiting until the first coin has landed before you decide whether to go with heads or tails for the long haul. It’s a statistical freebie.
Otherwise, a full DNA match, with an RMP in the quadrillions, predicts that no one will have any particular DNA make-up. That’s usually an accurate prediction if you start with a hypothetical DNA sequence no living human is known to share, but it’s always off by one if you compare it to the real DNA sample. The odds of you having your own DNA may be a quadrillion to one, but I’ll bet you have it anyway.
August 24th, 2008 at 10:26 pm
The odds of you having your own DNA may be a quadrillion to one, but I’ll bet you have it anyway.
Ok, that’s the quote of the day.
August 25th, 2008 at 11:03 am
The +1 factor is the perp himself. He wasn’t chosen based on matching or not-matching any particular genetic makeup.
No. The hypothetical doesn’t say how he was chosen. He may have been caught at the scene. He may have been chosen because the cops took a DNA sample from the scene, put him into a database, and found a hit.
Your example is based on the case of Mr. Puckett; You cannot say that Mr. Puckett “wasn’t chosen based on matching or not-matching any particular genetic makeup”!
* * *
What you’re talking about–that the DNA at the scene must belong to somebody–is much less than what I describe above.
The +1 effect I describe does occur where you would expect on average 0 matches for each DNA pattern. (Such as if you have 6B people and 6 trillion possible DNA patterns)
But the effect I describe also holds true even where you expect about 9 matches on average for each DNA pattern (such as if you have 10M people, and a 1-in-1.1M chance of a match for any given random sequence). The effect I describe is not based on the fact that some patterns will be “empty”–in fact, it holds true even if every single pattern has at least 1 person.
August 25th, 2008 at 11:21 am
Right, but in both of your examples, a DNA profile was found at the scene, before you had looked at a single record in the database. That’s why he’s a gimme. Theoretically, a record from the database could have the same effect, if he was pulled early enough in the process, such that you have 1 known member of that DNA race, and still have essentially 18 million unknowns to go. Where it differs is if you had to dig through 1.1 million records to find that first sample.
Puckett (the record selected from the database because it generated a match) was chosen based on matching or not matching a particular make-up. The perp (the DNA found at the scene of the crime) was not. He’s the freebie.
I don’t think the pattern you describe holds true for all patterns that could match anybody. It does hold for all patterns known to match one person, who can then be matched up against a database full of unknowns. In other words, in a database of 10 million unknowns, and with an RMP of 1.1 million to 1, you should expect to find aproximately 9 matches, not 9 + 1. The +1 guy is the one you already knew about independently of the database (or didn’t).
How many Californians match a particular profile not konwn to match anybody? Based on todays’ population size, probably about 20. How many match yours? 21.
August 25th, 2008 at 2:22 pm
All this statistical stuff depends on what you’re looking for. If you’re looking at the efficacy of chemotherapy for Stage IV stomach cancer with a goal of five years post-debulking survival, a one in nine chance that it might work is good enough, for example.
August 25th, 2008 at 4:37 pm
OK, here’s what we’ve been given.
1) A murder case depends *entirely* on DNA evidence.
2) DNA from the perpetrator was found at the scene.
3) The probability of a match between the DNA found at the scene and any one person at random is 1.1 million to one.
4) Given the population of the state or area, we can expect nine people to match the DNA found at the scene, at random.
5) We do not know how the police came across the defendant, and we may not speculate about it.
If the DNA was definitely left by the perpetrator, and we have good reason to believe the perpetrator has not left the state or area, we know that one person in the state or area must have committed the murder.
We may assume that anyone who is not a match for the DNA left at the scene is actually innocent.
We are left with the genetic equivalent of an Agatha Christie mystery, where there are nine possible suspects, one of whom is definitely guilty.
In the absence of any other information, each suspect has one chance in nine of being the guilty party. If you put their pictures on a dartboard and throw a dart, you have one chance of hitting the perpetrator, and eight chances of hitting an innocent person.
So my vote is 8:1, in favor of innocence.
BONUS ROUND: Having given me your best shot at the answer, which of the above 13 do you think your fellow jurors are most likely to believe? If it’s a different answer from yours, why?
Having served on two juries, I believe my fellow jurors would believe it’s too hard to decide. Between 7 and 11 jurors would go with whoever sounds the most authoritative, and 1-4 with whatever supports their biases.
August 28th, 2008 at 1:48 am
You would expect usually 1 perp + 9 random matches in the state.
The person whose DNA was found at the scene is a “freebie” in the sense that he does not count against the other 9.
That’s why the odds of guilt are 1:9.
If you pick DNA at random, you would expect 9 matches total. If you pick a PERSON at random, you would expect 10 people with the same DNA, including the person you originally picked.
August 28th, 2008 at 9:50 am
I agree. I like to think of it this way: if you’re betting on how many heads vs. tails I’ll get in the next 100 throws, you should bet on a tie. But if you get to see the first throw before placing the bet, you should bet on whatever came up first to win by one.
September 2nd, 2008 at 8:43 am
[...] Xrlq: The odds of you having your own DNA may be a quadrillion to one, but I