The case
In 2012, the Obama Justice Department charged the sheriff of Alamance County NC with discriminating against Latino drivers at traffic stops.
Statistical experts were hired to assess the strength of evidence for three kinds of targeting:
- The sheriff’s office placed checkpoints at sites more likely to encounter Latinos;
- Checkpoint screenings led to targeted arrests of Latinos;
- Latinos were more likely to be stopped for minor violations in non-checkpoint situations.
Data sources include Alamance County Sheriff’s Office, the U.S. Census Bureau, and the North Carolina state government.
Your task: phase one
Your client has sent you a horrific pile of files containing the traffic stop records of the sheriff’s office from 2009 - 2012:
- Mop up the mess;
- Write an expert witness report;
- Reach whatever conclusion seems correct to you.
Judge, jury, and attorneys are the audience. Not statisticians.
Your task: phase two
After you submit the report, we will review it, and then your team will “take the stand” for 25 minutes:
- An “attorney” sympathetic to your conclusion will examine you;
- An “attorney” hostile to your conclusion will cross-examine you.
The “attorneys” will come from Duke’s Mock Trial team.
Timeline (Section 1)
This case is not about technique, so no content lectures. Furthermore, we’re asking you to write an airtight report on an accelerated timeline. So:
- Friday April 3: lab as usual;
- Tuesday April 7: in-class work period;
- Thursday April 9: Q&A with Dr. David Banks, who has been an expert witness;
- Friday April 10: lab as usual;
- Monday April 13: final report due;
- Tuesday April 14: intro Case 6, in-class work period;
- Thursday April 16: Presentation (3 teams);
- Friday April 17: Presentation (3 teams);
- Tuesday April 21: Presentation (3 teams).
Timeline (Section 2)
This case is not about technique, so no content lectures. Furthermore, we’re asking you to write an airtight report on an accelerated timeline. So:
- Friday April 3: lab as usual;
- Monday April 6: in-class work period;
- Wednesday April 8: Q&A with Dr. David Banks, who has been an expert witness;
- Friday April 10: lab as usual;
- Monday April 13: intro Case 6, in-class work period and final report due;
- Wednesday April 15: Presentation (3 teams);
- Friday April 17: Presentation (3 teams);
- Monday April 20: Presentation (3 teams).
Words of caution
- This is not about technique. Your audience (judge, jury, and lawyers) know virtually nothing about statistics. Stick to simple things you can actually explain!
- You may not have written every part of your team’s report, but your name is on it. Be prepared to explain and defend everything in it. If the question is to you, a teammate can’t bail you out;
- When you’re on the stand, the only resource you can consult is a paper copy of your report;
- You are a statistical expert witness. Don’t opine about things beyond your expertise;
- You wouldn’t arrive late to court, would you?
- You wouldn’t wear sweatpants to court, would you?
The Daniela Poggiali case
Background
- Daniela Poggiali (DP) was a nurse at ‘Umberto I’ hospital in Lugo Italy from 9 April 2012 to 8 April 2014;
- On 12 March 2014, 95-year-old Massimo Montanari (MM) dies;
- On 8 April 2014, 78-year-old patient Rosa Calderoni (RC) dies;
- DP is accused of killing RC by lethal does of KCl;
- DP sentenced to life on 11 March 2016, but then acquitted, re-tried, acquitted, and re-tried again;
- DP is separately tried on 15 December 2020 for killing MM;
- Several more rounds of acquittal and re-trial follow;
- She is acquitted once and for all on 24 October 2021.
Suffice it to say…there was some disagreement here.
The hospital ward where DP worked
Specialized in seriously ill patients. Most days have at least one death.
From the prosecution’s experts:
- If a nurse is working in sector A while a death occurs in sector B, the death is desginated as being “in the same zone.”
- A death in C while she works in A is called “in the opposite zone.”
The prosecution’s case
All circumstantial. No one actually saw DP murder anyone.
However:
- She was known to complain about the two patients;
- There was an incident of KCl going missing and turning up in a weird place;
- Using data on hospital shift assignments and patient mortality, the prosecution’s expert witnesses argued that a disproportionate number of patients died while DP was on duty, and this could not be mere coincidence.
Several triers of fact were clearly convinced.
The prosecution’s expert testimony
Are you convinced by this evidence?
The defense’s expert testimony
Just poke a million holes:
- The prosecution’s conclusions are sensitive to the choice of grouping of hospital sections;
- Observational data: association does not imply causation;
- Prosecutor’s fallacy and base rate fallacy;
- Misinterpretation of the \(p\)-value as a posterior probability of guilt/innocence given data;
- Most of the deaths on DP’s watch are recorded in broad daylight when multiple other nurses are working.
Bayes for dayes
Can the available evidence distinguish:
\[
\begin{aligned}
H_d:\, & \text{misconduct did not take place}\\
H_p:\,& \text{misconduct did take place}.
\end{aligned}
\]
The prosecution’s experts seem to inappropriately interpret a \(p\)-value as \(P(H_d\mid E)\). Not so. Let’s do it right:
\[
\frac{P(H_p\mid E)}{P(H_d\mid E)}=\frac{P(H_p)}{P(H_d)}\times \frac{P(E\mid H_p)}{P(E\mid H_d)}.
\]
Don’t commit the base rate fallacy!
What are the prior probabilities? Really f%$#ing low:
- Grant the prosecution’s premise that \(P(E\mid H_p)\) is 0.99;
- There were 280,000 nurses in Italy in 2014, and you could count on one hand the number of cases where a nurse murdered patients and there was concrete evidence of guilt. The defense conservatively estimates that \(P(H_p)=0.000001\)
Logistic regression
Each observation is a patient, and the response variable is whether or not they died during their hospital stay. Predictors are:
- Sector (categorical): which sector was the patient in?
- Times (numerical): how many times during their stay was DP on duty in that sector;
- Age (numerical): the patient’s age;
- Present (dummy): 1 if a nurse works in the same “zone” when death is recorded, else 0.
Takeaways
- Common mistakes repeated over and over: correlation \(\neq\) causation, base rate fallacy, misinterpreting \(p\)-values;
- An opposing expert need not prove their side’s case. Just poke a million holes in the other expert’s testimony;
- For a non-technical audience like judge and jury, we’re basically back to 199 tools: plots, tables, and simple regressions.