$$ \newcommand{\initvar}{v_0^2} \newcommand{\cd}{\overset{d}{\to}} \newcommand{\var}{\text{var}} \newcommand{\cov}{\text{cov}} \newcommand{\argmax}[1]{\underset{#1}{\arg\max}} \newcommand{\argmin}[1]{\underset{#1}{\arg\min}} \newcommand{\iid}{\overset{\textrm{iid}}{\sim}} \newcommand{\indep}{\overset{\textrm{indep}}{\sim}} \newcommand{\tr}{{\scriptscriptstyle\mathsf{T}}} \newcommand{\com}{,\,} \newcommand{\given}{\,|\,} \newcommand{\acal}{\mathcal{A}} \newcommand{\bcal}{\mathcal{B}} \newcommand{\ccal}{\mathcal{C}} \newcommand{\dcal}{\mathcal{D}} \newcommand{\ecal}{\mathcal{E}} \newcommand{\fcal}{\mathcal{F}} \newcommand{\gcal}{\mathcal{G}} \newcommand{\hcal}{\mathcal{H}} \newcommand{\ical}{\mathcal{I}} \newcommand{\jcal}{\mathcal{J}} \newcommand{\kcal}{\mathcal{K}} \newcommand{\lcal}{\mathcal{L}} \newcommand{\mcal}{\mathcal{M}} \newcommand{\ncal}{\mathcal{N}} \newcommand{\ocal}{\mathcal{O}} \newcommand{\pcal}{\mathcal{P}} \newcommand{\qcal}{\mathcal{Q}} \newcommand{\rcal}{\mathcal{R}} \newcommand{\scal}{\mathcal{S}} \newcommand{\tcal}{\mathcal{T}} \newcommand{\ucal}{\mathcal{U}} \newcommand{\vcal}{\mathcal{V}} \newcommand{\wcal}{\mathcal{W}} \newcommand{\xcal}{\mathcal{X}} \newcommand{\ycal}{\mathcal{Y}} \newcommand{\zcal}{\mathcal{Z}} \newcommand{\EE}{\mathbb{E}} \newcommand{\NN}{\mathbb{N}} \newcommand{\PP}{\mathbb{P}} \newcommand{\QQ}{\mathbb{Q}} \newcommand{\RR}{\mathbb{R}} \newcommand{\ZZ}{\mathbb{Z}} \newcommand{\BA}{\mathbf{A}} \newcommand{\BB}{\mathbf{B}} \newcommand{\BC}{\mathbf{C}} \newcommand{\BD}{\mathbf{D}} \newcommand{\BE}{\mathbf{E}} \newcommand{\BF}{\mathbf{F}} \newcommand{\BG}{\mathbf{G}} \newcommand{\BH}{\mathbf{H}} \newcommand{\BI}{\mathbf{I}} \newcommand{\BJ}{\mathbf{J}} \newcommand{\BK}{\mathbf{K}} \newcommand{\BL}{\mathbf{L}} \newcommand{\BM}{\mathbf{M}} \newcommand{\BN}{\mathbf{N}} \newcommand{\BO}{\mathbf{O}} \newcommand{\BP}{\mathbf{P}} \newcommand{\BQ}{\mathbf{Q}} \newcommand{\BR}{\mathbf{R}} \newcommand{\BS}{\mathbf{S}} \newcommand{\BT}{\mathbf{T}} \newcommand{\BU}{\mathbf{U}} \newcommand{\BV}{\mathbf{V}} \newcommand{\BW}{\mathbf{W}} \newcommand{\BX}{\mathbf{X}} \newcommand{\BY}{\mathbf{Y}} \newcommand{\BZ}{\mathbf{Z}} \newcommand{\Ba}{\mathbf{a}} \newcommand{\Bb}{\mathbf{b}} \newcommand{\Bc}{\mathbf{c}} \newcommand{\Bd}{\mathbf{d}} \newcommand{\Be}{\mathbf{e}} \newcommand{\Bf}{\mathbf{f}} \newcommand{\Bg}{\mathbf{g}} \newcommand{\Bh}{\mathbf{h}} \newcommand{\Bi}{\mathbf{i}} \newcommand{\Bj}{\mathbf{j}} \newcommand{\Bk}{\mathbf{k}} \newcommand{\Bl}{\mathbf{l}} \newcommand{\Bm}{\mathbf{m}} \newcommand{\Bn}{\mathbf{n}} \newcommand{\Bo}{\mathbf{o}} \newcommand{\Bp}{\mathbf{p}} \newcommand{\Bq}{\mathbf{q}} \newcommand{\Br}{\mathbf{r}} \newcommand{\Bs}{\mathbf{s}} \newcommand{\Bt}{\mathbf{t}} \newcommand{\Bu}{\mathbf{u}} \newcommand{\Bv}{\mathbf{v}} \newcommand{\Bw}{\mathbf{w}} \newcommand{\Bx}{\mathbf{x}} \newcommand{\By}{\mathbf{y}} \newcommand{\Bz}{\mathbf{z}} \newcommand{\Balpha}{\boldsymbol{\alpha}} \newcommand{\Bmu}{\boldsymbol{\mu}} \newcommand{\Btheta}{\boldsymbol{\theta}} \newcommand{\Beta}{\boldsymbol{\eta}} \newcommand{\Bepsilon}{\boldsymbol{\varepsilon}} \newcommand{\Bbeta}{\boldsymbol{\beta}} \newcommand{\BLambda}{\boldsymbol{\Lambda}} \newcommand{\BSigma}{\boldsymbol{\Sigma}} \newcommand{\BOmega}{\boldsymbol{\Omega}} \newcommand{\BPsi}{\boldsymbol{\Psi}} \newcommand{\BGamma}{\boldsymbol{\Gamma}} \newcommand{\BPhi}{\boldsymbol{\Phi}} \newcommand{\Bphi}{\boldsymbol{\phi}} \newcommand{\BUpsilon}{\boldsymbol{\Upsilon}} \newcommand{\BXi}{\boldsymbol{\Xi}} \newcommand{\Bpi}{\boldsymbol{\pi}} \newcommand{\Bzero}{\mathbf{0}} \newcommand{\Bone}{\mathbf{1}} \newcommand{\N}{\text{N}} \newcommand{\dd}{\text{d}} $$

Case Study 5

Ed Iversen and John Zito

Duke University
STA 440 Spring 2026

The journey so far…

  • Case 1: particle adhesion
    • GLMs, linear mixed models
  • Case 2: macroeconomic revisions
    • time series, probabilistic prediction
  • Case 3: hormonal etiology of meningioma
    • Bayesian model averaging
  • Case 4: Beijing air pollution
    • splines, generalized additive models

And Now for Something Completely Different

Case 5: statistics in the courtroom

  • You will pretend that you have been hired as an expert witness in a court case;
  • In teams, you will conduct an analysis and write an expert witness report for a non-technical audience (judge, jury, lawyers);
  • Instead of giving a slide presentation, you will then “take the stand” and testify about your report.

Expert witnesses in general

  • The American system is adversarial;
  • There is prosecution and defense, and each may hire their own experts;
  • While one side or the other hires the expert, they are not supposed to be an advocate for that side;
  • A lawyer hires an expert to conduct a neutral analysis:
    • If the analysis doesn’t help their case, they bury it and hire someone else;
    • If it does help, the expert might be deposed or testify at trial (very lucrative);
  • The expert is subjected to questions under oath about their background, credentials, report, and the report of other expert witnesses;
  • If the expert testifies, everything they did is discoverable. So, if they conduct two analyses, one incriminating and one exculpatory, but then bury one…that’s bad.

Expert witnesses in general

Expert witnesses in general

Statistical experts in particular

A statistician (often a PhD academic, but not always) might be hired to…

  • explain statistical concepts to the non-expert jury;
  • respond to opposing experts;
  • evaluate statistical evidence or analysis at issue in the litigation;
  • quantify uncertainty, estimate likelihood, evaluate causality;
  • estimate damages.

An expert being deposed

Statistician Anthony Hayter enduring a wearying line of questioning about the relevance of his expertise:

Other very amusing depositions

(These have nothing to do with anything. They just tickle me.)

Case 5

The case

In 2012, the Obama Justice Department charged the sheriff of Alamance County NC with discriminating against Latino drivers at traffic stops.

Statistical experts were hired to assess the strength of evidence for three kinds of targeting:

  • The sheriff’s office placed checkpoints at sites more likely to encounter Latinos;
  • Checkpoint screenings led to targeted arrests of Latinos;
  • Latinos were more likely to be stopped for minor violations in non-checkpoint situations.

Data sources include Alamance County Sheriff’s Office, the U.S. Census Bureau, and the North Carolina state government.

Your task: phase one

Your client has sent you a horrific pile of files containing the traffic stop records of the sheriff’s office from 2009 - 2012:

  • Mop up the mess;
  • Write an expert witness report;
  • Reach whatever conclusion seems correct to you.

Judge, jury, and attorneys are the audience. Not statisticians.

Your task: phase two

After you submit the report, we will review it, and then your team will “take the stand” for 25 minutes:

  • An “attorney” sympathetic to your conclusion will examine you;
  • An “attorney” hostile to your conclusion will cross-examine you.

The “attorneys” will come from Duke’s Mock Trial team.

Timeline (Section 1)

This case is not about technique, so no content lectures. Furthermore, we’re asking you to write an airtight report on an accelerated timeline. So:

  • Friday April 3: lab as usual;
  • Tuesday April 7: in-class work period;
  • Thursday April 9: Q&A with Dr. David Banks, who has been an expert witness;
  • Friday April 10: lab as usual;
  • Monday April 13: final report due;
  • Tuesday April 14: intro Case 6, in-class work period;
  • Thursday April 16: Presentation (3 teams);
  • Friday April 17: Presentation (3 teams);
  • Tuesday April 21: Presentation (3 teams).

Timeline (Section 2)

This case is not about technique, so no content lectures. Furthermore, we’re asking you to write an airtight report on an accelerated timeline. So:

  • Friday April 3: lab as usual;
  • Monday April 6: in-class work period;
  • Wednesday April 8: Q&A with Dr. David Banks, who has been an expert witness;
  • Friday April 10: lab as usual;
  • Monday April 13: intro Case 6, in-class work period and final report due;
  • Wednesday April 15: Presentation (3 teams);
  • Friday April 17: Presentation (3 teams);
  • Monday April 20: Presentation (3 teams).

Words of caution

  • This is not about technique. Your audience (judge, jury, and lawyers) know virtually nothing about statistics. Stick to simple things you can actually explain!
  • You may not have written every part of your team’s report, but your name is on it. Be prepared to explain and defend everything in it. If the question is to you, a teammate can’t bail you out;
  • When you’re on the stand, the only resource you can consult is a paper copy of your report;
  • You are a statistical expert witness. Don’t opine about things beyond your expertise;
  • You wouldn’t arrive late to court, would you?
  • You wouldn’t wear sweatpants to court, would you?

Suggested reading

ASA Ethical Guidelines for Statistical Practice

Shared Values — Ethical Conduct

  • The social value of their work and the consequences of how well or poorly it is performed. This includes respect for the life, liberty, dignity, and property of other people.
  • The avoidance of any tendency to slant statistical work toward predetermined outcomes.
  • Statistics as a science. The field is evolving. There are established practices as well as unresolved issues.

Shared Values — Ethical Conduct, Ctd

Shared Values — Good Citizenship

  • Collegiality and civility with fellow professionals.
  • Support for improved public understanding of and respect for statistics.
  • Support for sound statistical practice, especially when it is unfairly criticized.
  • Exposure of dishonest or incompetent uses of statistics.
  • Service to one’s profession as a statistical editor, reviewer, or association official and as an active participant in ethical review panels.

Ethical Guidelines — Professionalism

  • Strive for relevance in statistical analyses. Base work on a competent understanding of the subject-matter issues, statistical protocols and technical criteria.
  • Guard against the possibility that a predisposition by investigators or data providers might predetermine the analytic result. Employ data selection or sampling methods and analytic approaches designed to ensure valid analyses.
  • Remain current in dynamically evolving statistical methodology.

Ethical Guidelines — Professionalism, Ctd

  • Ensure that adequate statistical and subject-matter expertise are both applied to any planned study.
  • Use only statistical methodologies suitable to the data and to obtaining valid results. Address potential confounding factors in observational studies and use caution in drawing causal inferences.
  • Do not join a research project unless you can expect to achieve valid results and you are confident that your name will not be associated with the project or resulting publications without your explicit consent.

Ethical Guidelines — Professionalism, Ctd

  • A procedure being automated does not ensure correctness or appropriateness; it is necessary to understand the theory, data, and methods used in each statistical study.
  • Recognize that any frequentist statistical test has a random chance of indicating significance when it is not really present. Multiple comparisons, p-value fishing.
  • Disclose conflicts of interest.
  • Provide only such expert testimony as you would be willing to have peer reviewed.

Ethical Guidelines — Responsibilities to Others

  • Where appropriate, present a client or employer with choices among valid alternative statistical approaches that may vary in scope, cost, or precision.
  • Clearly state your statistical qualifications and experience relevant to your work.
  • Clarify the respective roles of different participants in studies to be undertaken.
  • Explain any expected adverse consequences of failure to follow through on an agreed-upon sampling or analytic plan.

Ethical Guidelines — Responsibility to Others, Ctd

  • Apply statistical sampling and analysis procedures scientifically, without predetermining the outcome.
  • Make new statistical knowledge widely available to benefit society beyond your own applications.
  • Guard privileged information of the employer, client, or funder.
  • Fulfill all commitments.
  • Accept full responsibility for your professional performance.

Ethical Guidelines — Publication and Testimony

  • Maintain personal responsibility for all work bearing your name; avoid work you would not want to acknowledge responsibility for.
  • Report statistical and substantive assumptions made in the study.
  • In publications or testimony, identify who is responsible for the statistical work if it would not otherwise be apparent.

Ethical Guidelines — Publication and Testimony, Ctd

  • Make clear the basis for authorship order if determined on grounds other than intellectual contribution.
  • Account for all data considered in a study and explain the sample(s) actually used.
  • Report the sources and assessed adequacy of the data.
  • Report the data cleaning and screening procedures used, including any imputation.

Ethical Guidelines — Publications and Testimony, Ctd

  • Address the suitability of analytic methods and their assumptions relative to the study; identify computer routines used.
  • Where appropriate, address potential confounding variables not included in the study.
  • Identify the ultimate financial sponsor of the study, the stated purpose, and intended use of results.
  • When reporting analyses of volunteer or non-representative data, include appropriate disclaimers.

Ethical Guidelines — Publications and Testimony, Ctd

  • Report the limits of statistical inference and possible sources of error.
  • Share data used in published studies to aid peer review and replication while protecting confidential data.
  • As appropriate, promptly and publicly correct any errors discovered after publication.
  • Write with consideration of the intended audience.

Ethical Guidelines — Research Subjects

  • Know and adhere to rules for the protection of human subjects.
  • Avoid excessive or inadequate numbers of research subjects using informed recommendations for study size.
  • Avoid excessive risk to research subjects and imposition on their time and privacy.

Ethical Guidelines — Research Subjects, Ctd

  • Protect the privacy and confidentiality of research subjects and their data.
  • Be aware of legal limitations on privacy and confidentiality assurances.
  • Before participating in a study or reviewing related manuscripts, consider whether appropriate research subject approvals were obtained.

Ethical Guidelines — Research Subjects, Ctd

  • Avoid or minimize the use of deception; ensure independent ethical review when necessary.
  • When full disclosure is not advisable, inform participants of the nature of withheld information and the reason.
  • Adhere to animal welfare guidelines in research involving animals.

Ethical Guidelines — Research Team & Colleagues

  • Inform colleagues from other disciplines about relevant aspects of statistical ethics.
  • Promote effective and efficient use of statistics by the research team.
  • Respect the ethical obligations of members of other disciplines.
  • Ensure professional reporting of statistical design and analysis.
  • Avoid compromising statistical validity for expediency, while using reasonable approximations when appropriate.

Ethical Guidelines — Other Statisticians

  • Promote sharing of nonproprietary data and methods.
  • Strengthen the work of others through appropriate peer review.
  • Assess methods, not individuals.
  • Respect differences of opinion.

Ethical Guidelines — Other Statisticians, Ctd

  • Instill in students an appreciation for the practical value of statistical concepts and methods.
  • Use professional qualifications and contributions as the basis for professional decisions.
  • Avoid harassment or discrimination on professionally irrelevant bases.

The Sally Clark case

Background

  • Sally Clark (1964 - 2007): english solicitor;
  • 1996: first son died in bed at 11 weeks;
  • 1998: second son died in bed at 8 weeks;
  • Clark arrested on suspicion of murder;
  • Defense claimed it was SIDS (sudden infant death syndrome).

Expert testimony

  • Sir Roy Meadow (1933 - );
  • Pediatrician specializing in “Munchausen syndrome by proxy;”
  • Gave testimony that there is a “1 in 73,000,000” chance of two SIDS deaths in the same family;
  • A second expert was not engaged to push back on this.

1 in 73,000,000?

Behold:

  • In the 1990s, the incidence of SIDS in the UK was approximately 1 in 8,500 in middle-class families with no known risk factors;
  • Meadows assumed the two death were independent and just squared that.

Any problems?

The prosecutor’s fallacy

Even if you accept the number (which you shouldn’t):

saying that there is a 1 in 73 million chance that the babies died of SIDS is not the same as saying that there is a 1 in 73 million chance that the mother did not kill them.

\[ P(evidence \mid innocence) \neq P(innocence\mid evidence). \]

Why not?

The prosecutor’s fallacy

A sad story

  • 1999: Clark convicted.
  • Stats community loudly panned Meadow’s testimony;
  • 2003: conviction overturned on appeal after revelation that exculpatory evidence was suppressed;
  • 2007: Clark drinks herself to death.

The Daniela Poggiali case

Background

  • Daniela Poggiali (DP) was a nurse at ‘Umberto I’ hospital in Lugo Italy from 9 April 2012 to 8 April 2014;
  • On 12 March 2014, 95-year-old Massimo Montanari (MM) dies;
  • On 8 April 2014, 78-year-old patient Rosa Calderoni (RC) dies;
  • DP is accused of killing RC by lethal does of KCl;
  • DP sentenced to life on 11 March 2016, but then acquitted, re-tried, acquitted, and re-tried again;
  • DP is separately tried on 15 December 2020 for killing MM;
  • Several more rounds of acquittal and re-trial follow;
  • She is acquitted once and for all on 24 October 2021.

Suffice it to say…there was some disagreement here.

The hospital ward where DP worked

Specialized in seriously ill patients. Most days have at least one death.

From the prosecution’s experts:

  • If a nurse is working in sector A while a death occurs in sector B, the death is desginated as being “in the same zone.”
  • A death in C while she works in A is called “in the opposite zone.”

The prosecution’s case

All circumstantial. No one actually saw DP murder anyone.

However:

  • She was known to complain about the two patients;
  • There was an incident of KCl going missing and turning up in a weird place;
  • Using data on hospital shift assignments and patient mortality, the prosecution’s expert witnesses argued that a disproportionate number of patients died while DP was on duty, and this could not be mere coincidence.

Several triers of fact were clearly convinced.

The prosecution’s expert testimony

Are you convinced by this evidence?

The defense’s expert testimony

Just poke a million holes:

  • The prosecution’s conclusions are sensitive to the choice of grouping of hospital sections;
  • Observational data: association does not imply causation;
  • Prosecutor’s fallacy and base rate fallacy;
  • Misinterpretation of the \(p\)-value as a posterior probability of guilt/innocence given data;
  • Most of the deaths on DP’s watch are recorded in broad daylight when multiple other nurses are working.

The defense’s expert testimony

The patients are already old, frail, seriously ill

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

The defense’s expert testimony

Bayes for dayes

Can the available evidence distinguish:

\[ \begin{aligned} H_d:\, & \text{misconduct did not take place}\\ H_p:\,& \text{misconduct did take place}. \end{aligned} \]

The prosecution’s experts seem to inappropriately interpret a \(p\)-value as \(P(H_d\mid E)\). Not so. Let’s do it right:

\[ \frac{P(H_p\mid E)}{P(H_d\mid E)}=\frac{P(H_p)}{P(H_d)}\times \frac{P(E\mid H_p)}{P(E\mid H_d)}. \]

Don’t commit the base rate fallacy!

What are the prior probabilities? Really f%$#ing low:

  • Grant the prosecution’s premise that \(P(E\mid H_p)\) is 0.99;
  • There were 280,000 nurses in Italy in 2014, and you could count on one hand the number of cases where a nurse murdered patients and there was concrete evidence of guilt. The defense conservatively estimates that \(P(H_p)=0.000001\)

The defense’s expert testimony

Logistic regression

Each observation is a patient, and the response variable is whether or not they died during their hospital stay. Predictors are:

  • Sector (categorical): which sector was the patient in?
  • Times (numerical): how many times during their stay was DP on duty in that sector;
  • Age (numerical): the patient’s age;
  • Present (dummy): 1 if a nurse works in the same “zone” when death is recorded, else 0.

The defense’s expert testimony

Takeaways

  • Common mistakes repeated over and over: correlation \(\neq\) causation, base rate fallacy, misinterpreting \(p\)-values;
  • An opposing expert need not prove their side’s case. Just poke a million holes in the other expert’s testimony;
  • For a non-technical audience like judge and jury, we’re basically back to 199 tools: plots, tables, and simple regressions.