Mobile menu icon

Citizens’ Juries

In February 2019, the NIHR Greater Manchester Patient Safety Translational Research Centre (NIHR Greater Manchester PSTRC) and the Information Commissioner’s Office (ICO) co-funded two Citizens’ Juries, the first in Coventry and the second in Manchester.

What is a Citizens’ Jury?

A Citizens' Jury is a method of deliberation developed by the Jefferson Center, a US-based charity, where a small group of people (between 12 and 24), representative of the demographics of a given area, comes together to deliberate on a specific issue. Our set of two juries focus on the use of Artificial Intelligence (AI) to make decisions, across a number of different situations.

Citizens juries Coventry small group

Decision making and explainable Artificial Intelligence

AI is being used to train computers how to do many things, for example, making medical diagnoses. Scientific studies have shown that AI can diagnose skin cancer from skin images as reliably as dermatologists. But how do we know whether we can trust a diagnosis? If we are given it by a human doctor, we can ask for an explanation. However, the most advanced AI systems are very complex and therefore they may not be able to explain to humans how they reach their decisions. We could require that these systems are always able to give an explanation that people can understand, but doing so may come at the expense of less accurate decisions. So, there is a tradeoff between accuracy and ‘explainability’ of AI systems.

The set of two Citizens’ Juries consider this tradeoff in four different scenarios:

  • Healthcare: diagnosis of acute stroke
  • Healthcare: finding matches between kidney transplant donors and recipients
  • Criminal Justice: deciding which offenders should be referred to a rehabilitation programme
  • Recruitment: screening job applications and making shortlisting decisions

For more information, contact the Director of Citizens’ Juries c.i.c., Dr Malcolm Oswald at info@citizensjuries.org.


The following video was filmed at the citizens’ jury held at the Bridgewater Hall in Manchester.


Jury design documentation

No.NameBrief descriptionReviewed by Oversight Panel?File for download
A1 AI Citizens' jury specification A specification of the design for the citizens' juries, including the jury questions and juror selection criteria. Yes A1 AI citizens jury specification
A2 Brief for expert witnesses A brief provided to all five expert witnesses to guide their presentations to the juries, including specific brief for each named witness. Yes A2 Brief for expert witnesses 
A3 Citizens jury oversight panel brief A brief to the three members of the oversight panel (Soren Holm, Reema Patel, David Leslie) describing their role to monitor bias. Yes A3 Citizens jury oversight panel brief 
A4 Oversight panel signed questionnaires The set of forms comleted and signed by the three members of the oversight panel with their assessments and statements on bias. N/A A4 Oversight panel signed questionnaires 
A5 Jury recruitment questionnaire A copy of the online form completed by people applying to be Coventry jurors (an equivalent form was used for Manchester) Yes A5 Jury recruitment questionnaire 
A6 Start of jury questionnaire The questionnaire that all jurors completed at the start of day 1 of the jury process Yes A6 Start of jury questionnaire 
A7 End of jury questionnaire The questionnaire that all jurors completed at the end of day 5 of the jury process Yes A7 End of jury questionnaire 
A8 Daily bias questionnaire A form used to capture feedback from the jurors, particularly about potential bias, at the end of days 1, 2,3 and 4. No A8 Jefferson Center bias questionnaire 

 Jury materials

NoNameBrief descriptionReviewed by Oversight Panel?File for download
B1 Jurors ring binder contents A folder of materials produced by Citizens Juries c.i.c. and the Jefferson Center, printed out and provided in a ring binder to each jury member. It had 16 sections including a table of contents (section 0), and all the slides from the expert witnesses. Jefferson Center copyright material has been redacted from this downloadable version. All witness slides were reviewed B1 Jurors ring binder contents 
B2 Scenario handouts Paper handouts provided to jury members during the course of the five-day event, namely the four scenarios with associated questions that the juries considered. Yes B2 Scenario handouts 

Jury outputs

NoNameBrief descriptionFile for download
C1 Coventry jurors report The report of the jury conclusions produced by Kyle Bozentko of the Jefferson Center on day 5 of the Coventry jury with 18 jurors C1 Coventry Jurors report 
C2 Manchester jurors report The report of the jury conclusions produced by Kyle Bozentko of the Jefferson Center on day 5 of the Manchester jury with 18 jurors C2 Manchester Jurors report 
C3 AI citizens' juries management summary A two-page report summarising the findings of the two citizens' juries C3 AI citizens' juries management summary
C4 AI citizens' juries report The report of the design and findings of the two citizens' juries (with functioning hyperlinks to report sections) C4 AI citizens' juries report
C5 Juries word cloud A word cloud generated automatically using the answers given by the 36 jurors from the two juries in the end-of-jury questionnaire to the question "overall, what was it like participating in the citizens' jury over the five days? Please say 3 things in 3 words to sum up your experience." C5 Juries word cloud 
C6 Coventry start-of-jury questionnaire results A summary of the answers given for the questionnaire completed by the Coventry jurors at the start of day 1 C6 Coventry start-of-jury questionnaire results 
C7 Manchester start-of-jury questionnaire results A summary of the answers given for the questionnaire completed by the Manchester jurors at the start of day 1 C7 Manchester start-of-jury questionnaire results 
C8 Coventry scenario question results A summary of the answers given by the Coventry jury for the four scenarios on days 3 and 4 C8 Coventry scenario question results 
C9 Manchester scenario question results A summary of the answers given by the Manchester jury for the four scenarios on days 3 and 4 C9 Manchester scenario question results 
C10 Coventry general question results A summary of the answers given by the Coventry jury for the general questions considered on day 5 C10 Coventry general question results 
C11 Manchester general question results A summary of the answers given by the Manchester jury for the general questions considered on day 5 C11 Manchester general question results 
C12 Coventry daily bias questionnaire results A zip file of 4 spreadsheets of the results from the daily bias questionnaire completed by jurors at the end of days 1, 2, 3 and 4 of the Coventry jury (on day 5 the jury completed the end-of-jury questionnaire instead) C12 Coventry daily bias questionnaire results (zip file) 
C13 Manchester daily bias questionnaire results A zip file of 4 spreadsheets of the results from the daily bias questionnaire completed by jurors at the end of days 1, 2, 3 and 4 of the Manchester jury (on day 5 the jury completed the end-of-jury questionnaire instead) C13 Manchester daily bias questionnaire results (zip file) 
C14 Coventry end-of-jury questionnaire results A spreadsheet of the results from the end-of-jury questionnaire completed by Coventry jurors at the end of day 5 C14 Coventry end-of-jury questionnaire results 
C15 Manchester end-of-jury questionnaire results A spreadsheet of the results from the end-of-jury questionnaire completed by Manchester jurors at the end of day 5 C15 Manchester end-of-jury questionnaire results 

Notes:

  1. Other than where specified, the main author of the documentation above was Dr Malcolm Oswald, Director of Citizens Juries c.i.c.. Reviewers varied depending on the naure of the document.
  2. The Oversight Panel brief was to review the main jury design documentation, but not the jury outputs.

Blog posts

We have published a number of blog posts about these Citizens' Juries: