Use of LLMs in Research

Large Language Models (LLMs) are artificial intelligence systems trained on vast collections of text, to learn the patterns of human language. They generate text and can be used to summarise, answer questions, and interpret clinical notes. Because of the variety of text they are trained on, they can adapt to new queries and provide human-like responses rather than being restricted by fixed rules – but their outputs can reflect biases and inaccuracies in the data going into them. We are exploring how they may assist by looking at large sets of data that would take humans many hours or days to read, working with humans as part of a team to improve safety in healthcare.

How will LLMs be used in our research?

Patient safety management and electronic patient record (EPR) systems contain a large amount of free-text data, which provides valuable insights into patient safety and care. However, analysis of this data often relies on staff manually searching records, extracting data and conducting analysis, limiting the ability to improve the safety of care.

This programme of research is looking to see if it is possible for large language models to support this work by automating parts of the process to support the analysis of the free text within patient records. To understand whether the large language model produces accurate and suitable results, its outputs will be reviewed by a number of healthcare and patient safety stakeholders. This will be captured using 1-2-1 interviews and focus groups (depending on the stakeholder).

Open-source LLMs that are held within the hospital’s IT system and are not connected to the internet will be used to perform the research. Patient data will not be used to train the models. This means that your data will be kept secure at all times during the research, and there is no chance that the LLM will reproduce your information.

What patient safety analysis tasks will we investigate?

The models will be used across four patient safety areas:

• Generation of discharge summaries from EPRs.
• Analysis of EPRs for evidence of patient safety concepts, like preventable harm and the adaptations staff make to avoid this harm.
• Supporting HSSIB safety investigations in two areas:
  • Using EPRs to assist in generating incident reports.
  • Analysing PSIRF reports and incident records to assess the quality of the investigations.

What data will we analyse?

During this research, we will analyse three data sources using the large language models:

• Electronic patient records
• Incident records
• HSSIB safety investigation reports

What approvals and permissions do we have for the research?

This research has received ethical approval from Leeds East NHS REC and the Confidentiality Advisory Group. The research has the support of BTHFT, including the support of the Trust’s Caldicott Guardian.

How can I get in contact if I would like to know more, or opt-out from the research?

If you would like to opt-out from the research or would like more information about how we will process and store your data securely, please contact the study team at the details below.

Prof Tom Lawton, Consultant in Anaesthesia and Intensive Care BTHFT

Email: SI.optout@bthft.nhs.uk

Phone: 01274 383952