Automatic for the people – and their personal case narratives

Technology / 10 May 2019

UMC data scientist Eva-Lisa Meldau presents the de-identification research at the Data Innovation Summit in Stockholm, March 2019. Photo: Data Innovation @datasweden2019

Deep learning shows promise removing personal identifiers from case reports, according to new work presented by UMC researchers.


Case narratives can be a valuable source of insight about adverse drug reactions, potentially adding details that are less readily available – or not available at all – from the more structured, standardised data fields of individual case safety reports. But sharing case narratives raises legal and ethical duties to protect the confidentiality of patient’s private details.

Currently, removing personal identifiers from case narratives is a manual task that can be both time consuming and tedious. However, new work from UMC suggests there may be a better way.

At the Data Innovation Summit 2019, held in Stockholm in March, UMC data scientist Eva-Lisa Meldau discussed the research into automatically de-identifying case narratives to protect patient privacy. 

The UMC researchers trained neural networks with more than 500 medical records, developing a deep learning algorithm that “read” the narratives in multiple ways to predict identifying data, assisted by standard natural language processing tools and manually constructed rules and dictionary queries.

“We developed the system to be conservative in a way that it only retains words in the text if it is highly confident that they are not personal identifiers,” said Meldau.

The results have been encouraging, suggesting that the algorithm could be trained to perform as well as or possibly better than a human annotator, albeit at the expense of removing more non-personal text.

The algorithm has so far only been developed using medical records from a de-identification challenge. “As we fine-tune the algorithm with annotated original narratives, we expect the performance to improve even further,” said Meldau.

“Automatic detection  and removal of personal identifiers in case  narratives using deep learning” is available at bit.ly/umc-posters.

“Automatic detection and removal of personal identifiers in case narratives using deep learning” is available at bit.ly/umc-posters.

You may also like


The power of going digital: HSA ADR News Bulletin now reaches more readers

To reach more readers, the HSA undertook a digitalisation project of their news bulletin for communicating signal data. The results reveal the impact of going digital.

Technology / 17 May 2023

VigiFlow eReporting for Industry increases the ‘flow’ of communication between industry and regulators

Since implementing this module into its safety reporting process, INVIMA has seen marked improvements in the quality, quantity, and timeliness of reports sent to VigiBase.

Technology / 14 February 2024

How do we ensure trust in AI/ML when used in pharmacovigilance?

Ensuring trust in AI is paramount to fully reap the benefits of the technology in pharmacovigilance. Yet, how do we do so while grappling with its ever-growing complexity?

Technology / 26 February 2024

Our website uses cookies

Cookies are small text files held on your computer. They allow us to give you the best browsing experience possible and mean that we can understand how you use our site. Some cookies have already been set. You can delete and block cookies but parts of our site won't work without them. By using our website you accept our use of cookies.

Find out more