A certified de-identification system for all clinical text documents for information extraction at scale

Lakshmi Radhakrishnan; Gundolf Schenk; Kathleen Muenzen; Boris Oskotsky; Habibeh Ashouri Choshali; Thomas Plunkett; Sharat Israni; Atul J Butte

doi:10.1093/jamiaopen/ooad045

A certified de-identification system for all clinical text documents for information extraction at scale

JAMIA Open. 2023 Jul 4;6(3):ooad045. doi: 10.1093/jamiaopen/ooad045. eCollection 2023 Oct.

Authors

Lakshmi Radhakrishnan¹, Gundolf Schenk², Kathleen Muenzen², Boris Oskotsky², Habibeh Ashouri Choshali², Thomas Plunkett³, Sharat Israni², Atul J Butte^{2

4

5}

Affiliations

¹ Academic Research Services, Information Technology, University of California, San Francisco, San Francisco, California, USA.
² Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, California, USA.
³ ArcherHall LLC, Sacramento, California, USA.
⁴ Department of Pediatrics, University of California, San Francisco, San Francisco, California, USA.
⁵ Center for Data-Driven Insights and Innovation, University of California Health, Oakland, California, USA.

Abstract

Objectives: Clinical notes are a veritable treasure trove of information on a patient's disease progression, medical history, and treatment plans, yet are locked in secured databases accessible for research only after extensive ethics review. Removing personally identifying and protected health information (PII/PHI) from the records can reduce the need for additional Institutional Review Boards (IRB) reviews. In this project, our goals were to: (1) develop a robust and scalable clinical text de-identification pipeline that is compliant with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule for de-identification standards and (2) share routinely updated de-identified clinical notes with researchers.

Materials and methods: Building on our open-source de-identification software called Philter, we added features to: (1) make the algorithm and the de-identified data HIPAA compliant, which also implies type 2 error-free redaction, as certified via external audit; (2) reduce over-redaction errors; and (3) normalize and shift date PHI. We also established a streamlined de-identification pipeline using MongoDB to automatically extract clinical notes and provide truly de-identified notes to researchers with periodic monthly refreshes at our institution.

Results: To the best of our knowledge, the Philter V1.0 pipeline is currently the first and only certified, de-identified redaction pipeline that makes clinical notes available to researchers for nonhuman subjects' research, without further IRB approval needed. To date, we have made over 130 million certified de-identified clinical notes available to over 600 UCSF researchers. These notes were collected over the past 40 years, and represent data from 2757016 UCSF patients.

Keywords: Philter; clinical note text; de-identification; unstructured data.