Mitigating imperfect data validity in administrative data PSIs: a method for estimating true adverse event rates

Abstract

Question: Are there ways to mitigate the challenges associated with imperfect data validity in Patient Safety Indicator (PSI) report cards?

Findings: Applying a methodological framework on simulated PSI report card data, we compare the adjusted PSI rates of three hospitals with variable quality of data and coding. This framework combines (i) a measure of PSI rates using existing algorithms; (ii) a medical record review on a small random sample of charts to produce a measure of hospital-specific data validity and (iii) a simple Bayesian calculation to derive estimated true PSI rates. For example, the estimated true PSI rate, for a theoretical hospital with a moderately good quality of coding, could be three times as high as the measured rate (for example, 1.4% rather than 0.5%). For a theoretical hospital with relatively poor quality of coding, the difference could be 50-fold (for example, 5.0% rather than 0.1%).

Meaning: Combining a medical chart review on a limited number of medical charts at the hospital level creates an approach to producing health system report cards with estimates of true hospital-level adverse event rates.

Keywords: Administrative data; Adverse event; Bayesian adjustment; Patient safety; Patient safety indicators.