Failure to Measure: Identifying the Limits of a Critical Patient Safety Metric in Neurosurgery

From the 2022 HVPA National Conference

Emily Rodriguez BS (The Johns Hopkins Hospital), Tej Azad MD, MS, Divyaansh Raj BA, Yuanxuan Xia MD, Joshua Materi BA, Jordina Rincon-Torroella MD, L. Fernando Gonzalez MD, Jose Suarez MD, Rafael Tamargo MD, Henry Brem MD, Elliott Haut MD, PhD, Chetan Bettegowda MD, PhD

Background

Failure to rescue (FTR) is a framework developed to capture a hospital’s ability to identify and manage preventable complications to prevent death. Patient safety indicator (PSI) 04 is a quality index designed by the Agency for Healthcare Research and Quality (AHRQ) to measure FTR resulting in post-operative mortality.

Objective

To determine whether the PSI 04 measure accurately captures FTR in the neurosurgical population.

Methods

We conducted a single-center retrospective cohort study. We identified patients from 1/12/2017-6/1/2021 who sustained a PSI 04 attributed complication (pneumonia, deep venous thrombosis/pulmonary embolism (DVT/PE), sepsis, shock/cardiac arrest, or gastrointestinal (GI) hemorrhage/acute ulcer), undergoing a neurosurgical procedure, and had inpatient mortality. The primary outcome was whether the attributed PSI 04 was the primary driver of mortality.

Results

We identified 67 patients who met PSI 04 criteria (median age, 61; female sex, 43.4%). The distribution of PSI attribution was pneumonia (34.3%), shock/cardiac arrest (31.3%), sepsis (26.9%), DVT/PE (6%), and GI hemorrhage/acute ulcer (1.5%). Patients who underwent emergent bedside procedures were more likely to present with poor GCS (P = 0.016), were more likely to be intubated prior to admission (P = 0.016), and were less likely to have mortality due to a PSI 04-related complication (P = 0.002). Approximately 20% of patients met the PSI complication criteria prior to admission, and a PSI 04-related complication was identified as the cause of death in only 43.2% of cases (emergency department, 28.6%; inter-hospital transfer, 53.3%; elective, 55.6%). In a multi-variable model, procedures occurring in the interventional radiology suite (OR, 23.2; 95% CI, 3.5-229.3; P = 0.003) or the operating room (OR, 6.2; 95% CI, 1.25-39.5; P = 0.03) were more likely to have mortality due to a PSI 04-related complication compared to procedures at the bedside.

Conclusion

Most neurosurgical procedures meeting PSI 04 criteria took place outside of a traditional operating room and may capture a fundamentally different patient population. Nearly one in five patients met PSI 04 criteria prior to admission and less than 50% of all patients had a cause of death that was attributable to their PSI 04 category. In our study, 65.7% of patients were inappropriately categorized as having a PSI 04 complication, suggesting the criteria need to be re-examined in the neurosurgical population. PSI-04 does not accurately measure the intention of the failure to rescue concept in neurosurgical patients.

Clinical Implications

This study identifies limitations of a key quality metric that is meant to improve patient safety outcomes and is employed by all surgical specialties. We found more than 50% of patients had a primary cause of mortality that was not their PSI 04 designation, thus erroneously capturing FTR. FTR is a vital concept in improving patient outcomes, and the ability to properly measure FTR is critical to mitigating complications, especially fatal complications, in post-surgical patients. We identify a clear need to modify the PSI 04 metric to achieve its proposed objective of improving patient safety.

What are academic medical centers across the country doing to improve healthcare value?

Value improvement guides: Published reviews in JAMA Internal Medicine coauthored by experienced faculty from multiple leading medical centers, with safety outcomes data and an implementation blue print.

Review article detailing 25 labs to refine for high value quality improvement | July 2020

MAVEN campaign: Free 4 year high value care curriculum online.

Join the Alliance! Membership is free with institutional approval and commitment to improving value in your medical center.

Learn more about HVPA on Health Affairs Blog