Patient Safety Event Reporting, also commonly referred to as incident reporting or occurrence reporting, is the system and process in which frontline staff document events that cause or have the potential to cause patient harm, allowing for analysis that can lead to improved care. Event reporting systems include third-party vendor applications, EHR-embedded tools, in-house built electronic tools, and manual paper-based processes.
Table of Contents
In 2018, American Data Network Patient Safety Organization (ADNPSO) conducted a deep-dive analysis of the event type “Other.” Over a four-year period, this catchall category accounted for over 40% of events reported by ADNPSO members. Among the myriad of striking revelations, the death rate of incidents categorized as Other was 3 times higher than ALL remaining event categories combined. The results of the Other analysis proved so compelling that ADNPSO was selected to present at the National Association for Healthcare Quality (NAHQ) Next Conference in November 2018. The findings underscore the valuable insight buried in the Other event category.
As part of its mission, ADNPSO conducts deep-dive analyses to continuously improve its understanding of the data and disseminate best practices to members.
Among patient safety events submitted through ADN’s Patient Safety Event Reporting Application, the event type “Other” continuously ranks as one of the most frequently reported. With aggregate data from April 2014 through May 2018, ADNPSO determined that Other accounted for over 40% of all the submitted patient safety events, a trend other PSOs around the country are also finding.
Other tends to serve as a catchall category for events that do not fit neatly into the types standardized by the Agency for Healthcare Research and Quality’s (AHRQ) Common Formats. Moreover, because there is no nationally-defined or uniform collection for Other events, the data captured is largely narrative, which makes it more difficult to analyze and may result in missed opportunities to mitigate risk and harm.
In the project, ADNPSO set out to further refine the management process for Other events to help members better identify and prioritize emerging subcategories, and to analyze associated vulnerabilities, improvement opportunities and sustainability efforts.
Patient safety incidents, near misses and unsafe conditions occur constantly throughout healthcare organizations, whether or not facilities know about all—or even most—of them. The Iceberg model (used for guiding systemic thinking in healthcare for years)1 is a strong visual to illustrate that what you see above the waterline is only the tip of the problem (in this case, patient safety incidents). However, what lies beneath the surface provides a much deeper understanding of the problem. For example, near misses and unsafe conditions can reveal fractured systems and processes leading to errors. Focusing primarily on incidents forces staff into a reactive posture, rushing to diagnose and fix what has already happened. Many organizations are striving for a more proactive focus by concentrating on near misses and unsafe conditions. Patterns and trends, what influences them, and assumptions about the systems in place are all important drivers in regards to patient safety.
ADNPSO flipped the Iceberg model upside down and created the Whirlwind Model2 in order to operationalize it. The Whirlwind Model illustrates the key functions, parameters, and compliance efforts that must be coordinated, collaborated, and communicated in order to deliver safe patient care.
There is no straight line from admission to discharge as patients are impacted by all the blowing whirlwinds; i.e., interactions with multiple healthcare professionals and navigation through thousands of process steps that create fertile ground for defects and errors. Organizations must embrace the complexity and execute well-designed, targeted and coordinated care to mitigate patient risk and harm. A first step is appropriately categorizing patient safety data and delving below the surface to reveal fractured systems and processes.
AHRQ Common Formats, created to standardize patient safety data collection, capture 3 top-level categories: incidents, near misses, and unsafe conditions. An incident is a safety event that reached the patient, regardless of harm. A near miss is an event that did not reach the patient. And an unsafe condition is a circumstance that increases the probability of a safety event.
AHRQ’s Common Formats define 8 standard event types in addition to Other:
ADNPSO’s deep-dive analysis of the 4-year period of data began with the large concentration of events in the Other category.
As the Other event type accounts for 43.56% of overall events, it’s critical to understand the data and to aggregate similar events wherever possible for learning purposes. ADN’s event reporting application also captures data about preventability. Strikingly, the analysis revealed that 62.12% of the Other events almost certainly or likely could have been prevented.
One of the most noteworthy findings from the ADNPSO analysis was that the mortality rate of incidents categorized as Other was 3 times higher than ALL remaining event types combined.
In 2014, ADNPSO conducted an extensive analysis of the catchall Other event type, identifying reporting trends and creating resultant subcategories in ADN’s event reporting application as a means for further segmentation. The list of 33 subcategories allows providers to better isolate and understand the types of events that do not fit neatly within the AHRQ Common Formats.
In reviewing the subcategory utilization between April 2014 to May 2018, Other Other accounted for 20.56% of the events followed by Specimens representing 16.03%. Using this to drive data collection priorities, ADNPSO carved Specimens out of the subcategory listing and designated it as an independent event type.
Of equal importance are the subcategories that do not have many events but should have higher volumes. It raises questions about the reporting culture or the staff’s understanding of what constitutes a safety event. This analysis can lead to the revelation of the need for more education around the definitions of the subcategories so frontline staff doesn’t default to the path of least resistance and use the Other Other subcategory.
A good example of how frontline staff’s perception of event categorization can affect your data is the subcategory Telemetry, which only accounted for 2.52% of all subcategories. Does staff not recognize Telemetry issues as a patient safety event? Or are those being reported but buried in the Other Other subcategory?
For Patient Safety and Risk Managers, it’s common to go after the easy targets of events that are High Volume. But Low Volume/High Risk should also be a regular target of focus.
When assessing areas that need improvement, a quality team’s efforts can be focused to maximize impact by using the Pareto Principle (also known as the 80/20 rule), which states that roughly 80% of effects result from 20% of the causes.
A notable application of this rule can be seen in analyzing events where the Final Harm Level is Death. Among all incidents reported to ADNPSO between April 2014 and May 2018, 83.05% of all deaths occurred in 3 event types, specifically Other, Healthcare Associated Infections, and Surgery or Anesthesia.
One of the most remarkable discoveries was revealed when comparing the Final Harm of Other incidents to that of the remaining 8 event types combined. Death among Other events was almost TRIPLE.
With that in mind, ADNPSO used the Pareto Rule to dig deeper and concluded that 81.81% of Other deaths were attributed to just 3 of the subcategories, specifically Code/Emergency, Unexpected Death, and Other Other.
Focusing on just 21.42% (3 of the 14) subcategories with deaths will likely deliver the highest return, which is always a best practice when deciding where to focus time and energy. Identifying areas for improvement using the Pareto Principle can often bring high ROI with just small changes.
To further dissect Other deaths, ADNPSO looked at these 3 subcategories by department, discovering that just five department types accounted for 84.45% of deaths. Evaluating death by time of day at the department levels is recommended to ascertain if a pattern emerges or to determine what unit-specific factors correlate with the times of death.
To validate the four-year findings, ADNPSO evaluated data for CY 2018. The goal was to confirm whether the same trends are reflected in the most recent data set and to expose any new insights. The team ran aggregate reports for CY 2018 and also selected 500 random Other Other events for manual, narrative reviews.
The comparison of the two time frames revealed that Other events remained the most frequently reported type in CY 2018, followed by Medication, Falls, and Surgery/Anesthesia. The subcategory analysis revealed no significant changes from the analysis of the first data set. The highest reported subcategories were Other, Specimens, Delay in Treatment and Failure to Follow Policy/Order.
Also, over 50% of incidents continue to be reported as preventable. While the majority of events did not result in harm, the trend of Other events having a 3X higher death rate than all other events held true.
Of the 500 random Other Other event narratives reviewed, ADNPSO recommended that 70% (or 346) should be recategorized to one of the more specific subcategories or event types. If these 346 events were recategorized as recommended, a shift would occur as to which patient safety subcategories top the list.
If your team is using the 80/20 Rule to prioritize time and resources, events that are miscategorized and hiding in Other may be skewing your data, meaning you may not actually be focusing on the top priorities. Risk Managers should review and appropriately categorize events prior to closing them to produce a more accurate set of priorities. ADNPSO also recommends using miscategorizations as an opportunity to educate frontline staff and curb future errors.
Strikingly, nearly all of the Other events that resulted in death were miscategorized. Of the 500 random Other Other events reviewed, 7 resulted in death; and 6 of the 7 warranted recategorization to more specific subcategories such as Delay in Treatment, Code/Emergency or Medications.
The goal of ADNPSO’s Other event analysis project was to determine what type of patient safety issues were captured in this catchall category, then use our findings to aggregate similar events for learning purposes, improve future data collection efforts and isolate key issues that should be evaluated at individual organizations.
References and Resources
Good Catch Initiative: 45 Hospitals Report 47% More Near Misses