Summary
- “Without data, you don’t know if you have a problem, you don't know if you’re making any headway in solving that problem, and you don't know whether the interventions that you’re trying to test or implement are holding.”
When it comes to measurement, there is no one-size-fits-all approach; it’s all a matter of context.
That’s a key takeaway for Michael A. Posencheg, MD, Associate Chief for Clinical Affairs, Division of Neonatology and Medical Director of Clinical Effectiveness at the Children’s Hospital of Philadelphia.
For teams who may feel intimidated by the prospect of measurement, Posencheg offers a helpful concept: “just enough” data. “Teams sometimes feel like they need to have all of the data to make a decision about something,” he said in a recent interview. “I think many teams get into this ‘analysis paralysis,’ where all they want to do is look at all the available data before making a change.” The impulse to be comprehensive, while understandable, can needlessly stall progress.
In many cases, as a team begins testing improvement, all they need is a basic sense of whether the changes they are implementing are leading to any degree of improvement, said Posencheg. They need “just enough data to make the next decision or move the project along.” For example, if you’re running a very small test of change, you could try it one time to start. “You’re trying something new on one patient or on one day, you may find that just having a qualitative experience of that event is enough data to get a sense of whether you should do more testing,” Posencheg explained.
Similarly, in the right context, sampling can be very helpful. With process measure data, when a team is trying to determine whether a new change idea is taking hold, sampling can provide a rough sense of how frequently it is occurring. “You don’t need to differentiate 75 percent [compliance] from 77 percent because it’s not that important,” said Posencheg. “You just want to get a sense of the degree of magnitude.”
Learn to Coach Your Team to Success
That said, in some cases, more data will be necessary, and sampling is not always appropriate. Again, it all depends on context.
More data is called for if you are further along in the process — if you have already conducted multiple Plan-Do-Study-Act (PDSA) cycles around a given change idea, and you are close to moving from testing to implementation, for example. Additional data will provide reassurance, said Posencheg.
Sampling is also not appropriate for rare outcomes or an outcome measure that is particularly important. “In those contexts, you want to get every patient’s outcome,” Posencheg explained. Central line-associated bloodstream infections or catheter-associated urinary tract infections are two examples. “These are rare enough events that you should be able to collect data on every occurrence,” he noted.
No Quality Without Equity
Equity has been a neglected aspect of quality improvement measurement. Increasingly, health systems are collecting and using REaL (race, ethnicity, and language) data to identify inequities, help set priorities, and drive improvement. “There is no quality without equity,” Posencheg said. “One of the things that we’ve tried to do at our institution is try to incorporate equity into all of our projects.”
For example, they have found racial disparities in their heart failure patients. “No one thinks that they treat patients differently,” Posencheg noted, “but when you look at the data, you can see that there are reasons, whether it’s unconscious or conscious, that different patients get different interventions.” According to Posencheg, stratifying data and finding inequities can be uncomfortable but also transformative. “When you look, you find gaps,” he said. “And when you find gaps, you have a real drive and an obligation to strive for improvement.”
The Cornerstone of Quality Improvement
Once data is collected, there are revelatory possibilities. “My true passion is teaching measurement and control charts and run charts,” said Posencheg. Looking at data over time, he believes, is “so much more powerful in telling a story than doing a pre- and post-analysis.” He is gratified that, over the past 10 or 15 years, the field of improvement has changed to account for this. “I think it just shows how the field is adapting and is more mature than it was before,” he said.
Ultimately, improvement is not possible without measurement. Measurement “is the cornerstone of quality improvement,” Posencheg said. “It answers the second question in the Model for Improvement, which is, how will you know the change is an improvement?” he noted.
Posencheg recalled a lesson he learned when he was a new medical director in an intensive care unit. A new senior faculty member told him the premature babies being admitted to his unit had a low admission temperature. Low admission temperatures are associated with poor outcomes in premature babies. “When she brought it to my attention, I didn’t believe her,” he recounted.
Posencheg decided to review admission temperature data for the previous two years. He found that the faculty member was right. More than 50 percent of the babies had a low admission temperature. “I was really surprised at the magnitude of that,” he recalled. The data became a clarion call for his unit. The team started an improvement project. As a result, more than 95 percent of the babies now being admitted to the unit have a normal body temperature.
“Without data, you don’t know if you have a problem, you don't know if you’re making any headway in solving that problem, and you don't know whether the interventions that you’re trying to test or implement are holding,” said Posencheg. “For all of those reasons, measurement is at the crux of all quality improvement efforts.”
Photo by Rob McGlade | Unsplash
You may also be interested in:
“What is the Cost of Failure?” and Other Questions Your Team Should Ask